What are the must read papers in distributed systems/big data?

New / Eng sha512
Feb 4, 2018 14 Comments

Here are some classics:
- MaReduce paper
- BigTable paper
- Amazon DynamoDB paper
- Spark RDD paper

Papers from the comments:
- Cassandra
- Kafka

And ?

comments

Want to comment? LOG IN or SIGN UP
TOP 14 Comments
  • Microsoft kng
    There's a lot more to the field than just what's used in industry...

    Lamport: Time, clocks, and the ordering of events in a distributed system. The Byzantine Generals problem. Specifying systems, Paxos made simple

    Lampson: Hints for computer system design

    Gray: Transaction processing, Data Cube, The notions of consistency
    Feb 4, 2018 2
    • Amazon / Eng Code🐵
      Just to be clear, you have 4 papers listed for Lamport in your post. Might be useful to split them in new lines.
      Feb 4, 2018
    • Microsoft kng
      Yeah. If you ever get the chance of taking a seminar with Lamport (he does some at Amazon time to time) jump on it. I've been lucky enough to take two week long seminars with him and its great. He's truly a special mind. He thinks about problems completely differently than computer scientists do, probably the reason he's made so many important contributions
      Feb 4, 2018
  • Amazon / Eng Code🐵
    Start with basics (I don’t know the exact titles of the papers):
    1. CAP theorem by Eric Brewer
    2. Logical clocks by Leslie Lamport
    Feb 4, 2018 2
    • Microsoft kng
      Gilbert: Brewers conjecture and the feasibility of available partition tolerant web services
      Feb 4, 2018
    • New / Eng sha512
      OP
      Is this the original paper title?
      Feb 5, 2018
  • Microsoft fNEH83
    Paxos
    Feb 4, 2018 0
  • Apple chipotlefa
    Cassandra , Kafka.
    Feb 4, 2018 1
  • How can Roy Fielding’s REST paper not be in this list yet?!
    Feb 4, 2018 0
  • New Kasd
    Gfs, raft
    Feb 4, 2018 0
  • Microsoft / Eng ShortBTC
    Cops and cops+
    Feb 4, 2018 0
  • Microsoft / Eng 949
    Spanner paper, azure storage
    Feb 4, 2018 0
  • New / Eng sha512
    OP
    Looks like we are getting good suggestions here. Looking forward to more replies.
    Feb 4, 2018 0

Salary
Comparison

    Real time salary information from verified employees