NewtheOG

Distributed system design

Are there any resources to learn distributed system design? I am comfortable coming up with highly available system design with AP properties. I struggled in interviews where consistency is important like CC payments, ticket reservation system. Looking for articles, blogs for microservices with ACID properties.

LinkedIn ChindiChor Aug 6, 2018

You trade-off between high availability and consistency. Strong consistency will require dual writes at data layer or reads going to leader/master partition to guarantee read-after-write symantics. This means that some clients requests may take longer to serve or becomes unavailable as system can handle fewer qps of traffic and have more failures points in the system. For example, you could use MySQL or Oracle db to store records. You can still partition into multiple dbs, but you lose availability since all writes and reads wait until they are propagated to slave db instances. MySQL and Oracle both have ACID properties by default. If you're using a non-acid db such as Cassandra, you'll have to ensure that each write is written to at least two places for durability, and reads are always sent to leader/master. Again you'll lose on availability in this case. You could also use a consensus algorithm such as paxos to drive reads and writes, again trading off availability for consistency. The primary trait an interviewer is looking for with such interviews is whether you're aware and considering the different trade-offs. Most interviewer know that you may not have "working" knowledge of everything, but having the necessary theoretical knowledge is important. Grokking the system design course (designgurus.org) talks about this in brief and also has an example solution for a ticket booking system.

Samsung wdQd17 Jun 26, 2019

Wait what? "... and reads are always sent to leader/master." I thought the whole point of replication is so that reads can happen from replicas. Writes happen on leader and atmost k slaves where k is generally <= 2

New
TC or GTF0 Aug 6, 2018

Are you familiar with the CAP theorem?

New
theOG OP Aug 6, 2018

I am. What i am looking for is examples of distributed system design with transactions in mind.

Microsoft 0to1 Oct 5, 2018

Look for scheduler agent supervisor pattern and compensating transaction pattern

Groupon howdu Aug 6, 2018

The book designing data intensive applications.

Lyft aztp81 Aug 6, 2018

Read grokking the system design in educative io