Large scale system design experience in real world

Tango Health / Eng WaterColor
Jul 27, 2018 15 Comments

Reading about large scale system for an interview is one thing. There are a few good resources like grokking the system interview, hiredtech, designing data intensive applications, etc.

What if I have not worked on large scale problems in real world and would like to experiment/learn say building an app with something random as in here.

1M writes/sec
Spikes upto 3X during certain intervals
A search query
NoSQL data store
And some fairly intensive computation for every query

Can be anything - ecommerce, sports betting, stock trading apps, let's assume the requirements are almost the same across the board.

Is there a playground like leetcode? What is a good book/tutorial with an end to end scaling solution in a step by step approach? I'd like to build out the whole thing. Obviously Azure or AWS can help me stretch the scale, is there a better way to test the scaling and identify the bottlenecks in an "experimental playground" without high bucks. I'm ok to pay of the order of a monthly Leetcode subscription for this experiment.

Trolls please excuse.
TC: Around the Midpoint of 2^17 and 2^19

comments

Want to comment? LOG IN or SIGN UP
TOP 15 Comments
  • Amazon Fr00pid
    Is that the geometric or arithmetic midpoint of 2^17..19?
    Jul 28, 2018 4
    • Tango Health / Eng WaterColor
      OP
      I wish it was arithmetic ;)
      Jul 28, 2018
    • Microsoft compiler
      To save everyone's time
      Geometric = 256k, Arithmetic = 320k.
      Jul 28, 2018
    • Tango Health / Eng WaterColor
      OP
      @Msft, ur name checks out!
      Jul 28, 2018
    • Amazon Fr00pid
      @WaterColor how quaint and charming, like how back in the day “computer” meant clerical lady who manually calculated ballistic trajectories? Similarly, maybe a compiler could be another clerical lady who fetches and summarizes the results for you. In that way, I suppose the name checks out... in 1940.
      Jul 30, 2018
  • Intel riverdale
    That many reads and writes you are not gonna get for monthly leetcode price. My advice is to keep making more free tier accounts for AWS/Azure and building on those.
    Jul 27, 2018 3
    • Tango Health / Eng WaterColor
      OP
      Thanks, how much do I have to stretch my budget to build at that scale?
      Jul 27, 2018
    • Intel riverdale
      Depends on what exactly you’re building and how often it’s up and running. Most of aws is billed on usage.

      Just build your app, run tour tests and see how you do. Do you really need 1m simultaneous read writes? Or can you test with 10k, then 100k, and extrapolate performance based on that?
      Jul 27, 2018
    • Tango Health / Eng WaterColor
      OP
      Yes, a baby step approach will work. Never done a scaling out before. Where do I start?
      Jul 27, 2018
  • Splunk mAKi70
    Yeah I have been thinking about this and here is my plan 1) create a docker image of the code I want to deploy 2) get images of Cassandra 3) scale my app until o start hitting limits of container 4) then try using more container s of either my code or Cassandra to try scale horizontally if something fucks up try to come up with a a new plan
    Jul 27, 2018 1
    • Tango Health / Eng WaterColor
      OP
      Good start!
      Jul 28, 2018
  • Amazon LoanerBadg
    When your system does 1M writes, that's because it was built to scale horizontally. It's not a single host that is doing that much writes.
    So what you need to do is to get experience building systems that can scale horizontally.
    Jul 29, 2018 1
    • Tango Health / Eng WaterColor
      OP
      How and where do I start for a real world DIY project please?
      Jul 29, 2018
  • Amazon / Eng piora
    I don't see how 1M writes home lab will help with the large scale design. You often have to deal with less writes, but in a distributed micro services environment, with the goal of not having your micro-service becoming bottleneck, or page alarm generator. Unless you are writing a distributed database from scratch, then 1M writes is a great benchmark.
    Jul 28, 2018 1
    • Tango Health / Eng WaterColor
      OP
      Ahh okay. I just wanted to build out something. I cooked up the requirements - usually once I'm done satisfactorily in a system design question in terms of breadth - the follow up is how do we scale up to 1M writes, 50M customers, PB of data ..... Now you see, why I wanted to try building.:)

      The whole point is I want to learn how to scale out - and willing to do what it takes.
      Jul 28, 2018