I am back but I need help with system design

Oscar
🐨koala

Oscar

BIO
I am a Koala. DM me if you want to cuddle.
🐨koalamore
Sep 1 16 Comments

I said I wouldn’t come back until I get an offer lol but I really need help brainstorming. Interview in one week.

I am studying “rate limiter”. I know the various algorithms, but I am stuck with what exactly to discuss. Here is my plan:

1. Clarifying goals with the interviewers
2. High level of client -> proxy -> backend and establish I want to implement rate limiter at the proxy level (and explain why)
3. Discuss a couple rate limiter algorithms like leaky bucket
4. Then I might suggest a discussion on Little’s Law and AIMD.

Essentially on how to address overprovisioned limit - client A asked for 50 rps but 99.99% of the time using 10rps, but system has capacity of 100 rps and later client A bursts to 60 rps and could be allowed (?) and how to rate limit when latency and error rate increasez I think the latter case should be implemented using circuit breaker.

5. Finally I have to account for running rate limiter in a distributed environment (multiple proxies). I suppose I need to have a gloabl table to check current user’s rate limit. But remote network communication is expensive, so I choose to run a redis local to the proxy and periodically syncs to a global redis as a backup. This might just work if user lands on a proxy using consistent hashing with bounded load algorithm (google reseafch, available in haproxy) so they always land on the same proxy server.

Sorry for the long post. What you think? Really frustrated at myself right now. Any help is always appreicated.

comments

Want to comment? LOG IN or SIGN UP
TOP 16 Comments
  • New / Eng LCGrind
    @koala, what resources are you using for preparing system design. Thanks and welcome back.
    Sep 1 1
    • Oscar
      🐨koala

      Oscar

      BIO
      I am a Koala. DM me if you want to cuddle.
      🐨koalamore
      OP
      Grokking, youtube and a lot of googling lol. Like I would spend hours on just understand on little piece of buzz word. I hope my hard work will pay off... and also a book called Designing Data intensive applications
      Sep 1
  • Facebook / Mgmt vjuiu
    Remember the whole idea of consistent hashing is that you can drop a server and not rehash. There’s not guarantee it goes to the same server if it was the dropped server.

    Intra DC calls are low latency (~10ms). Having a local and a global Redis seems overly complex. Just make sure your Redis calls are atomic. CAP theorem will always apply so figure out which one you can trade off.
    Sep 1 4
    • Oscar
      🐨koala

      Oscar

      BIO
      I am a Koala. DM me if you want to cuddle.
      🐨koalamore
      OP
      Thanks vjuiu! But if you drop a server, all the original clients have no where to go but onto a different server (the “new” destination server would take responsibility of more clients). And yes 10ms isn’t that bad indeed. I will keep a note of that.
      Sep 1
    • Facebook / Mgmt vjuiu
      Basically, I was referring to the reliance on local values will not work since the server can fail and the caller will get redirected to a new server where the server thinks the caller has not used any quota.
      Sep 1
    • Bloomberg / Eng F.U. money
      Facebook, all redis calls are atomic. That’s not MemcacheD. Also, of course you have to rehash when you lose a node... Are you kidding me? The keys that were associated with that node will be distributed to the other nodes. Also, having a distributed system that lives in a single DC is a no-no.

      Koala, nothing ever guarantees you that you will land on the same server because 10 minutes later that server might not exist. That’s why having sticky sessions is such a bad idea.
      Leaky bucket might work to limit the total number of requests or to implement a back-pressure strategy, but you can’t really have one queue per client. You should mention it for sure though.
      Sep 1
    • Oscar
      🐨koala

      Oscar

      BIO
      I am a Koala. DM me if you want to cuddle.
      🐨koalamore
      OP
      Thanks. Yeah I was leaning toward token bucket and sliding window (or hybrid of the two).

      Right. I am accounting the fact that if the server doesn’t exist anymore, the next burst could well go unchecked: so I wss thinking to query the global redis if the local one is not accessible. But you brought up a good point... i just don’t want to come up as opinionated because I obviously have never implemented a rate limiter 😂 at scale.

      Maybe I am micro-optimizing here. If a server dies and we get 100k, soon or later the system will rate limit system eventually.
      Sep 1
  • Amazon cuang
    Wtf. Now we have people grinding design problems? Interviews are increasingly becoming an arms race of shit instead of a way to assess experience
    Sep 1 1
    • Oscar
      🐨koala

      Oscar

      BIO
      I am a Koala. DM me if you want to cuddle.
      🐨koalamore
      OP
      Grinding of any sort definitely isn’t fun, though the stuff I learn from system design are interesting. Another way to look at this system grinding is writing up a design doc. Well, you never know 100% of anything so you do research; maybe run some simulation and finally finish your design doc and present. It’s effectively the same IMO.
      Sep 1
  • Workday / Product tdf
    Lol welcome back!
    Sep 1 1
    • Oscar
      🐨koala

      Oscar

      BIO
      I am a Koala. DM me if you want to cuddle.
      🐨koalamore
      OP
      Thanks 🙏
      Sep 1
  • Roku cruella
    Koala. When did u say u weren't going to be back??
    Sep 1 2
    • Oscar
      🐨koala

      Oscar

      BIO
      I am a Koala. DM me if you want to cuddle.
      🐨koalamore
      OP
      Hello. Check out this post! "See y’all later! (Misc.)"
      https://us.teamblind.com/s/dTv0YmY6
      Sep 1
    • Roku cruella
      So u leave without me noticing and now u r back. Why do I care?

      Jk koala. I care. I was just locked out of my blind. So didn’t notice.
      Sep 1
    • Oscar
      🐨koala

      Oscar

      BIO
      I am a Koala. DM me if you want to cuddle.
      🐨koalamore
      OP
      Yeah. But it is too basic hence why I went into the rabbithole of finding stuff like little’s law and netflix’s adaptive rate limit lol but thanks
      Sep 1

Salary
Comparison

    Real time salary information from verified employees