Scaling is easy
Walmartlabs crams everything into 1 Kafka topic. You can complain about their site, but reality is it works as well as Amazon or whatever else. It doesn't matter how.dumb your architect is, anything will work. Scaling is only a problem if you are blithely incompetent. If you hold the fact that your system is able to scale as a gold standard your standards are low. The vast majority of programmimg time is.spent on cosmetic things about the code and inane business requirements. The actual scaling nowadays is just hitting a button in AWS. And of course you'll talk about Albertsons because there isn't anything hard about scaling. Or about network switches and whatever that are now completely irrelevant in serverless. Face it, modern programming is baby stuff.
How come Albertsons is always out of my favorite fruit pops in the frozen aisle?
Because all messages, I mean pops, must go through one topic, I mean distribution center.
You have very narrow view to see things, please change your diaper first, then get more experience
Tell me about the scaling challenges at ZILLOW GROUP
Mmm it’s perfectly fine to have one topic, as long as you have enough partitions to distribute the load and process each partition consistently? I have managed a pretty sizeable infrastructure with 500 MB/s (megabyte) flowing into one single Kafka topic. But obviously I needed to have multiple partitions and multiple consumer groups.
Lol. Sizeable.
Glad to see you’re addressing a point that’s not relevant to the discussion. Even at 500MB/s you need multiple partitions or won’t be able to keep up if you’re doing anything slightly meaningful with the messages.
Sure. Sure.
Just click the autoscale button
you also don’t need network switches because you have serverless 🤦♂️
I mean how are u sure Walmart runs on a single Kafka queue ? That can’t be possible
It does, they cram everything into the queue into various partitions, then force consume the zero offset over and over. Kafka will handle millions of tps, this isnt even a big deal. I think the cluster is a few dozen nodes.
Anyone can scale with PHP and a dial-up connection
That's literally facebook
if (scaleLimitReached){ dont(); }
LimitReached ? Scale() : Dont();
I think he has a point. Its like design patterns some years back. You don’t always need to know them to do your work but you will be asked in the interviews.
Even in interviews they just ask trivia questions about pcf. There is nothing to quiz, it's just hitting buttons, so algo questions are used as a weedout to artificially increase the difficulty.
I’m wearing diapers