Divide work amongst instances - system design
In system design interviews, I’ve been asked this question a couple times and the interviewer doesn’t seem to quite like my answers. How would you answer this question?
A database has units of work to be done every <interval>. The same unit of work will need to be done again and again. A service will filter through the database looking for any work that needs to be done. It’ll grab the entries and do the work.
But the load of work is too much for one instance. You load balance it and make it highly available. Now you have multiple instances all trying to grab for the same work, but you need to prevent the work from being done multiple times at once. Especially consider a fresh start when the first service starts up, then a second, then a third. How to divide work amongst them?
I suggested adding a lock to the database during a grab. Lock, Grab any data that needs to be done, release lock. Now each instance will grab its own set of work to be done sequentially.
Is there a better alternative that I’m missing?