If you scroll down to the bottom of home feed or a specific tweet, you get a loading image and the next dozen or so tweets/replies are retrieved and displayed. I am wondering what the best approach would be, as I couldn't find a good answer online. My thoughts are that 1. The comments in a tweet are pre-sorted/ranked and put into cache+DB from "fanout" process and so user queries DB/cache with an offset (Cassandra and redis I think work) continuously to get next N tweets/comments until reaching max offset. If the ranking changes, we can leave offset as is and accept potential duplicate tweets. 2. Set rate limit to 1000 max tweet requests to prevent overload. all tweets and comments are precomputed/ranked and only offset changes are tracked. Also Let comments of tweets expand to full size, as ranking for a tweet isn't unique per user and is pre-cached for many users. 3. If user refreshes page, the pagination offset should be reset so a kafka queue streaming user events has a topic another "offset service" can pull from and update offset status in cache. Also if user "pulls" data, offset status is updated as well by adding N to it. And home feed service always reads offset from cache before delivering next N tweets. Pulling is blocking client and so offset will be reflected accurately. To handle different concurrently logged in devices, we will map offsets to specific device and user ids in data model as composite key. However, if user pulls data but refreshes web browser or closes it before it finishes, I'm wondering how I can handle something like that as well. Assume the request for new tweets at an offset is processing in server but web browser is refreshed so server will send HTTP response to disconnected client as a new request expecting 0 offset is queried while offset in cache/db are not correct as they are like 100 and not 0. since http can't tell server a client disconnected, not sure how such event can be handled. Should I use bidirectional protocol like websocket then to handle "get next N tweets" API to maintain offset and so if client disconnect that implies an offset reset? Or a whole new approach ? #twitter #facebook #linkedin #snap
I will start by not fucking up DNS entries first
Haha
0. Put the DNS entries in a chest burry them at the bottom of Pacific Ocean, and pray no one finds it
O_O
can you go to sleep? it's 12 am