Does Facebook, twitter etc newsfeed compute them as you refresh is its pre computed?

New / Eng

New Eng

May 22, 2018 6 Comments

As in title. Feels like if it’s computed during refresh that can take a long time. Of course fb and twitter have large computation power to do this but for someone without that resource will you still do it in same way? Or compute frequent users newsfeed and cache them etc. If so how often will you run such algorithm?

Just curious ty x


Want to comment? LOG IN or SIGN UP
TOP 6 Comments
  • Facebook toomp
    Well... We used to have control over this. But the AI has started to take over. It precomputes when it wishes and computes on the fly randomly. We’ve even observed instances where it computes as if you were using your dad’s account. It will reverse the order if it detects you have ever complained about Instagram moving away from chronological order.
    May 22, 2018 0
  • Twitter / Eng

    Twitter Eng

    Oracle, Facebook
    Create reverse chronological timelines on write, inject, rank and enrich on read
    May 23, 2018 1
    • Facebook / Eng unicorn-
      I can see how that prevents hot spots on read but you pay a huge cost each time a write by someone with a lot of followers happens. Majority of these writes are wasted because your DAU/MAU is low. Why not just use caching to work around hot spots and aggregate on read ?
      May 23, 2018
  • Facebook / Eng unicorn-
    It is a common distributed computing problem - aggregate on read (when you request your feed) or aggregate on write (when the event happens). Facebook aggregates on read - look at talk from F8 2008 or 09
    May 22, 2018 1
    • Amazon Alrightok
      That would make sense why I get a completely different feed 5 mins after I reopen the app. Interesting.
      May 22, 2018
  • New / Eng

    New Eng

    It's a combination of both. The recommendation model would work against a smaller set of data that is cached based on some criteria. We do that @ Bloomberg news. You could also use the search engine to creatively solve this problem.
    May 22, 2018 0


    Real time salary information from verified employees