Twitter has finally made most of its recommendation algorithm open source which is now available for independent third parties and users.According to Twitter CEO Elon Musk, many embarrassing issues will be discovered, but we "will fix them fast".
"Acid test is that independent third parties should be able to determine, with reasonable accuracy, what will probably be shown to users," Musk posted on Saturday.Musk said that most of the recommendation algorithms will be made open source and the rest will follow.
According to Twitter, "the recommendation pipeline is made up of three main stages"."Fetch the best tweets from different recommendation sources in a process called candidate sourcing; rank each Tweet using a machine learning model; and apply heuristics and filters, such as filtering out tweets from users you've blocked, NSFW content, and tweets you've already seen," the micro-blogging platform explained.
The service that is responsible for constructing and serving the 'For You' timeline is called Home Mixer."Home Mixer is built on Product Mixer, our custom Scala framework that facilitates building feeds of content. This service acts as the software backbone that connects different candidate sources, scoring functions, heuristics, and filters," the company further elaborated.
The goal of the For You timeline is to serve people relevant tweets.Twitter has several Candidate Sources that it uses to retrieve recent and relevant tweets for a user."At this point in the pipeline, we have 1,500 candidates that may be relevant. Scoring directly predicts the relevance of each candidate Tweet and is the primary signal for ranking tweets on your timeline," said the company.
At this stage, all candidates are treated equally, without regard for what candidate source it originated from."Our recommendation system is composed of many interconnected services and jobs. There are many areas of the app where tweets are recommended -- Search, Explore, Ads," said the company.