What is a two-tower recommender?
Introduction
Project introduction
What is a two-tower recommender?
A video site, a shop or a music app can have millions of items and only a few milliseconds to decide what to show. Recommenders usually split the work into two stages. Retrieval quickly picks a few hundred candidates out of the whole catalog, and a slower ranking model puts those candidates in order. This project builds the retrieval stage.
A two-tower model does retrieval with two small neural networks. The user tower turns a user into a vector, which is a list of numbers, and the item tower turns an item into a vector of the same length. The score is the dot product of the two vectors, so every item vector can be computed ahead of time and searched quickly when a user arrives.
Why is training it tricky?
Interaction logs record what people chose and never what they skipped. The usual trick is to treat the other items in a training batch as wrong answers, which costs almost nothing. Items appear in batches as often as people choose them, though, so a popular item is a wrong answer for almost every user and the model learns to push it down.
The project uses generated data with 2,000 users, 1,000 items and a long tail of item popularity. A model trained the plain way finds under 10% of the held-out choices in its top 20, while recommending the most popular items to everyone finds 35%. Subtracting each item's log sampling probability inside the loss, a correction published by Google researchers in 2019, lifts the model to about 48%.
Two towers, one score
1,000 itemsWhat you will build
01
Score and train
The dot-product score and the in-batch softmax loss that trains both towers.
02
Correct the bias
The log q correction that stops popular items being punished for their popularity.
03
Retrieve and judge
Top K retrieval without seen items, then recall, tail recall and coverage.
Start with the score
Score every user against every item with one matrix product.