04 · Features and representations

New users and new items

9 min read

Cold start is the problem of making a good prediction for a user who has no history or an item that nobody has interacted with yet. Everything built from past behaviour, from the behaviour features of the features topic to the embeddings of the last one, is empty for them. The design decision it changes is what the system shows in the first minutes and how it earns the history it is missing.

An everyday example

A café has a new barista on her first day and a regular walks in. The old barista would have started the flat white before he reached the counter, while the new one has nothing to go on except that he is here at 8 am in a suit. She offers the house coffee, asks one question, and remembers the answer for tomorrow.

The same morning the café puts a new pastry on the counter. It has sold none, so there is no evidence that it is popular, and it will stay unsold if it is hidden at the back. She puts it next to the croissants it resembles and gives it the front spot for a day.

Those two mornings are the two halves of cold start. The new customer with no history gets a sensible default, a question or two, and an attentive first few visits, and the new item with no sales gets placed by what it is made of and given a trial in front of people. Every method below is one of those moves at a scale of millions.

The barista remembers a face after one visit, while a recommendation model learns only from clicks, and that difference is where the analogy breaks. A new pastry that is never shown gets no clicks and so is never learned about, a trap the café never falls into because the barista can see the pastry sitting there. The running example is a video app that recommends what to watch next, with a new viewer who opened the app for the first time and a video uploaded an hour ago.

Two kinds of cold start

A new user has no watch history, so every feature about their behaviour is empty, and the embedding that would place them near viewers with similar taste cannot be computed because it is learned from that history. The system still knows some things from the request, such as country, language, device, time of day, and whatever the sign-up flow asked, and those context features are the only inputs available for the first session.

A new video has no views, so nothing about how people reacted to it exists, and a model trained on watch counts, completion rates and co-watching sees a row of zeros. The system does know the video’s content, its title, its thumbnail, its length, its uploader and which channel it belongs to, and those item features describe it from the first second. Assume the app gets 50,000 new videos a day and 100,000 new sign-ups a day, so cold start is a permanent share of the traffic rather than a launch-week problem, and both paths need a design rather than a patch.

Two things can be newRequest arrives, A viewer and a video. New viewer, No watch history. New video, No views yet. Each branch loses a different half of the inputs.Two things can be newRequest arrivesA viewer and a videoNew viewerNo watch historyNew videoNo views yet
Each branch loses a different half of the inputs.On a narrow screen, scroll the figure sideways to see every step.

The standard ways around it

The first move is a default built from popularity, which means showing a new viewer the videos that people like them, by country and language, have watched most in the last day. It needs no history about the viewer and it is far better than random, since a small number of videos account for a large share of all watches on a video platform, a share treated here as an assumption rather than a measured fact. Its weakness is that it shows everyone the same thing, so it should last a session rather than a month, as the barista’s house coffee lasts one visit.

The second move is to use content instead of behaviour. A new video gets an embedding from its title, thumbnail and audio using an encoder trained on older videos whose behaviour is known, so it can be placed near videos that people with this viewer’s taste already watch. A new viewer’s few answers at sign-up, such as three topics they picked, do the same job from the other side. This is why the encoder in the embeddings topic is trained from content as well as behaviour, because a behaviour-only encoder has nothing to say about a video with no views.

The third move is exploration, which means deliberately showing a new video to a small sample of viewers it might suit so that it earns the history it lacks. Assume 50,000 new videos a day are each shown to 200 viewers in their first hour, which is 10 million exploratory impressions a day, or half a percent of traffic spent on learning for an app serving 2 billion impressions a day. The number is a design knob, since more exploration finds good new videos faster and costs more views that go to videos most people would not have chosen.

The fourth move is to learn fast once the first signals arrive. A viewer who has watched two videos is no longer cold. A system that updates its picture of the viewer within the session, using a running summary of what they have watched rather than waiting for tomorrow’s retrain, turns a cold viewer into a warm one in minutes. That requires the serving layer to keep a small per-viewer state that is updated on each event, which raises the same consistency question as any other history-based feature.

A new viewer’s first sessionOpen the app, No history. Popular list, Same country, no history. First watches, Two signals arrive. Personal list, Running summary. Getting from the second box to the fourth takes minutes, with no retrain.A new viewer’s first sessionOpen the appNo historyPopular listSame country, nohistoryFirst watchesTwo signals arrivePersonal listRunning summary
Getting from the second box to the fourth takes minutes, with no retrain.On a narrow screen, scroll the figure sideways to see every step.
The standard ways around it
MethodHelps withWhat it costs
Popularity defaultNew viewersEveryone sees the same list.
Content featuresNew videos and new viewersAn encoder trained on content.
ExplorationNew videosAbout half a percent of impressions.
Fast learningNew viewersPer-viewer state kept at serving time.

Earning history for a new item

A new video follows a different path from a new viewer, because the system chooses who sees it. In its first hour it is shown to the 200 viewers whose taste its content embedding sits nearest to, and their watches, skips and completions are logged like any other impressions. By the end of the hour it has enough history for the ordinary ranking model to score it alongside videos that have been up for a year. A video that does badly with its 200 viewers is not shown further, which is the cost of exploration landing on the viewers rather than on the creator.

The design question is how to tell whether the exploration budget is right. Too little and good new videos take days to surface, which you see as a long gap between upload time and the first thousand views for videos that later do well. Too much and the overall watch time drops, because viewers are shown things that did not suit them. Both are measured on the cold slice separately, since a system can look healthy on average while every new creator’s video sits unseen.

A new video’s first hourUploaded, No views. Content match, From title and thumbnail. 200 viewers, Shown, nearest by taste. Ranked normally, History now exists. Without the third box the video never reaches the fourth.A new video’s first hourUploadedNo viewsContent matchFrom title andthumbnail200 viewersShown, nearest bytasteRanked normallyHistory now exists
Without the third box the video never reaches the fourth.On a narrow screen, scroll the figure sideways to see every step.

Where people get confused

People defer cold start with ‘we will handle it later’ and never come back. Interviewers hear that as not knowing the answer, and the fix is a sentence that names the default, the content features and the exploration budget, then moves on.

A new item is also assumed to get discovered on its own if it is good. A ranking model trained on watch history gives a video with no history a low score, so it is never shown, so it never earns a score, and that loop only breaks if the design spends traffic on exploration on purpose.

Cold start also gets treated as a rare edge case. On a growing app, new users and new items are a steady share of every day’s traffic, and the first session decides whether a new user comes back, so the cold path deserves its own evaluation slice and its own metrics.

In the interview

Cold start is one of the most predictable follow-ups in any recommendation or search design, and two minutes on it is enough. Name it yourself when you describe the ranking model, for example ‘for new viewers I would fall back to popularity by country and language and learn within the session. For new videos I would use content embeddings and spend about half a percent of impressions on exploration’. That covers both halves, and it signals that you know history-based features are empty on day one.

Interview questions

How does your video recommender handle a viewer who signed up a minute ago?

For the first session I would show the most-watched videos in the last day for their country and language, because that is the best I can do with no history and it is far better than random. If the sign-up flow asked for a few topics of interest, I would use those to pick within the popular set.

Then I would learn fast. After their first two or three watches I would update a running summary of their taste inside the session and re-rank from it, rather than waiting for a nightly retrain, so the viewer is personalised within minutes. The trade is that the serving layer has to keep a small per-viewer state, which is more infrastructure than a lookup that remembers nothing between requests, and I would measure the new-viewer slice separately because the first session decides whether they come back.

A creator uploads a video. How does it ever get recommended if the model relies on watch history?

On its own it would not, because a ranking model trained on history scores a video with no history at the bottom, and a video that is never shown never earns history. I would break that loop in two ways.

I would give the video a content embedding from its title, thumbnail and audio, using an encoder trained on older videos, so it can be placed near videos that similar viewers already watch. Then I would spend a small exploration budget, say 200 impressions in its first hour to the viewers nearest to it by taste, which across 50,000 uploads a day is about half a percent of traffic. After that hour it has enough watches and skips to be ranked like any other video, and the cost is that those 200 viewers saw something less certain than usual.

How much traffic would you spend on exploration, and how would you know it is too much?

I would start at around half a percent of impressions, which with 2 billion impressions a day is 10 million a day, enough to give each of 50,000 new videos 200 views in its first hour. That is an assumption to tune rather than a known right answer.

I would know it is too much if overall watch time or completion rate drops on the slice of impressions used for exploration more than the gain in watch time from newly surfaced videos pays back. I would know it is too little if good new videos take days to reach their first thousand views, and I would watch both numbers on their own dashboard rather than letting them hide inside the averages.

What to practice next

Design these systems on the canvas and get your design graded.

These problems have you rank items by how popular they are, which is the default a new viewer sees, and measure closeness between the lists of numbers that stand for a video’s content, which is how a new video gets placed before anyone has watched it.