Choosing a model and loss

A courier company in a crowded city has to choose how its parcels move. A bicycle is cheap and quick through traffic but carries little, and a van carries everything but crawls at rush hour. Neither is better in general, and the company picks by looking at its parcels, its streets and what a late delivery costs it.

The second choice is how to score the drivers. Counting every minute late treats a two-minute delay and a twenty-minute delay as one tenth and ten tenths of the same thing. Squaring the delay makes the twenty-minute one a hundred times worse, and a scheme that cares only whether the parcel arrived in the promised hour ignores the minutes altogether. The vehicle is the model structure, the scoring rule is the loss, and the two have to be chosen together.

A van does not learn, which is where the courier company stops matching a model. A model structure decides which patterns the model is able to pick up from data at all, so a structure that cannot represent the interaction between two inputs will never find it however long it trains. The rest of this topic is about matching that ability to the data and the budget.

For a table of payment facts such as amount, time of day and account age, a linear model that gives each input one weight is a readable first step. A boosted tree model, which is a stack of small decision trees that each correct the mistakes of the last, can learn combinations such as a large amount from a brand-new account, which a linear model cannot. Text and images need something that turns raw words or pixels into useful numbers. That is almost always a pretrained model, covered in the next topic.

For recommendation, a two-tower model learns one vector of numbers for the person and one for the item, so that item vectors can be computed ahead of time and searched quickly. A model that looks at the person and the item together can learn richer patterns, but it would have to run once per item, and against a catalogue of millions that is far too slow. The usual design therefore runs a cheap two-tower stage to collect a few hundred candidates and a richer model to order them.

The loss is the rule that scores a prediction against the truth during training, and it should match what the product will do with the answer. A yes-or-no event such as a click uses a loss called binary cross-entropy, which rewards a probability close to what happened and punishes confident wrong answers heavily. A quantity such as minutes until arrival uses a regression loss, the kind used when the answer is a number, and the choice between squaring the error or taking its plain size decides how much the model fears rare large misses.

Suppose two deliveries were off by 2 and 20 minutes. Under absolute error the large miss is about ten elevenths of the total, and under squared error it is about 99 hundredths. A model trained on squared error therefore spends nearly all its effort avoiding the big miss and accepts a few more small ones. If the product promises a window rather than a single minute, a point estimate is not enough. The model should then predict a range or a high percentile, which is the time it will beat on, say, 90 percent of trips.

Ranking losses learn from a relevance grade per item, from which of two items a person preferred, or from a whole list at once. Say how the training pairs are built and why the loss should improve the order you will measure. No loss can repair a dataset that treats every item the person never saw as something they disliked, because those items were never shown.

A candidate is a hypothesis until it beats the baseline under the same conditions, and the comparison has to include what it costs to run. Suppose a richer ranker lifts an offline ranking score by half a percent but doubles the serving machines from 20 to 40. Check whether the gain holds across months and across the user groups the product cares about, and whether it shows up as a change in what people do. Then ask whether a smaller candidate set or better inputs would recover most of it for less.

Response time is the usual hard limit. Suppose the ranking stage has 40 milliseconds and 300 candidates to order. A tree model that scores one candidate in 0.02 milliseconds finishes in 6 milliseconds, while a neural ranker that needs 0.3 milliseconds per candidate takes 90 milliseconds and misses the budget. The design then has to choose between cutting the candidates to 100, scoring them in batches on a graphics processor, or keeping the tree model. The interviewer wants to hear that arithmetic out loud.

Size affects more than speed, because a large model takes longer to copy to every serving machine, slows a rollback and may need special hardware. A model that depends on a new live input adds a service that has to stay up even if the model file itself is small. Compare the whole request path and the whole training process, never the model benchmark on its own.

Naming the model before naming the limitation is the confusion interviewers notice first. An interviewer who hears the name of a large neural network in the first two minutes assumes the candidate reached for a favourite. One who hears that the baseline misses the interaction between sender and content, and that a tree model is a cheap way to learn it, hears a design.

The loss gets treated as a detail of training rather than a product choice. Squared error, absolute error and a percentile loss give three different models from the same data. The right one depends on whether a 20-minute miss is ten times as bad as a 2-minute one or a hundred times as bad.

A better offline score is then mistaken for a better system. A half-percent gain that doubles cost or breaks the response time budget is a worse system until the gain is shown to change what people do.

Open with the limitation the baseline showed, name one structure that can learn it, state the loss in one sentence tied to the product, and give the response time arithmetic for the candidate count you expect, all inside about five of the ten modelling minutes in a 45-minute round.

Leave architecture details such as layer sizes for a deep dive unless the interviewer pulls you there. A one-sentence offer to come back to them keeps the clock moving.