Embeddings and similarity search

The model that produces an embedding is called an encoder, and its training determines what information the vector preserves. In visual search, matching the same object and finding a similar-looking alternative are different goals. Training pairs that represent the intended relationship, together with contrasting examples, help the encoder learn comparisons that can be evaluated against the actual search task.

In a two-tower retrieval model, one encoder represents the user or query and another represents the candidate item. Training encourages relevant pairs to score above sampled alternatives. Item vectors can be computed ahead of time because the item encoder does not need the current request; a richer joint model can later rerank retrieved pairs.

The similarity function determines how vectors are compared, so it must fit the relationship learned during training. Dot product, cosine similarity, and Euclidean distance do not generally produce the same ranking, because vector magnitude can affect the comparison. For unit-normalized vectors, maximizing dot product is equivalent to minimizing squared Euclidean distance, which is why the normalization choice needs to remain consistent between training and retrieval.

Exact search compares the query with every eligible vector and can be a useful baseline. Approximate nearest-neighbor search reduces work by accepting some missed neighbors. Measure ANN recall against exact search on representative queries, then separately measure whether the retrieved items are relevant to users. High ANN recall only means the approximation matches the chosen vector space.

Eligibility filtering changes how many retrieved neighbors can actually be returned. Retrieving 100 products and then removing 95 unavailable ones leaves only five results, even if the vector search itself performed well. Index-supported filtering, retrieving additional candidates, or an appropriate partitioning scheme can help, with evaluation needed for restrictive filters and newly added items.

A new encoder can rotate or otherwise change the embedding space without changing vector dimensions. Querying old item vectors with it may produce meaningless neighbors. Build and validate a compatible item index, then switch the query encoder and index together or use a transition explicitly designed for compatibility.

The index also needs a way to handle new items, deletions, and access changes between full rebuilds, with eligibility checks enforcing any immediate restrictions. These responsibilities illustrate why representation quality and operating behavior must be evaluated together. We now have the data and feature concepts needed to build a simple baseline before considering a more complex model.