Multiple Retrieval Paths & Blending
By Ludovico Bessi, Senior ML Engineer at Google · Updated
Production recommenders retrieve candidates from many sources at once — two-tower embeddings, item-to-item co-engagement, graph walks, follows, trending and fresh content — because each retriever has blind spots. Blending merges these candidate sets with quotas and scores so the ranker sees a diverse, high-recall pool.
No Single Retriever Is Enough
Each retrieval method has blind spots:
- Two-tower embeddings capture general taste but blur specific, recent intents.
- Item-to-item similarity captures "more like what I just watched" but not broader interests.
- Graph walks capture community structure but need engagement history.
- Follow/subscription feeds capture explicit choices but miss discovery.
- Trending and fresh content capture time-sensitive items that no model has learned yet.
Production systems therefore run many retrieval paths in parallel — often a dozen or more — and merge their outputs before ranking. Recall of the union is much higher than recall of any single source.
Common Retrieval Paths
| Path | Signal | Strength |
|---|---|---|
| Two-tower / embedding | Learned user and item vectors | General personalization |
| Item-to-item (I2I) | Co-engagement, content similarity | Short-term intent, "more like this" |
| User-to-user | Similar users' engagement | Discovery through peers |
| Graph-based | Random walks on user–item graphs | Community and niche interests |
| Social / follow | Followed creators, friends | Explicit preferences |
| Multi-interest | Several user vectors per user | Users with diverse tastes |
| Content / semantic | Text, image, video embeddings | Cold-start items |
| Trending / fresh | Popularity velocity, recency | Time-sensitive content |
| Exploration | Random or uncertainty-based | Data for new items and interests |
Item-to-item in practice
For each item a user recently engaged with (the seeds), retrieve its nearest neighbors in an item–item similarity space. Seed selection matters: the most recent items capture current intent; the most-engaged items capture stable interests. Many systems weight seeds by recency and engagement strength.
Graph-based retrieval
Pinterest's Pixie performs biased random walks on a bipartite graph of pins and boards, starting from the user's recent pins, and returns the most-visited pins. It runs in real time on an in-memory graph. PinSage learns graph-convolutional embeddings for pins that combine content features with graph structure, and those embeddings feed embedding retrieval.
Multi-interest retrieval
A single user vector averages across interests: someone who likes jazz and mountain biking gets a vector that matches neither. Multi-interest models such as MIND (Alibaba) use capsule-style routing to extract several interest vectors from a user's history; each vector retrieves its own candidates. A simpler variant clusters a user's history and uses each cluster centroid as a query.
Blending: Merging Candidates
Once each path returns its candidates, the system must build one set of a fixed size (say 2,000) for pre-ranking. Approaches, from simplest to most sophisticated:
Fixed quotas
Each source gets a fixed number of slots. Simple, predictable, and protects small but important sources (fresh content, exploration). The downside: quotas do not adapt to the user — a user who never engages with trending content still gets trending candidates.
Score normalization and merging
Sources produce scores on different scales (cosine similarity, walk visit counts, popularity). To merge by score you must first normalize — per-source rank, percentile, or calibration against engagement probability — then take the top items overall. Deduplicate, and record which sources produced each item (it is a useful feature for ranking).
Learned allocation
Treat quota allocation as a learning problem: estimate, per user segment or per request, which sources contribute items that end up being engaged with or ranked highly, and allocate slots accordingly. Bandit approaches adapt allocation online while keeping some exploration.
Let the ranker decide
The most common pattern in large systems: take a generous union of all sources (with light quotas to bound cost), deduplicate, and let the pre-ranker and ranker sort everything out. Source identity becomes a ranking feature. This works as long as the ranker sees enough candidates from every source in training to judge them fairly.
Measuring Each Path
- Unique contribution: how many of the final top-N items came only from this source? A source with zero unique contribution adds cost but no value.
- Recall of engaged items per source.
- Final impression share per source over time — a sudden drop usually means the source broke.
- Latency and cost per source, since all paths run in parallel and the slowest one bounds retrieval latency.
Regularly ablate sources (turn one off in an experiment) to confirm each still earns its place.
Key Takeaways
- Every retrieval method has blind spots; production systems run many paths in parallel.
- Item-to-item, graph, multi-interest, social, content, fresh and exploration paths complement embedding retrieval.
- Blending ranges from fixed quotas to learned allocation; most large systems take a union and let the ranker decide.
- Normalize scores before merging and keep source identity as a ranking feature.
- Measure each source's unique contribution and ablate regularly.