Feedback Loops & Data Flywheel

By Ludovico Bessi, Senior ML Engineer at Google · Updated

A feedback loop forms because a recommender learns from data its own recommendations produced, which can narrow content, entrench popular items and hide new ones. Turned into a data flywheel — with exploration, debiasing and good logging — the same loop makes the system improve with every interaction.

The Loop

model → recommendations → user behavior → logged data → training → model

Every recommender is a closed loop. The model decides what users see; users can only react to what they see; those reactions become the next training set. Two consequences follow:

  1. Data quality compounds. Better recommendations produce more and better engagement data, which trains better models — the data flywheel that makes large platforms hard to compete with.
  2. Mistakes compound too. Biases in the model are reflected in the data and learned again, more strongly. Without intervention, the loop narrows what gets shown.

Degenerate Loops

Popularity amplification

Popular items get more exposure, therefore more engagement, therefore higher scores, therefore more exposure. Over time, impressions concentrate on fewer items, and new or niche items never get the data they would need to prove themselves.

Filter bubbles

A user's early clicks narrow their recommendations; narrower recommendations produce narrower clicks. Short-term engagement may hold, but discovery and long-term satisfaction suffer.

Self-confirming errors

If the model wrongly believes users dislike a category, it stops showing it and never collects evidence to correct the belief. Missing data looks like confirmation.

Training on your own outputs

A model trained only on impressions it chose learns to imitate its predecessor. Offline metrics improve ("the new model agrees with what was engaged") while real quality stagnates.

Breaking Out: Exploration

The fundamental fix is to collect data the current model would not have chosen.

Explore-exploit strategies

  • Epsilon-greedy: a small fraction of slots or requests show items chosen with some randomness. Simple; must be carefully bounded to protect user experience.
  • Upper confidence bound (UCB) and Thompson sampling: prefer items whose value is uncertain, not just random ones. New items with few impressions get a bonus until their quality is known.
  • Dedicated exploration surfaces or slots: for example, one slot per page reserved for new creators or new interests.

Exploration for new items

Cold-start items need initial impressions to get any signal. Common practice: guarantee a minimum budget of impressions for new items, target them to users predicted (from content features) to be receptive, and graduate items into normal ranking once their engagement estimates are confident.

Exploration for users

Occasionally show items from adjacent interests the model is unsure about, to detect new tastes early and keep the interest profile from collapsing.

Log propensities

Whatever randomness you add, log the probability with which each item was shown. Propensities make exploration data usable for unbiased training and off-policy evaluation (see Bias & Debiasing).

Building the Flywheel Deliberately

A healthy flywheel improves models through data, not just through volume:

  • Richer feedback: collect more than clicks — watch completion, explicit likes and dislikes, hides, surveys. Negative feedback is especially valuable because the loop otherwise under-collects it.
  • Fresher feedback: shorten the time from interaction to model update (see Training at Scale), so the loop corrects itself faster.
  • Broader feedback: use signals from search, external links and other surfaces, which are not controlled by the recommender and break the self-reference.
  • Content signals: content understanding models give the system a view of items that does not depend on past exposure.

Measuring Loop Health

Metric What it reveals
Catalog coverage Share of items receiving impressions
Impression concentration (Gini, top-1% share) Popularity amplification
New-item time-to-first-engagement Cold-start health
Per-user diversity over time Filter-bubble effects
Share of impressions from exploration Whether exploration is still funded
Model agreement with predecessor Risk of self-imitation

Track these over months, not just within A/B tests, because loop effects are slow.

Experiments in a Loop

Feedback loops also distort experiments:

  • If treatment and control models train on shared logs, a better treatment improves the data both learn from, shrinking the measured gap.
  • A treatment that changes what gets shown changes future training data for everyone after launch, so short tests may underestimate or overestimate long-term effects.

Mitigations: train treatment models on treatment-only data where feasible (data-isolated experiments), run long holdouts, and measure ecosystem metrics.

Key Takeaways

  1. Recommenders are closed loops: the model chooses the data it learns from.
  2. Unchecked loops amplify popularity, narrow interests and hide the model's own errors.
  3. Exploration with logged propensities is the fundamental correction; uncertainty-aware methods explore efficiently.
  4. Build the flywheel with richer, fresher and broader feedback, especially negative feedback.
  5. Monitor coverage, concentration and diversity over long horizons, and isolate data in experiments.