Multi-Objective Optimization
By Ludovico Bessi, Senior ML Engineer at Google · Updated
Multi-objective optimization predicts several outcomes per item — click, watch time, like, share, hide — and combines them into one ranking score. Multi-task architectures such as MMoE and PLE share learning across tasks, and a value model with tuned weights turns predictions into the trade-off the product wants.
One Score, Many Goals
A feed cannot optimize a single number. Clicks alone breed clickbait; watch time alone favors long passive content; likes alone favor certain genres. Platforms care about clicks, watch time, likes, shares, comments, follows, satisfaction, hides, reports, creator health and revenue at the same time. The ranking stage therefore:
- Predicts several outcomes per candidate (multi-task learning).
- Combines them into one score that reflects what the product values.
Multi-Task Learning
Training one model per objective wastes compute and data: tasks share a lot of structure (a user who clicks is more likely to watch). Multi-task models share most parameters and add task-specific heads.
Shared-bottom
The simplest design: shared layers, then one tower per task. Works when tasks are closely related. When tasks conflict, they fight over the shared parameters and some get worse — negative transfer.
MMoE (Multi-gate Mixture-of-Experts)
Google's MMoE (2018) replaces the shared bottom with several expert networks and gives each task its own gating network that decides how to mix the experts' outputs. Related tasks can share experts; conflicting tasks can rely on different ones. YouTube described using MMoE for multi-objective ranking in "Recommending What Video to Watch Next" (2019).
PLE (Progressive Layered Extraction)
Tencent's PLE (2020) separates task-specific experts from shared experts explicitly and stacks several extraction layers. It reduces the "seesaw" effect, where improving one task hurts another, which they observed with MMoE.
Loss weighting
The total loss is a weighted sum of task losses. Weights can be tuned by hand, or set by methods such as uncertainty weighting or gradient normalization (GradNorm). In practice, teams often tune them against online results.
Modeling Dependent Outcomes: ESMM
Some outcomes happen only after others: a conversion only after a click. Training a conversion model only on clicked impressions creates sample selection bias (the model never sees unclicked impressions, but must score them at serving time) and data sparsity.
Alibaba's ESMM (Entire Space Multi-task Model, 2018) models the chain over the entire impression space:
p(click and convert | impression) = p(click | impression) × p(convert | click, impression)
Both the CTR and the click-then-convert (CTCVR) probabilities are supervised over all impressions, and the CVR tower is learned implicitly. This removes the selection bias and shares data between tasks.
Combining Predictions into a Ranking Score
Weighted sum or product
The most common approach:
score = Σ_k w_k · p_k or score = Π_k p_k^{w_k}
Weights encode product priorities (how much is a share worth relative to a like?). Products of powers are scale-invariant per task, which helps when predictions have very different magnitudes.
Expected value
Convert each prediction into expected value in a common unit — for example, expected watch seconds, or expected "value" where each action has a measured long-term worth — and sum. This makes trade-offs explicit and auditable.
Calibration is mandatory
If any predicted probability drifts by 20%, a weighted sum silently reweights the objectives. Combined scores need calibrated predictions, monitored per task.
Tuning the weights
Weights are tuned with A/B tests, often by searching over weight vectors online (with Bayesian optimization or bandits) and picking the point that best trades off the target metrics. Teams sometimes define an explicit objective function in terms of online metrics (for example, retention-weighted engagement) and search weights to maximize it.
Beyond Engagement
- Negative signals: predicted probability of hide, report, or skip enters the score with negative weight.
- Satisfaction: models trained on survey responses ("was this worth your time?") can be included as an objective even though surveys are sparse; YouTube described using such signals.
- Integrity and quality: borderline or low-quality content predictions demote items.
- Ecosystem objectives: creator diversity and new-creator exposure are often handled in re-ranking rather than in the score.
Pareto Thinking
Multi-objective ranking is a search along a Pareto frontier: improving one metric without hurting others is a free win; beyond that, every gain costs something. Make trade-offs explicit, document the weight choices, and revisit them as product goals change.
Key Takeaways
- Ranking predicts many outcomes and combines them, because no single metric captures value.
- MMoE and PLE reduce negative transfer between tasks by sharing experts selectively.
- ESMM removes selection bias for dependent outcomes like click-then-convert.
- Combine calibrated predictions with explicit weights or expected-value formulas, tuned online.
- Include negative and satisfaction signals; treat trade-offs as movements along a Pareto frontier.