Machine learning at scale 🤖
← Back to book index

System Architecture Overview

A high-level view of the components and data flows in a production advertising system.

Component Inventory: Ad Server, Retrieval, Ranking, Auction, Feature Store, Logging

Ad Server

The entry point that receives ad requests and orchestrates the serving pipeline. Handles request routing, load balancing, and response formatting.

Retrieval System

Finds relevant ad candidates from billions of available ads. Uses inverted indexes, embeddings, and approximate nearest neighbor search.

Ranking System

Scores and ranks candidates using ML models. Incorporates predictions (CTR, CVR) with advertiser bids and quality signals.

Auction System

Executes the auction mechanism (GSP, VCG, etc.) to select winners and determine prices. Handles reserve prices and floor optimization.

Feature Store

Centralized storage for features used in ML models:

  • User features (demographics, behavior history)
  • Ad features (creative, targeting, historical performance)
  • Context features (time, location, device)
  • Cross features (user-ad interactions)

Logging System

Captures all signals for:

  • Model training data
  • Performance monitoring
  • Debugging and analysis
  • Revenue attribution

Data Flows and Feedback Loops

Forward Flow (Request to Serve)

  1. User request → Ad server
  2. Ad server → Retrieval → Filtering → Ranking → Auction
  3. Auction → Ad server → User

Feedback Loops

  1. Immediate: Click/conversion signals feed back to real-time features
  2. Short-term: User behavior updates feature store within minutes
  3. Long-term: Historical data trains new model versions

These feedback loops are critical for continuous improvement.

Offline vs. Near-Real-Time vs. Real-Time Components

Offline Components

  • Model training pipelines
  • Feature engineering jobs
  • Historical analysis and reporting
  • Budget planning and forecasting

Near-Real-Time Components

  • Feature store updates
  • Budget pacing adjustments
  • Frequency cap management
  • Streaming model updates

Real-Time Components

  • Request handling
  • Candidate retrieval
  • ML inference
  • Auction execution

Understanding the latency requirements and update frequencies for each component is essential for proper system design.

Content to be expanded...