Machine learning at scale 🤖

LLM Inference At Scale

A comprehensive guide to optimizing and scaling Large Language Model inference at production scale.

This book is currently a work in progress. Click here to learn more about the project and stay updated.

Navigation

Part 1: Fundamentals

Part 2: Training Optimization Techniques

Part 3: Inference Optimization Techniques

Part 4: Computation Optimization

11. Inference: Computation Reduction

Part 5: Batching & Production

12. Inference: Batching Techniques