Deployment & Scaling · Practitioner
Model Serving & Inference Optimization
Squeeze throughput and latency from GPU inference: KV cache, continuous batching, quantisation, and speculative decoding.
- 55 min lesson
- advanced level
- Engineering Track
- Narrated presentation
What you’ll learn
- Prefill vs. decode phases
- KV cache and paged attention
- Continuous (in-flight) batching
- INT8/INT4 quantisation for serving
- Speculative decoding
- Latency SLOs: TTFT and inter-token latency
Inside this lesson
A narrated, slide-by-slide presentation with 1 prediction exercise, 1 knowledge check, 3 videos. You answer as you go, and the platform checks your work.