How LLMs actually run.
I'm learning LLM inference from the ground up and explaining each idea in plain language, with playgrounds you can poke and diagrams you can follow.
Concepts
- week 8Why LLM decoding is memory-bound
To write each new token, the GPU has to read every weight of the model. That read, not the math, sets the speed limit.
Where I am
Week 1: Let's build GPT. 0/56 tasks ยท see the full path