From four matrices of random numbers to a working model you can download and publish — eleven chapters and seven hands-on labs, taken one clear step at a time.
Meet the four matrices of random numbers that a trillion corrections turn into a mind.
Day one: 268 million meaningless fractions, every one perfectly balanced.
Tokenisation — chopping text into the units a model actually reads.
Build byte-pair encoding from scratch and watch it discover its vocabulary.
Give every token 4,096 coordinates — a position in the space of meaning.
Pre-training: guess the next word, be wrong, nudge every number a hair, repeat.
Meaning becomes location — words drift into clusters entirely on their own.
Train embeddings and watch the map sort itself out, live, from nothing.
Why a frozen dictionary definition can't be the whole story.
Self-attention: queries, keys and values, and the 8×8 map of who listens to whom.
One attention head, forward and backward pass, written out by hand — no autograd.
Stacking, residual connections, and why the rectangle never changes size.
Twelve models: see why residuals and LayerNorm are non-negotiable at depth.
Fine-tuning: turn a brilliant text-predictor into something that will actually help.
Train a full model over 600 sentences, then catch it memorising.
Inference: the weights freeze, the thought forms, and then it dissolves.
Every step of the loop, at speed, from text in to the next word out.
Where the “7B” in a model's name actually comes from.
Write a real safetensors file by hand, then read it straight back.
Give it a model card and push your model to the Hugging Face Hub.