Runbooks/LLM Inference RunbookTrack A · From text to numbersRAG Runbook →0%
  1. 00 Start
  2. /
  3. 01 Tokens
  4. 02 Anatomy
  5. 03 Journey
  6. /
  7. 04 Hardware
  8. 05 Phases
  9. 06 KV cache
  10. /
  11. 07 Attention
  12. 08 Position
  13. 09 Precision
  14. 10 Cache ops
  15. /
  16. 11 Many GPUs
  17. 12 Engines
  18. 13 Decode loop
  19. /
  20. 14 Planning
  21. 15 Production
LLM Inference Runbook · Document 02 of 15 · Track A — From text to numbers

Track A · Document 02 · From text to numbers

Inside the Model: Anatomy of a Forward Pass

One lookup table, one block repeated thirty-two times, one matrix at the end. Know the shapes and every memory figure in this runbook becomes something you derive rather than remember.

Reads in about 30 minutes · 7 figures, one a live parameter counter · 7 interview questions · prints to clean A4

What is in this document

  1. The shape of the thing
  2. The cheap part: RMSNorm
  3. Attention: where tokens meet
  4. The feed-forward block: 70% of the model
  5. Counting the parameters yourself
  6. The top of the stack
  7. What each part costs at run time
  8. Interview questions
  9. FAQ
  10. Cheat sheet

1 · The shape of the thing

A large language model is less complicated than its reputation. It is one lookup table, one block repeated thirty-two times, and one matrix at the end. The block is always the same shape; only its learned numbers differ from layer to layer.

You need this document for one practical reason: it is what makes every memory figure in the runbook derivable. If you can turn a config file into a parameter count, you can turn it into weight bytes at any precision, into cache per token, and into a defensible answer to “how many users fit on this card”.

LLAMA 3.1 8B, END TO END · EVERY NUMBER HERE IS READ FROM config.json Embedding 128,256 × 4,096 THE RESIDUAL STREAM — 4,096 numbers per token every block adds to it; nothing overwrites it. This is the wire the whole model is built around × 32 IDENTICAL BLOCKS — same shape, different learned weights RMSNorm 4,096 params ATTENTION — tokens mix 41.9M params · 17% of the model · makes the KV cache + RMSNorm 4,096 params FEED-FORWARD — each token alone 176.2M params · 70% of the model · no cache at all + back into the stream, and on to the next of the 32 blocks — the rhythm is: mix, refine privately, mix again Final norm + LM head 4,096 × 128,256 LOGITS — 128,256 scores, one per vocabulary entry recomputed from scratch every step · 0.49 MiB per position in fp32 · sampling happens here, on the way out THE WHOLE MODEL IN ONE LINE 525,336,576 + 32 × 218,112,000 + 4,096 + 525,336,576 = 8,030,261,248 — which is where the “8B” comes from, to the digit. You can do this for any model from its config file in about ninety seconds, and being able to is worth more than remembering any single number in this runbook.

Two things to notice. The feed-forward block is four times the size of attention and gets almost none of the discussion. And attention is the only place tokens interact — which is exactly why it is the only place that needs a cache.

The residual stream is the idea worth holding on to

There is one vector per token, 4,096 numbers wide, and it runs the entire height of the model. No block ever overwrites it. Each block reads it, works out a small correction, and adds that correction back.

That design is why a 32-layer model trains at all — gradients have an unobstructed path from top to bottom — and it is why every head in every layer writes into a shared space rather than into a private slot. It also gives you the right mental model for depth: not a pipeline of stages, but a shared working surface that ninety-six components (two per layer) take turns amending.

The analogy

Picture a single shared document being passed down a long table of sixty-four specialists. Nobody may delete anything; each person may only write additions in the margin. Half of them are allowed to glance at everyone else’s copy before they write — those are the attention blocks. The other half must work from their own copy alone — those are the feed-forward blocks. At the end, one clerk reads the marked-up document and names the next word. The document is the residual stream, the specialists are the layers, and the strict no-deleting rule is the residual connection.

2 · The cheap part: RMSNorm

Before each half of the block there is a normalisation step. It divides the vector by its own root-mean-square magnitude and then scales it by a learned per-dimension weight.

what it does makes the magnitude of the vector predictable, so the next block sees inputs in a stable range no matter how much earlier layers added
parameters 4,096 per norm, two norms per layer, plus one at the very top — 266,240 in an 8B model, or 0.003% of it
vs the older LayerNorm RMSNorm skips the mean subtraction and the bias. Slightly cheaper, and empirically just as good, which is why modern models all use it
why you should care at all because it is the reason activations stay in a sane range — and activation range is exactly what decides whether quantisation works, which is document 09

3 · Attention: the part where tokens meet

Attention is the only place in the entire model where one token can see another. That single fact explains why it is the only place that produces a cache, and why the feed-forward block can be sharded, routed or replaced without anything breaking.

ONE ATTENTION BLOCK · d_model 4096 · 32 QUERY HEADS · 8 KV HEADS · head_dim 128 one token, 4,096 wide 4,096 is the width of the vector, not a token count and nothing to do with context length three projections from the same input — and they are not the same size W_Q  4096 × 4096 = 32 × 128 W_K  4096 × 1024 = 8 × 128 W_V  4096 × 1024 = 8 × 128 THE ARITHMETIC TRAP: 8 KV heads gives 8 × 128 = 1024 columns. It is not 4096 ÷ 8 = 512. Head dim is fixed; you multiply it by the head count. only K and V are ever written to the cache. Q is used once and discarded, because no future token will ever consult this token’s question Q — used now, thrown away K and V — cached: 2 × 8 × 128 × 2 bytes = 4 KiB per token, per layer W_O  4096 × 4096 full width again, in all three variants total 41,943,040 parameters — 16.78M + 4.19M + 4.19M + 16.78M
  1. One token arrives as a vector 4,096 numbers wide. That width is the model dimension — not a token count, and unrelated to context length.
  2. Three projections come from that same input, and they are not the same size: W_Q is 4096×4096, but W_K and W_V are 4096×1024.
  3. The arithmetic trap: 8 KV heads gives 8 × 128 = 1024 columns. It is not 4096 ÷ 8 = 512. Head dimension is fixed; the head count multiplies it.
  4. Only K and V are written to the cache — 4 KiB per token per layer. Q is used once and discarded, because no future token consults a past token’s question.
  5. W_O projects back to full width, 4096×4096, and is unchanged across MHA, MQA and GQA.
  6. The four matrices total 41,943,040 parameters per layer.

The asymmetry in step 2 is the whole mechanism of grouped-query attention, and document 07 is about nothing else. The query side stays full width; only the side that gets cached shrinks.

Q, K and V in one paragraph

Each token emits three vectors from the same input. The query is what this token is looking for. The key is what it advertises to others. The value is what it hands over if it is selected. Dot every query against every key to get scores, soften those into weights that sum to one, and take that weighted blend of values. In “the bank of the river”, the query from bank matches the key from river strongly, so river’s value dominates the blend and bank ends up meaning the riverside kind.

The asymmetry that matters for serving: queries are consumed immediately and thrown away, because attention is causal and no future token will ever ask a past token’s question. Keys and values are read by every future token and never change. Cache what is re-read; discard what is not. That is the whole argument, and it is why the cache formula starts with a 2 rather than a 3.

What W_O is actually for

Head 7 writes its 128 numbers into slice 7 of the concatenated output. Without a final projection, head 7 could only ever influence dimensions 896 to 1023 of the residual stream — permanently, in every layer. W_O, at 4096×4096, is what lets each head’s output be spread across the full width and mixed with the others.

The useful way to see it: W_O is 32 stacked blocks of 128×4096, one per head. Each head’s output is projected to full model width independently and the 32 results are summed. Heads therefore write into a shared space, and the next layer’s attention sees the sum rather than 32 labelled slots.

Two things that are not what they look like

4,096 is a width, not a length. It is how many numbers describe one token. A model 4,096 wide can have a 128,000-token context window; the two are different axes entirely. Head count is width ÷ head dim = 4096 ÷ 128 = 32, so the heads partition the vector — they do not each get a copy of it.

32 heads learn 32 relationships, not 32 meanings. Studied examples include heads that attend to the previous token, heads that track syntactic dependencies, heads that resolve pronouns, and induction heads that spot a repeated pattern and point at what followed it last time. Some heads do very little and can be pruned. Nothing resembling a word reappears until the very top of the stack.

4 · The feed-forward block: 70% of the model

Attention gets the papers and the interview questions. The feed-forward block gets four times the parameters.

THE PART NOBODY ASKS ABOUT, WHICH IS 70% OF THE MODEL one token 4,096 wide gate  4096 × 14336 up    4096 × 14336 SiLU(gate) × up element by element, 14,336 wide down  14336 × 4096 + three matrices × 4,096 × 14,336 = 176,160,768 parameters per layer, against attention’s 41,943,040 — four times as many WHY 14,336 AND NOT A ROUND NUMBER The classic feed-forward block had two matrices at 4× width, costing 8×d². SwiGLU has three, so 8/3×d keeps the count equal. CHECK IT 8/3 × 4096 = 10,923, then Llama’s 1.3 multiplier gives 14,199, rounded up to a multiple of 1,024 = 14,336. Exactly the config value. THE THREE THINGS THAT FOLLOW, AND THEY ALL MATTER LATER 1 · Each token goes through this alone — no token sees another. That is why mixture of experts replaces this block and not attention, and why routing different tokens to different experts breaks nothing. 2 · It produces no cache. 70% of the parameters contribute nothing to the per-user memory problem. 3 · It dominates the arithmetic, so it is where quantising the weights pays most.

If you can only remember one unfashionable fact about transformer anatomy, make it this one: most of a model is the part that processes each token in isolation, and all of the per-user memory cost comes from the smaller part that does not.

Why this is worth raising unprompted

Almost every candidate can talk about attention and almost none mention that it is a sixth of the model. Saying “the feed-forward block is 70% of the parameters, so it is where weight quantisation actually pays, and it produces no KV cache at all” does two things at once: it shows you have looked at a config rather than a diagram, and it correctly separates the weight-memory problem from the cache-memory problem, which is the distinction the rest of this runbook runs on.

WHY 32 LAYERS AND NOT ONE BIG ONE · DEPTH IS COMPOSITION, NOT REPETITION “The trophy did not fit in the suitcase because it was too big.” Resolving it needs several passes, because each round of attention allows exactly one hop. LAYERS 1–10 · surface establish the entities: a trophy, a suitcase LAYERS 11–22 · relations compare their sizes, retrieve stored facts LAYERS 23–32 · the next token bind the pronoun, commit to an answer TWO CAUTIONS ABOUT THAT PICTURE It is a tendency, not an architecture. The layers are identical in shape, nothing assigns them roles, and the division of labour that emerges during training is far messier than three clean bands. And nothing resembling a word reappears until the very top. A head’s output is 128 numbers and the stream is 4,096 wide — there is no layer at which you could read off “it means the trophy”. Claiming otherwise invites a follow-up you cannot answer.

The serving consequence is what matters here: depth is a multiplier in the cache formula. An 80-layer model costs 2.5× the cache of a 32-layer one at the same KV head count, because every layer keeps its own keys and values.

5 · Counting the parameters yourself

This is the most useful thing in the document. Seven numbers out of a config file, four multiplications, and you have a figure you can defend under questioning — plus the weight footprint at any precision and the cache per token, both for free.

COUNT IT YOURSELF · SEVEN NUMBERS FROM config.json, NOTHING ELSE embedding table 525,336,576 vocab × d_model = 128,256 × 4,096 attention, per layer 41,943,040 W_Q + W_K + W_V + W_O, and only two of the four are full width feed-forward, per layer 176,160,768 3 × d_model × FFN width — gate, up and down × layers 6,979,584,000 32 × (attention + feed-forward + two norms) output head 525,336,576 the mirror of the embedding table — small models often tie the two, large ones do not TOTAL PARAMETERS 8,030,261,248 = 8.03B — and the model card says 8B weights in bf16 14.96 GiB in fp8: 7.48 GiB · in int4 at 4.8 effective bits: 4.49 GiB KV cache per token 128 KiB 2 × layers × kv heads × head dim × 2 bytes — note that only three of the seven inputs appear where the parameters are: feed-forward 70.2% · attention 16.7% · embedding and head 13.1%

Change the numbers and watch which ones matter. Raising layers raises both the parameter count and the cache; raising FFN width raises the parameter count and leaves the cache untouched; raising kv heads barely moves the parameter count and moves the cache proportionally. That last row is the entire argument of document 07.

The recipe, in the order you would say it

“Embedding is vocab times d_model. Per layer, attention is d_model times q-heads times head-dim, plus twice d_model times kv-heads times head-dim, plus the output projection back to d_model — and the feed-forward is three times d_model times the FFN width, because SwiGLU has a gate, an up and a down. Multiply the layer by the layer count, add the embedding, add the output head if it is not tied. For Llama 3.1 8B that lands on 8,030,261,248 — which is where the 8B comes from.”

Ninety seconds, out loud, with the arithmetic visible. It works on any decoder-only model of this family, and it is a genuine live-interview exercise: being handed a config.json and asked what it costs to serve.

WHERE THE PARAMETERS ACTUALLY ARE · DERIVED, NOT QUOTED feed-forward attention embedding and output head Llama 3.1 8B8.03B70%17%13%Llama 3.1 70B70.55B80%17%Llama 3.1 405B405.85B81%18%Llama 2 7B6.74B64%32% THE TREND AS MODELS GROW The embedding and head are a fixed 1.05B for any Llama 3 model, so they fall from 13% of an 8B to 1% of a 405B. AND WHY LLAMA 2 7B LOOKS DIFFERENT It has no grouped-query attention, so all four attention matrices are full width — 32% against the 8B’s 17%.

Two practical consequences. Quantising the weights mostly means quantising the feed-forward block, because that is where the bytes are. And on a small model the embedding tables are big enough that vocabulary size is a real memory decision, not a detail.

A refinement to the “2N FLOPs per token” rule

The standard estimate is two floating-point operations per parameter per token — one multiply and one add. For the 8B that gives 16.06 GFLOP per token. But the embedding table is a lookup, not a matrix multiply, so its 525 million parameters contribute nothing. The honest figure is 2 × (8.03B − 0.53B) = 15.01 GFLOP, about 6% lower.

It rarely changes a decision, but knowing why the rule is slightly wrong is a good signal. On a 405B the gap is under half a percent; on a small model with a large vocabulary it is worth carrying.

6 · The top of the stack

After the last block there is one more normalisation and one more matrix. That matrix is the mirror image of the embedding table, and it turns a 4,096-number vector into a score for every word the model knows.

THE TOP OF THE STACK · WHERE A VECTOR BECOMES A DISTRIBUTION OVER WORDS final vector, 4,096 wide the residual stream after all 32 blocks, normalised once more × LM head, 4096 × 128256 one matrix multiply — 1.05 GFLOP per position, 7% of the whole forward pass 128,256 LOGITS one raw score for every entry in the vocabulary. Not a shortlist, not a search — every possible token is scored, every step. In fp32 that is 0.49 MiB for a single position. Sampling, penalties and grammar masks all operate on this vector, and document 13 is about what happens to it. WHY PREFILL THROWS AWAY ALMOST ALL OF THEM An 8,192-token prefill passes 8,192 positions through the stack. If you computed logits for every one of them, that tensor would be 8,192 × 128,256 × 4 bytes = 3.9 GiB. But you only need the last position — the others are already known, they are the prompt. So the head is applied to one position and 3.9 GiB never exists. This is why the logits buffer shows up in the activation reserve rather than in a formula, and why asking for log-probabilities on every prompt token is an expensive request, not a free one.
  1. The residual stream after all 32 blocks, normalised once more: one vector 4,096 wide.
  2. Multiplied by the language-model head, 4096×128256 — 1.05 GFLOP per position, about 7% of the whole forward pass.
  3. Out come 128,256 raw scores, one per vocabulary entry, recomputed at every step. In fp32 that is 0.49 MiB for one position.
  4. Prefill keeps only the last position. Computing logits for all 8,192 would need a 3.9 GiB tensor, and the other positions are the prompt — already known. This is why asking for log-probabilities on every prompt token is expensive rather than free.

A good detail to have ready: the logits tensor is the largest single activation in the model, and the only reason it is not a problem is that almost all of it is deliberately never computed.

QuestionAnswer
Is the output head the same matrix as the embedding table? Sometimes. “Weight tying” reuses one matrix for both and saves 525M parameters on an 8B. Llama 3 8B and 70B do not tie; the smaller Llama 3.2 models do, because on a 1B model the two tables would otherwise be most of the model
Why is it 128,256 and not 128,000? 128,000 learned entries plus 256 reserved slots for special tokens — role headers, end-of-turn, and spares for later fine-tuning. Document 01 covers what those do
Where does temperature act? On the logits, after the head and before the softmax. So do top-k, top-p, repetition penalties and grammar masks. Everything user-facing about sampling happens to this one vector — document 13
Does the model “look up” a fact? No. The relationship between sunrise and east is distributed across the weights; when those particular numbers flow through, the score for east comes out high. That is all “knowing” means here, and it is why a model cannot tell you where it learned something

7 · What each part costs at run time

The same anatomy, read three different ways — the three-way sanity check from the map document, applied to the model itself.

ComponentBytes storedBytes moved per decode stepOperations per token
Embedding table
525M params
1.00 GiB in bf16 Only the rows you touch — one per token, so effectively nothing Zero. It is a lookup
Attention weights
1.34B params
2.50 GiBAll of it, every step2.68 GFLOP
KV cache
not parameters at all
128 KiB per token per user — 1 GiB at 8k The whole of every active sequence’s cache, every step Small, but it grows with sequence length while everything else is constant
Feed-forward weights
5.64B params
10.50 GiBAll of it, every step11.27 GFLOP
Output head
525M params
1.00 GiBAll of it, every step1.05 GFLOP
Norms
266k params
0.5 MiBNegligibleNegligible

Read the second column, then the third

Column two is why decode is slow: every weight in the model is read to produce one token, plus the cache. Column three is why that read is wasted at small batch: about 15 GFLOP of arithmetic against 16 GB of traffic is roughly one operation per byte, on hardware that wants 295. Document 04 turns that ratio into a number and document 05 turns it into a strategy.

And note the one row that behaves differently. Everything except the KV cache is a fixed cost — the same bytes, the same operations, for every user and every step forever. The cache is the only line that grows, and it grows in two directions at once: with the number of users and with how long each of them has been talking.

8 · Interview questions

ArchitectWalk me through what happens inside one transformer layer.

There is a residual stream — one vector per token, 4,096 wide on an 8B, running the full height of the model. A layer reads it twice and adds to it twice. First: normalise, then attention, then add the result back. Second: normalise again, then the feed-forward block, then add that back. Nothing ever overwrites the stream, which is what makes 32 layers trainable and what lets every head write into a shared space.

The two halves do different jobs. Attention is the only place tokens see each other — and therefore the only thing that produces a KV cache. The feed-forward block processes each token in isolation, produces no cache, and is four times the size: 176 million parameters a layer against attention’s 42 million. That split is worth stating, because it separates the weight-memory problem from the per-user-memory problem, and those two have completely different fixes.

ArchitectHere is a config file. How much memory will this model need?

I would count the parameters rather than trust the name. Embedding is vocab times d_model. Per layer: attention is d_model times q-heads times head-dim, plus twice that for the two smaller KV projections, plus the output projection — and the feed-forward is three times d_model times the FFN width, because SwiGLU has a gate, an up and a down. Multiply by layers, add the embedding and the output head unless they are tied.

For Llama 3.1 8B that gives 8,030,261,248, so 14.96 GiB in bf16. Then I would not stop there, because weights are not the answer to a memory question. On an 80 GB H100 reporting 79.65 GiB, at 0.90 utilisation that is 71.7 GiB, minus 14.96 for weights, minus about 3 for activations and CUDA graphs, leaves 53.7 GiB of KV budget. At 128 KiB a token that is 53 users at the full 8k limit. Weights are one line of an eight-line ladder, and stopping at the first line is the most common mistake in this question.

ArchitectWhy does the KV cache formula not contain d_model or the number of query heads?

Because neither is cached. What gets stored is the output of W_K and W_V, and their width is kv_heads × head_dim — 8 × 128 = 1024 on an 8B, regardless of the fact that the model is 4,096 wide and has 32 query heads. Query vectors are computed, used once and discarded, because attention is causal and no future token consults a past token’s question.

Which is exactly why grouped-query attention works. You can shrink the key-value side by a factor of four without touching the query side, and the diversity that makes heads useful lives in the queries. The trap to avoid on a whiteboard: 8 KV heads gives 8 × 128 = 1024 columns, not 4096 ÷ 8 = 512. Head dim is fixed and the head count multiplies it.

ArchitectWhy 32 layers rather than one much wider one?

Because one round of attention allows exactly one hop of information movement. “The trophy did not fit in the suitcase because it was too big” needs several: first establish the two entities, then compare their sizes, then bind the pronoun. Depth is composition rather than repetition, and you cannot buy it with width.

The honest caveat is worth adding: the layers are identical in shape and nothing assigns them roles. The tendency — surface features early, relationships in the middle, next-token specifics late — emerges from training and is much messier than three clean bands. I would not claim a specific layer does a specific human-legible job.

For serving, the consequence is direct: depth is a multiplier in the cache formula. An 80-layer model costs 2.5× the cache of a 32-layer one at the same KV head count, which is why the 70B is 320 KiB a token against the 8B’s 128.

Eng managerThe team wants to try a model with a much bigger vocabulary. What should they check?

Three things, and only one of them is about quality. First, the memory: the embedding table and the output head are each vocab × d_model, so doubling the vocabulary on an 8B adds about a gigabyte of weights in bf16 — a gigabyte that comes directly out of the KV budget and therefore out of concurrency. On a small model those two tables can be a third of everything.

Second, whether it actually helps their text. Measured on the same samples, going from a 50k to a 100k vocabulary saved 4% on English prose, 31% on Python, and cost 14% more on numeric text. So the answer depends on the traffic mix, and it is a half-hour measurement rather than an argument.

Third, everything token-denominated gets re-measured: cost per request, p95 prompt length, whether prompts fit the window, capacity. I would ask for that measurement before the model change is scheduled, not after.

ArchitectWhat is the largest single activation in the model, and why is it not a problem?

The logits: 128,256 scores, one per vocabulary entry. That is 0.49 MiB per position in fp32. The reason it is not a problem is that prefill deliberately never computes most of them. For an 8,192-token prefill the full logits tensor would be 3.9 GiB, but every position except the last is a prompt token whose identity is already known — so the output head is applied to the last position only.

This is worth knowing for two practical reasons. It is why the logits buffer shows up as part of the activation reserve rather than as a clean formula. And it is why requesting log-probabilities for every prompt token is genuinely expensive rather than free: you are asking the server to materialise the thing it was carefully avoiding.

Eng managerSomeone proposes pruning attention heads to save memory. How do you evaluate that?

I would start by checking the size of the prize, because the framing is usually wrong. Attention is 17% of an 8B model’s parameters; the feed-forward block is 70%. Pruning heads is work in the smaller half. And it does not touch the KV cache at all unless you prune key-value heads, which is a different and much harder operation because it changes the cached tensor shape and needs retraining to recover.

So the question back is: which memory problem are we solving? If it is weights, quantisation gives 2× to 4× for a day of work against pruning’s uncertain single-digit percentage. If it is the per-user cache, the levers are an fp8 cache, a shorter context limit, or a model with fewer KV heads — and none of those is a research project. I would want that comparison written down before anyone spends a sprint on pruning.

9 · FAQ

What is d_model, exactly?

The number of values that describe one token at every point inside the model — 4,096 for Llama 3.1 8B, 8,192 for the 70B. It is a width, not a length, and it has nothing to do with context. The heads partition it: 4096 ÷ 128 = 32 heads, each owning a 128-number slice.

Is head_dim always 128?

Not by law, but it is remarkably stable. Llama 3.1 8B is 4,096 wide with 32 heads, the 70B is 8,192 wide with 64, and the 405B is 16,384 wide with 128 — all land on 128. Models scale by adding heads and layers rather than by widening each head. Read it from the config rather than assuming, because a few families differ.

Why is the FFN width 14,336 and not 16,384?

Because SwiGLU has three matrices where the classic feed-forward had two. The old design was 2 × d × 4d = 8d²; keeping that parameter count with three matrices needs a width of 8/3 × d, which for 4,096 is 10,923. Llama then applies a 1.3 multiplier and rounds up to a multiple of 1,024, giving exactly 14,336. You can reproduce the config value from the heuristic, which is a nice thing to be able to do.

Do all layers have the same weights?

The same shape, different numbers. Every one of the 32 blocks has its own 218 million parameters, learned independently. A few research architectures share weights across layers to save memory; no mainstream open model does, and you should assume they are distinct unless told otherwise.

Where is the model’s knowledge stored?

Distributed across the weights, with the feed-forward blocks carrying most of it — they hold 70% of the parameters and there is research treating them as key-value memories. But nothing is localised in a way you can point at. That is why a model cannot cite a source for a fact it produced, and why removing a specific fact is an open research problem rather than an edit.

Why does the residual connection matter for inference?

Mostly it does not — it is a training device. The one inference-facing consequence is that the residual stream is the thing that stays in a stable numeric range, which is what makes activations quantisable and what RMSNorm exists to protect. If you ever see an activation-outlier discussion in a quantisation paper, it is about this stream.

Does the embedding table get read on every decode step?

Only one row per token — 8 KiB out of a 1 GiB table — so it is negligible as a read. The output head is the opposite: the full 4096×128256 matrix is multiplied on every single step, which is 1.05 GFLOP and about 7% of the forward pass. The two tables are the same size and have completely different runtime profiles.

Why 2 FLOPs per parameter?

A matrix multiply does one multiply and one add per weight, so the forward pass is about 2N operations per token. The refinement: the embedding is a lookup rather than a multiply, so the honest figure is 2 × (N − embedding) — 15.01 GFLOP rather than 16.06 for the 8B, about 6% lower. Training is roughly 6N, because backward costs about twice forward.

What is weight tying and should I care?

Using the same matrix for the input embedding and the output head, saving vocab × d_model parameters. It matters most where that table is a big fraction of the model: Llama 3.2’s 1B and 3B tie, Llama 3 8B and 70B do not. When counting parameters from a config, check tie_word_embeddings — getting it wrong on a small model throws your total out by 10% or more.

Could I just remove layers to make the model smaller?

People do — it is called depth pruning, and it works better than you would expect, particularly on middle layers. But it is a training project: quality drops and needs recovery fine-tuning. Compared with quantisation, which gives a reliable 2× or 4× in an afternoon with published methods, layer pruning is rarely the right first move in a serving context.

10 · Cheat sheet

the shape embedding table → 32 identical blocks around a residual stream → final norm → output head → logits
one block norm, attention, add · norm, feed-forward, add. Two reads and two additions to the stream, never an overwrite
attention per layer W_Q d×(qh·hd) + W_K and W_V d×(kvh·hd) + W_O (qh·hd)×d = 41,943,040 on an 8B. The only place tokens meet; the only source of cache
feed-forward per layer 3 × d × ffn = 176,160,768 — four times attention, 70% of the model, no cache, each token alone
the total vocab×d + layers×(attn + ffn + 2d) + d + vocab×d = 8,030,261,248 for Llama 3.1 8B, which is where “8B” comes from
the split feed-forward 70% · attention 17% · embedding and head 13% — and that last one falls to 1% on a 405B
the trap 8 KV heads = 8 × 128 = 1024 columns, never 4096 ÷ 8. And 4,096 is a width, not a context length
FLOPs per token 2 × parameters, minus the embedding which is a lookup — 15.01 GFLOP for the 8B, not 16.06
the largest activation the logits, 128,256 wide. 3.9 GiB if prefill computed all 8,192 positions, which is exactly why it computes one

The ninety-second version

“A token id indexes an embedding table and becomes a vector 4,096 numbers wide. That vector travels up a residual stream through 32 identical blocks; each block normalises, runs attention, adds the result back, normalises again, runs a feed-forward block, and adds that back too. Attention is the only place tokens see each other, which is why it is the only thing that produces a KV cache — and it is only 17% of the parameters. The feed-forward block is 70%, processes each token alone and caches nothing. At the top, one more matrix turns the vector into 128,256 scores. You can count all of this from the config file: for Llama 3.1 8B it comes to 8,030,261,248 parameters, which is 14.96 GiB in bf16, and that is the first line of the capacity ladder rather than the answer to it.”

Where this connects

Thread started herePicked up in
The embedding lookup that a token id resolves to 01 · Tokenisation
These stages placed on a clock, for one real request 03 · Journey of a token
Why 15 GFLOP against 16 GB of traffic is the wrong ratio for the hardware 04 · The GPU and the roofline
Reading the whole model per token, and the batch that fixes it 05 · Prefill and decode
The cached K and V, and the formula in full 06 · The KV cache
Shrinking the key-value side without touching the query side 07 · Attention variants
Quantising the 70% where the bytes actually are 09 · Precision
Replacing the feed-forward block with many of them 11 · Many GPUs and MoE
What happens to the logits on the way out 13 · The decode loop

Questions to ask them