Document 11 · Quick learn Structure · Build quality · The live dial

Three knobsunder four sets of names

Every index has the same three knobs under different names. Which one you reach for depends entirely on which currency you are short of — and a default cannot know that.

“A number nobody can explain will eventually be changed by somebody who cannot explain it either.”

4×M

the efConstruction floor

5×k

the efSearch target — and k is 100

0 memory

what every live dial costs

p99 not mean

where the diagnoses separate

Parameters are a budgetnot a default

Every index parameter spends one of three currencies to buy another. Which currency is scarce in your system decides every value — and it is the thing a default cannot know.

Recall

What fraction of the true answers you actually return.

Usually the thing being protected

Latency

How long a query takes.

What live dials spend — and they spend nothing else

Memory

What the structure costs, permanently and per replica.

What structural parameters spend. No live dial can buy it back

five index types · four sets of names · one shape

a structure knob, a build-quality knob, and a live dial — learn the shape and the names become trivia you can look up

HNSW: the three you will be asked abouteach spends a different currency

M — the structure knob

How many neighbours each node keeps: the width of every step.

Costs: memory, permanently and per replica — N × 2M × 4 bytes
Raise it when: the live dial has plateaued below target and you have memory to spend. Higher dimensions also want a higher M

efConstruction — build quality

How large a candidate pool the build searches before choosing each node’s M neighbours.

Costs: build time only. Nothing at query time — a well-built graph is free to search
Raise it when: mean recall is fine but the tail is bad. And always keep it at 4×M or above

efSearch — the live dial

How many candidates the bottom-layer walk keeps alive.

Costs: latency only. No memory at all, and it goes in the request
Raise it: always first. Free to test, free to undo, and the only knob that tells you which kind of problem you have

The same shape, in the other three

IVF

nlist · k-means iterations · nprobe

The ratio is what matters, not either number. A copied nprobe scans a completely different fraction of a differently-sized index

DiskANN

R · build L, alpha · search L, rescore depth

Build L must exceed R — you cannot fill a degree from a smaller candidate pool. And R should fill a disk page

ScaNN

leaves, chunk size · training sample, anisotropic weight · leaves_to_search, reorder depth

Reorder depth repairs chunk-size loss. Aggressive compression with no reorder loses ranking accuracy you cannot get back

Flat

—

No knobs, and that is its entire appeal

Vendor defaults, and the two trapsquote them with a date and a hedge

pgvector

M 16 · efConstruction 64

hnsw.ef_search default 40, max 1000

Qdrant

M 16 · efConstruction 100

hnsw_ef defaults to ef_construct

Milvus

M 16 recommended, range 5–48

ef, range [topk, 4096]

The HNSW paper

M 16 · efConstruction 100

Trap one — efConstruction 64 is low for production

It was chosen deliberately, balancing recall against build time for general use. But at 1536 dimensions with a 95% recall target it will usually leave recall on the table — and no amount of efSearch gets it back.

Trap two — probes = 1 with lists = 100

A genuinely bad configuration at scale. One probe loses every boundary query.

Neither default is wrong for a small table. Both are wrong for ten million rows

A dimension limit worth knowing

pgvector’s vector type has a 2,000-dimension limit for HNSW indexes. A 3072-dimension embedding cannot be HNSW-indexed there without reducing dimension first.

Which connects straight back to Matryoshka truncation — and is exactly the kind of constraint that should surface during model selection rather than during implementation

How the parameters interactmost are independent; these eight are not

M and efConstruction

efConstruction ≥ 4 × M

High M with low efConstruction fills good slots with bad neighbours — you pay the memory and do not get the quality

efSearch and k

efSearch ≥ k, target ~5×k

Cannot return k results at all, and silent recall loss below that

nlist and nprobe

The ratio is what matters

A copied nprobe scans a different fraction of a differently-sized index

nlist and N

nlist ≈ √(nprobe × N) as a sanity check

Buckets too large — or the routing stage costs more than the buckets

R and build L

L > R

You cannot fill the degree from a smaller candidate pool

R and disk block size

A node should fill a block

Wasted reads, or two reads per hop

chunk size and reorder depth

Reorder repairs chunk loss

Aggressive compression with no reorder loses ranking accuracy permanently

M and dimensions

Higher dimension wants higher M

The graph becomes unnavigable at high dimension with a low M

The trap this prevents

Recall is low, so raise efSearch

It improves — which feels like progress, because it is.

Still short, raise it again

Latency doubles. Recall moves a little.

And again

Latency is blown and recall is still short.

The curve was never going to reach it

The graph was built at efConstruction = 64.

No amount of search effort repairs a bad edge

Symptom to knobnotice how often the answer is “rebuild” rather than “tune”

Recall still climbing with efSearch

You have a search problem.

Keep going — free to test, free to undo, effective on the next request
And check the two constraints first: efSearch ≥ k, efConstruction ≥ 4×M
Remember k is 100, not 10, if a reranker follows

Read the shape
of the curve.
That plateau
is the diagnosis.

Recall plateaued below target, latency rising

You have a structure problem.

The structure cannot deliver the target at any search effort
Rebuild at a higher M or efConstruction
And the mirror mistake: rebuilding for six hours when efSearch 40→80 would have done it in one second

The distribution, not the average

Ninety-five per cent mean recall can mean every query is slightly imperfect, or it can mean five per cent of queries return near-garbage. Those are wildly different products and the mean cannot tell them apart. Users experience the tail.

The same applies to latency: k-means does not produce equally sized clusters, so at nlist = 4096 over 10M vectors most buckets hold 800–3,000 and a few hold 40,000 or more. Those are your p99

Graph index, bad tail

Poor edges from a low efConstruction — most queries route fine and some land in a badly connected region and stall.

That is a rebuild, not more efSearch

IVF, bad tail

Queries landing near partition boundaries.

More nprobe, or a re-cluster

Deriving a configuration from a briefthe interview version is never “what is a good M?”

A — latency-bound interactive search

5M products, 768-d. p99 25 ms. Recall@10 target 0.90. 2,000 QPS.

Scarce currency: latency
What the brief tells you first: k is not 10. A reranker needs candidates, so retrieval k is more like 100 — that changes efSearch by an order of magnitude
Memory: 15.4 GB of vectors + 1.3 GB of graph at M=32 → ~19 GB. Affordable
Answer: HNSW, M 32, efConstruction 200, efSearch swept from ~500. Buy recall with memory

Same question.
Different scarce
currency.

B — memory-bound scale

200M chunks, 768-d. p95 200 ms. Recall 0.95. Rebuilt nightly. One machine.

Scarce currency: memory
What the brief tells you first: 665 GB of RAM for HNSW is not available, so the index-type decision comes before any parameter decision
Memory: not affordable at all. DiskANN, or IVF with quantisation
Answer: DiskANN. R sized to the disk page, alpha ~1.2, search L and rescore depth swept. The 200 ms budget makes 3–5 ms free

The tuning runbookthe order matters more than any individual value

Establish a flat baseline over a samplewithout it you have no recall number — only a comparison of approximations

offline

Name the scarce currencyrecall, latency or memory

5 minutes

Check the two constraintsefSearch ≥ k · efConstruction ≥ 4×M. Fix those before measuring anything

free

Sweep the live dialacross its useful range, and plot recall against latency, not recall alone

an hour

Read the shapestill climbing → keep going. Plateaued below target → structure

the diagnosis

Look at the distributionnot the mean — the tail is where the diagnoses separate

p99

Only now rebuildand change one structural parameter at a time

hours

Record the operating pointthe recall it achieved, and the date

one line

Pin the operating point in configuration, not in a person’s memory

efSearch, nprobe and rescore depth should be explicit values in a config file with a comment saying what recall they were chosen for.

A number nobody can explain will eventually be changed by somebody who cannot explain it either

Put a recall gate in CI

A small gold set, a threshold, and a build that fails when retrieval quality drops.

Every parameter here can be broken silently. There is no exception thrown when efSearch is too low or the graph was built badly — you get k results with plausible scores. Assertions are the only thing standing in for the errors other subsystems get for free

Answer it out loudthe ninety-second version, then six questions

“Every index has the same three knobs under different names: a structure knob that costs memory permanently, a build-quality knob that costs build time and nothing at query time, and a live dial that costs latency and goes in the request. On HNSW those are M, efConstruction and efSearch. So the first question is which currency is scarce, because that decides every value. Then two constraints before measuring anything: efSearch at least k — and k is a hundred if a reranker follows — and efConstruction at least four times M. Then sweep the live dial and read the shape: still climbing means keep going; plateaued below target while latency rises is the diagnosis, and that is a rebuild.”

Six questions, and the spine of each answer

Architect

What is a good value for M?

It depends on what is scarce — three numbers first: corpus size and dimension, latency budget, recall target
M costs memory permanently and per replica: N × 2M × 4 bytes. At 10M, M=32 is 2.6 GB of edges
Latency scarce → higher M, because a denser graph reaches target recall with less search effort
Memory scarce → lower M, and buy the recall back with efSearch

And whatever M you pick, efConstruction has to be at least four times it, or the edges were chosen from too small a pool and no amount of efSearch repairs them.

Architect

Recall is below target. What do you do?

First ask: search problem or structure problem? Completely different fixes
If there is latency headroom, raise the live dial — free to test, free to undo, effective on the next request
Then read the shape of the curve. Still climbing → keep going
Plateaus below target while latency keeps rising → that plateau is the diagnosis

The mistake I would specifically avoid is the mirror image: rebuilding for six hours when efSearch from 40 to 80 would have done it in one second.

Architect

Mean recall is 95% but users complain.

Almost certainly the distribution. 95% mean can mean every query is slightly imperfect — or 5% return near-garbage
Users experience the tail, not the mean
On a graph index: poor edges from a low efConstruction — most queries route fine, some stall. A rebuild
On IVF: boundary queries — more nprobe, or a re-cluster

Either way, the first move is to plot the distribution rather than argue about the average.

Architect

Recall dropped and nobody changed anything.

Not at the parameters — if no parameter changed, the cause is not a parameter. It is the data or the traffic
New content in a region the centroids do not cover — IVF and ScaNN degrade as the corpus drifts from what k-means was trained on
The query distribution shifted — a new customer, language or document type
Insert-and-delete churn degrading the graph over time

The fix is re-clustering or a rebuild, not tuning. And the question to ask first is always: what changed — parameters, data or traffic?

Architect

Memory is over budget but recall and latency are fine.

M first, because it is a pure memory saving and the recall cost can be partly bought back with efSearch
That trades memory for latency — exactly what you want when memory is the binding constraint
Then quantisation, which saves far more and costs more recall
Then the index type, the largest win and the largest blast radius

And I would flag that this is the only symptom where you deliberately accept lower recall. Everywhere else recall is the thing being protected; here it is the currency you are spending.

Eng manager

An engineer wants a week to tune the index.

It depends on whether we have a flat baseline and a gold set — without those, a week of tuning produces a number nobody can defend
If we do, a day is usually enough to sweep the live dial and find the knee, and that is often the whole win
What I would push back on is tuning before measuring

And I would ask what the scarce currency is. If memory is the constraint, no amount of live-dial tuning helps — live dials cost latency and never memory. A five-minute conversation that can save the week.

Fast answersthe eight that come up most

QCan I copy a configuration from a benchmark or a blog post?

As a starting point, not as an answer. The numbers encode somebody else’s scarce currency, corpus size, dimension and recall target. nlist/nprobe is the clearest example: what matters is the ratio and the fraction of the corpus it scans.

QDoes raising efSearch cost memory?

No — and this is load-bearing in the runbook. Live dials cost latency and nothing else, in every index here. The consequence: if memory is your binding constraint, no amount of live-dial tuning will help — and none of it will hurt either. Memory problems are always structural.

QHow do I choose efConstruction if build time is not a problem?

Higher, within reason, because it costs nothing at query time. The returns diminish sharply, so build at two or three values on a sample and compare tail recall, not mean recall — efConstruction shows up in the tail, which is exactly where the mean will not show it.

QShould nprobe be the same for every query?

Not necessarily, and this is a genuinely useful production pattern. nprobe goes in the request, so an interactive path can run cheap while an analytical path runs thorough — against the same index, with no second copy. Very few teams exploit it.

QWhat if I cannot rebuild the index at all?

Then you are limited to the live dial and to what sits outside the index: a reranker over a deeper candidate list, hybrid retrieval, better chunking. And name why you cannot rebuild — usually a node sized so tightly that a rebuild does not fit, which is a capacity decision made months earlier.

QHow often should the sweep be repeated?

When the data or traffic changes materially, and quarterly otherwise. The reason is drift: the operating point that hit 95% on last year’s corpus is not guaranteed to hit it on this year’s, and nothing will tell you. That is what the gold set in CI is for.

QIs there a single number that summarises index quality?

No — be suspicious of anyone offering one. The honest summary is a pair: recall at a stated latency, on your corpus. Latency without recall is meaningless because anything can be fast if allowed to be wrong; recall without latency is meaningless because anything can be accurate if allowed to be slow.

QMy engine hides these parameters. Is that better?

Better until it is not. Auto-tuning handles the common case and removes a real source of misconfiguration. What you lose is the ability to make the tradeoff deliberately — to say “memory is what is scarce here, so buy recall with latency instead”. Ask what the service optimises for, and whether you can see the recall it is achieving.

Name the scarce currency first, because it decides every value — and no default can know it. Then check the two constraints before measuring anything: efSearch at least k (and k is a hundred, not ten, if a reranker follows) and efConstruction at least four times M. Then sweep the live dial and read the shape of the curve, because the plateau is the diagnosis: still climbing means keep turning the free knob; flat below target while latency rises means no amount of search effort repairs a badly built graph. And plot the distribution, not the mean — users experience the tail.

Where this goes next

Where M, efSearch and nprobe come from 09 · Index IR, L and alpha, and why R fills a page 10 · Index IIThe dimension limit that surfaces at model selection 06 · VectorsRescore depth, and what it repairs 12 · CapacityWhy filters change which index can hit the target 14 · FilteringThe reranker that makes k a hundred 15 · HybridThe gold set and the recall gate in CI 16 · Proof

Questions to ask them

Which currency is scarce here — recall, latency or memory?

Is there a flat baseline, and a gold set?

Is the operating point pinned in config with a comment saying what recall it was chosen for?

Is efSearch sized against k — or against the reranker’s candidate pool?

Has anyone looked at the distribution of recall and latency, not the mean?

When did the sweep last run, and what changed since?