Track C · Document 14 · Serving at scale
One number decides every filtered query, and the failure mode is silence. Multi-tenancy is the same problem with the filter always present and the values wildly unequal.
Selectivity is the fraction of the table that passes the filter. Nothing else about filtered search matters until you know it — not the engine, not the parameters, not the index type. Two filters on the same table can need opposite strategies, and the only thing separating them is this number.
The failure is silent, and that is the whole reason this topic gets asked. A tight filter does not produce an error. The query runs, returns three rows instead of ten, and the application shows three. Nothing in the logs says “the other seven are in the table, the walk just never reached them”. It is a footgun, not a crash — and it is the single most common filtered-search bug in production RAG.
| Filter | Rows passing | Selectivity | One in every… | Candidates for 10 results |
|---|---|---|---|---|
| tenant = A, the largest tenant | 4,000,000 | 40% | 2.5 | 25 |
| department = radiology | 1,000,000 | 10% | 10 | 100 |
| published in the last 30 days | 100,000 | 1% | 100 | 1,000 |
| published since yesterday | 5,000 | 0.05% | 2,000 | 20,000 |
| document = one specific file | 50 | 0.0005% | 200,000 | 2,000,000 |
Read the last row. With fifty matching rows in ten million, one candidate in every two hundred thousand is a survivor — so you would need to walk to roughly two hundred thousand candidates before you expected to see one. No sensible ef gets there, and the graph is simply the wrong tool for that query.
Under interview pressure people say “add an index on tenant” and then reason about the HNSW graph. Those are different structures with different costs and different failure modes, and the whole topic collapses if they are confused.
| Word | What it means here | Not to be confused with |
|---|---|---|
| table | The store: ten million rows, each a vector plus its payload | The index |
| index | The ANN structure — the HNSW graph over the vectors | The payload index |
| payload | The non-vector fields stored with each row: tenant, doc_id, published, department | The filter |
| payload index | A secondary structure over one payload field, so a predicate can be answered without scanning ten million rows — a B-tree, hash, inverted list or bitmap | The ANN index. It does nothing to the graph |
| filter / predicate | The WHERE clause that arrives with the query. Not stored anywhere | The payload |
| selectivity | The fraction of rows passing the filter | — Postgres uses the same word the same way |
| candidate | A row the graph walk reached and scored | A survivor |
| survivor | A candidate that also passes the filter | A candidate |
| bitmap / allow-list | One bit per row, set if the row passes. Ten million bits is 1.25 MB | The payload index, which is what builds it |
| over-fetch | Asking the walk for more than k so that enough survive the filter | — |
Every engine in this document ships two or three of these five. Knowing which five exist — and which one a given engine reached for — is more useful than knowing any particular parameter name.
Keep two words apart under pressure. The index is the ANN structure — the HNSW graph over the vectors. The payload index is a secondary structure over one metadata field. “Add an index on tenant” means the second one, and it does nothing whatsoever to the graph.
Run the ordinary walk with no knowledge of the filter. It hands back ef
candidates in distance order. Apply the predicate to those. Return the first k survivors. The
graph is untouched, no engine support is needed, and the cost is exactly one walk plus
ef predicate checks.
It is also where most production filtered-search bugs live.
Where post-filtering is the right answer: above roughly 20 percent selectivity. Over-fetch by 1 ÷ selectivity with a safety factor of two and the cost is a slightly wider walk. pgvector’s default plan for a WHERE plus an ORDER BY on the vector column is exactly this, and for a tenant owning 40 percent of the table it works well with ef_search left at its default of 40.
Use the payload index to find the exact set of rows passing the filter. Compute the distance from the query vector to every one of them. Sort. Take k. The graph is not consulted at all, and that single fact gives this strategy a property no other one in this document has.
Every engine bakes its own version of this crossover into a knob. Qdrant’s full_scan_threshold defaults to 10,000 KB, which is about 1,700 vectors at 1536 dimensions. Weaviate’s flatSearchCutoff defaults to 40,000 objects. Lucene decides live: if the walk has visited more nodes than there are survivors, brute force is provably cheaper and it switches mid-query. pgvector alone has no such knob — it lets the Postgres planner estimate both costs from table statistics, which is why stale statistics are pgvector’s most common filtered-search failure.
Approximation only enters a vector search through the graph. Brute force never touches the graph, so it looked at every possible answer and the top 10 it returns is the top 10. It is the only technique in this document with a recall guarantee.
“Find the 50 survivors” means a payload index lookup. Without one it means reading ten million payloads, which at a few hundred nanoseconds each is seconds — and in pgvector the planner will not even consider the plan.
Two rules follow, and they are in tension. Index every field you filter on — Qdrant’s own FAQ calls this the single biggest speed-up for filtered queries, larger than any HNSW tuning. And do not index fields you never filter on, because each one is resident memory that could have been vectors.
Twenty departments means twenty bitmaps of 1.25 MB, ANDed in microseconds. Two hundred thousand document ids means a bitmap per value would be 250 GB, so it has to be an inverted list or a tree. Milvus makes the rule explicit — bitmap index below roughly 500 distinct values, inverted index above. If an interviewer asks which payload index type, cardinality is the answer.
One percent of ten million is a hundred thousand rows: too many to score by brute force, too few for a plain walk to find ten of. The engine keeps walking but checks the filter as it goes. Two variants exist and they are genuinely different mechanisms.
| Variant | How it works | Who |
|---|---|---|
| Stepping-stone | The walk visits non-matching neighbours to keep moving through the graph, but only matching ones may enter the result list. Non-matches are used as roads, never as destinations | Weaviate sweeping, Milvus bitset, Lucene pre-filter |
| Walk–filter–walk | A normal walk returns a batch; the batch is filtered; if short of k the walk resumes from where it stopped and returns the next batch. Repeat until k, or until a cap | pgvector iterative scan |
Both produce the same shape of cost: a walk roughly 1 ÷ selectivity longer than an unfiltered one, capped by a limit the engine imposes so that a pathological filter cannot walk the entire table.
The mechanism that makes “check the filter as you go” affordable is a bitmap: one bit per row, set if the row passes. Ten million rows is ten million bits is 1.25 MB, small enough to live in L2 cache. The walk tests one bit per neighbour visited — a fraction of a nanosecond, against the roughly 500 ns of the distance computation sitting beside it.
So the answer to “does the filter live in the graph node?” is no. The payload lives with the row, the filter arrives with the query, and the bitmap is built per query from the payload index and thrown away. The filter is effectively free per step; the entire cost is in the extra steps.
The payload index is resident, and the per-value lists inside it are resident. A bitmap for a single hot predicate — tenant A, asked five hundred times a second — can be cached, because it only changes when tenant A’s rows change. Combined predicates are rarely cached; the AND is cheap enough to redo.
That is the bridge into the second half of this document: a tenant filter is the one predicate every query carries, so it is the one worth keeping hot — or baking into the structure entirely.
This is the part that makes filtered vector search genuinely hard rather than merely fiddly, and it is the thing most candidates have never thought about.
payload_m parameter controls how many.Three shapes of answer, and the trade is when the work happens. Extra edges at build time means a bigger graph and a hard ordering constraint — the payload index must exist before the graph is built. Multi-hop at query time means more neighbour visits per step and a small recall cost, but nothing to rebuild. A bitset handed to the walk, with non-matches used as roads only, means walk length grows with 1 ÷ selectivity and you need a brute-force fallback underneath it.
Everything in the last five sections, in one pass, inside the engine.
Step 4 is the one that matters. The strategy is chosen per query, from the popcount — so the same collection answers a 40 percent filter and a 0.0005 percent filter by completely different mechanisms without the application knowing either happened. It also answers the question people ask about step 3: no, the filter does not live in the graph node. The payload lives with the row, the filter arrives with the query, and the bitmap is built per query and thrown away. The bit test itself is a fraction of a nanosecond against the ~500 ns of the distance computation beside it — the filter is effectively free per step, and the entire cost is in the extra steps.
pgvector is worth its own figure, because it is the one engine with no vector planner — the ordinary Postgres planner chooses between three plans using ordinary row-count statistics, and it has no idea what selectivity means for HNSW.
Postgres has no query hints, so you steer rather than choose. Three levers: indexes — Plan B exists only if a B-tree exists on the filter column; statistics — ANALYZE after any large load, because autovacuum’s threshold on a ten-million-row table is 10 percent by default and a 500,000-row load can leave estimates stale for hours; and session settings, which shape Plan A. Use SET LOCAL inside a transaction, never plain SET — behind a connection pool the session outlives your request and the next borrower silently inherits your ef_search.
A partial HNSW index WHERE tenant = 'A' gives that tenant a graph in which every
node qualifies: no over-fetch, no bit checks, and in-filter recall identical to unfiltered
recall. The planner matches the query’s WHERE to the index’s WHERE automatically.
On the reference stack a partial index for a 4-million-row tenant at int8 is about 6.1 GB of vectors plus 1.0 GB of graph, and roughly 35 minutes to build. Ten big tenants means ten graphs plus the shared one, each rebuilt on its own schedule, each competing for maintenance memory. It does not scale to ten thousand values — and that limitation is the doorway into the second half of this document.
| Engine | Graph-side mechanism | Brute-force switch | What you must declare | The gotcha |
|---|---|---|---|---|
| Qdrant | Filterable HNSW: payload-aware edges via payload_m; ACORN since 1.16 |
full_scan_threshold, 10,000 KB — about 1,700 vectors at
1536-d |
A payload index per filtered field, before ingest | An index declared after ingest needs a graph rebuild — it does not catch up on its own |
| Weaviate | Sweeping or ACORN over a Roaring allow-list | flatSearchCutoff, 40,000 objects |
indexFilterable on filtered properties — on by default |
Older clients still default to sweeping; check the collection config |
| Elasticsearch | Bitset pre-filter inside the kNN walk, per Lucene segment | Live: matches ≤ num_candidates, or nodes explored > matches | keyword / date mappings; the filter inside the knn clause | A bool.filter wrapped around a knn query is a post-filter and can
return fewer than k |
| Milvus | Bitset pre-filter (standard), or iterative filtering for expensive expressions | Segment-level, internal | Scalar index per field: bitmap under ~500 distinct, inverted above | Iterative filtering processes one entity at a time and is slow when many must be checked |
| Pinecone | Single-stage; per-cluster metadata statistics skip clusters with no possible match | Internal | Store unordered numerics (ids) as strings, or the min/max statistics are defeated | A very selective filter can leave the chosen clusters unable to fill top_k |
Per segment: if the matching document count is small, skip the graph and brute-force the matches. Otherwise walk the graph with the bitset — but if the number of nodes explored exceeds the number of survivors, abandon the walk and brute-force them instead, because at that point brute force is provably cheaper.
That is a live version of the crossover from section 5, decided mid-query rather than by a threshold set at configuration time. It is also why Elastic warns that, unlike ordinary queries, a more restrictive kNN filter can make the query slower.
Document 13 flagged this; here is the full picture. A filter’s selectivity is computed per shard, and hash sharding scatters every filter value evenly — so a 1 percent filter is 1 percent on each of four shards. Each shard walks its own graph under the same tight filter. The over-fetch and the connectivity problem are paid four times.
When the filter key is the shard key, filtering becomes routing and costs nothing — Qdrant shard keys, Weaviate multi-tenancy, Pinecone namespaces, Postgres partitions by tenant.
When a filter is tight on the whole table but each shard can independently choose to brute-force its own survivors — Lucene’s per-segment rule — sharding parallelises the brute force.
Everything else, sharding makes worse. Fan-out plus a tight filter is the worst combination in this track: you pay the slowest-shard tail and the extended walk on every one of them.
Table partitioning by tenant gives one HNSW index per partition, and a query with
WHERE tenant = 'A' is pruned to one partition and walks a graph in which everything
qualifies. The planner manages the set, so you do not hand-maintain ten indexes.
The cost is the same skew problem: tenant A’s partition is four million rows and tenant Z’s is four hundred. And partition pruning needs the literal tenant in the WHERE — thousands of partitions also raise planning cost, so keep it to tens or low hundreds.
Everything about multi-tenancy follows from one decision: at what layer of the store is “tenant” enforced? There are exactly three places it can go, and they are the same three answers the first half of this document gave for any filter, moved one layer up.
What makes tenancy its own topic is that the filter value is always present, always equality, always the same column — and the values are wildly unequal in size. That last property is what turns a filter problem into a design problem.
The “native” versions are Model 3 with the operational cost engineered down. Weaviate’s tenant, Pinecone’s namespace and Qdrant’s dedicated shard are all a private room that is cheap to keep, can be turned off when the family is away, and shares one front desk and one schema with every other room. That is why the vendor answer to “how many tenants?” moved from “a few hundred” to 100,000 namespaces per index (Pinecone) and a stated design target of millions (Weaviate).
| Need | In plain English | Which model delivers it for free |
|---|---|---|
| Data isolation | I never see your rows | 3, and 2 physically. 1 only if the filter is never omitted |
| Recall parity | My search works as well as yours, whatever my size | 3. 1 only with a brute-force fallback |
| Performance isolation | Your traffic does not slow me down | None fully — this lives above the store |
| Lifecycle | Onboard me in seconds, delete me completely | 3, where deletion is a DROP. 1 and 2 pay for deletion later |
An interviewer will usually ask about the first two. Raising the last two unprompted is what separates an answer from a good answer.
Nothing about multi-tenancy makes sense until you hold one table in your head: twenty tenants own two thirds of the data, and nearly ten thousand share the remaining seventh.
The line is not “when the tenant has more data than a graph’s overhead”. It is when the tenant leaves the brute-force band — because below that a shared table answers it exactly, in microseconds, and above it a filtered walk starts costing recall. Qdrant’s ~20,000-point recommendation and Weaviate’s 10,000-object flat-to-HNSW default both sit at exactly that boundary. They are the crossover from earlier in this document, applied per tenant.
| Tenant | Rows | Selectivity | Survivors in 40 candidates | Candidates for 10 | Verdict |
|---|---|---|---|---|---|
| whale | 4,000,000 | 40% | 16 | 25 | A plain walk is fine |
| large | 150,000 | 1.5% | 0.6 | 667 | Needs an iterative scan |
| medium | 10,000 | 0.1% | 0.04 | 10,000 | Right at the brute-force line |
| small | 138 | 0.0014% | 0.00055 | 724,638 | A walk is useless — and brute force is exact in microseconds |
A 138-row tenant is 138 × 1,536 = about 212,000 multiply-adds. That is microseconds, and it is exact. The tail is easy once the engine has a brute-force fallback.
The hard band is the middle — tenants of ten thousand to a couple of million rows, too big to brute-force cheaply and too small for a plain walk. On the reference stack that is 180 tenants carrying 18 percent of the data, and they are precisely the ones nobody notices.
“Our recall@10 is 0.96” is a fine sentence for a single-tenant store. Here it hides everything: the tail is 1.00 because it is brute-forced, the whale is 0.96 because it gets a plain walk, and the middle tier can sit between 0.85 and 0.97 depending on connectivity.
The dashboard number is the whale’s number, because the whale is 40 percent of the queries. The medium tier is 1.8 percent of traffic, so it can be measurably worse forever and never move the average.
Measure per tenant and report the distribution — p50 and p10 across tenants, not the mean. An interviewer who hears “we track p10 recall across tenants” knows you have run one of these systems.
Rooms for the few families with thousands of books; one shared hall with name tags for everyone else; a rule at the door about how many requests each family may make. Every major vendor has now built some version of exactly that.
| Engine | Name | Mechanism | Threshold and limits |
|---|---|---|---|
| Qdrant | Tiered multitenancy, v1.16 | Custom sharding; a shared fallback shard holds small tenants, large tenants get a dedicated shard, and tenant promotion moves one from fallback to dedicated using the shard-transfer mechanism — reads and writes continuing throughout. Every request carries a shard-key selector naming both | Recommended promotion at ~20,000 points. Stay under about 1,000 dedicated shards per cluster |
| Weaviate | Native multi-tenancy + dynamic index | Every tenant is its own shard. With a dynamic index a tenant starts flat — vectors on disk, brute force — and converts, one-way, to HNSW when it crosses the threshold. Idle tenants go INACTIVE (disk) or OFFLOADED (S3) | Default conversion at 10,000 objects. Dynamic index requires async indexing |
| pgvector | Hand-built | Shared table with a B-tree on tenant_id and iterative scan for the middle; the planner picks Plan B for tiny tenants; a partial HNSW index or a LIST partition per whale | The threshold is yours. A partial index per tenant stops being sane past a few dozen |
| Milvus | Partition key + Partition Key Isolation | Hash tenants into num_partitions (default 16); with isolation enabled
Milvus builds a separate sub-index per key value and searches only that |
1,024 manual partitions per collection; collections advised under about 1,000 |
| Pinecone | Namespaces | One namespace per tenant on a serverless index, physically separate storage. No shared graph, so no size tiering is needed on the read path | 100,000 namespaces per index on Standard and Enterprise |
“Promote a tenant to its own graph when it leaves the brute-force band — around ten to twenty thousand rows — because below that a shared table answers it exactly in microseconds, and above that a filtered walk starts costing recall. Both Qdrant and Weaviate default to that line.”
Note what that is not: it is not “when the tenant has more data than a graph’s overhead”. Overhead sets an upper bound on how many graphs you can afford; the crossover sets where each one earns its place.
Resident memory, per replica at int8: a small tenant is 138 × 1,536 = 0.2 MB plus about a kilobyte of bitmap. A medium tenant is 15 MB. A large tenant with its own graph is about 240 MB plus fixed overhead. The whale is 6.4 GB.
Compute per query: the small tenant is 212 thousand multiply-adds. The whale is two to eight million. A tail tenant is cheaper per query than the whale, not more expensive.
Operations per tenant: in the shared graph, none — a row is a row. With its own graph, one more thing in every loop: rebuild, backup, health check, monitoring, migration. That is the real cost of the tail, and it is the whole argument for the shared hall.
An uncompressed bitmap over ten million rows is 1.25 MB per tenant regardless of its size — ten thousand of those is 12.5 GB, which is why nobody stores them uncompressed. Roaring compression makes a 138-row tenant about a kilobyte, a 10,000-row tenant about 20 KB, and the whale under a megabyte. All ten thousand tenants come to roughly 20 MB, resident.
That is why dedicated engines can afford to keep the membership sets persistent, and why the Postgres approach — materialise the bitmap per query from the B-tree — is fine for one tenant filter but adds work on every single call.
People say “tenant isolation” and mean either of two unrelated guarantees. An architect answer names both. An engineering-manager answer adds who owns each.
| Data isolation | Performance isolation | |
|---|---|---|
| The guarantee | Tenant Z never receives a row of tenant A | Tenant A’s traffic never slows tenant Z |
| Where it is enforced | In the store — filter, route or collection — and in the read-time join | Above the store: API gateway, queue, quota. Or by physical separation |
| Failure mode | A leak — a security incident | A slowdown — an SLA incident |
| Which model helps | 3 > 2 > 1 | None fully. Dedicated replicas help; rate limits are the real fix |
| Who owns it | The platform team, and they must be able to prove it | The API team — it is a product policy, not a store property |
Every tenant’s query lands in the same queue and competes for the same CPU and page cache. A whale at high QPS with wide walks fills the queue, and a tail tenant’s microsecond query waits tens of milliseconds behind it. A private room fixes what you find; only a rule at the door fixes how long you wait.
| Engine | Performance-isolation primitive | What it actually gives you |
|---|---|---|
| Qdrant | A dedicated shard per large tenant | True for I/O and graph pages. CPU and network on a shared node are still shared unless the shard is placed on its own node |
| Weaviate | Tenant states — an inactive tenant consumes nothing | Protects RAM. Does nothing for QPS |
| Pinecone | Namespaces isolate storage | Read and write unit limits apply per index, not per namespace. Enforce per tenant yourself |
| Elasticsearch | Index per tenant on separate node roles | Physical, coarse and expensive |
| Postgres | Nothing per tenant | Connection pools, statement_timeout, and separate replicas. Route the
whale’s read traffic to its own replica |
None of them gives a per-tenant CPU share inside a shared collection. That is the statement to make, and then qualify — because the honest fix is a per-tenant rate limit and a fair scheduler at the API layer, plus physical separation for the whale.
In the shared-table model the entire security boundary is one predicate that application code must remember to add to every query. Code paths multiply — admin tools, batch jobs, a new endpoint, a debugging script — and one of them will forget.
The policy attaches the tenant predicate to the table; the application only sets a session
variable. ALTER TABLE ... ENABLE ROW LEVEL SECURITY, then FORCE ROW LEVEL
SECURITY so the table owner is subject to it too, then a policy using
current_setting('app.tenant_id').
First probe: RLS gives correctness, not recall. The policy is inlined into the plan as an ordinary Filter, so the nearest-neighbour query still walks the index broadly and filters afterwards — it is Plan A with a predicate you cannot forget. Pair it with iterative scan, a B-tree on tenant_id, and partial indexes or partitions for the big tenants.
Second probe: SET against SET LOCAL. A plain SET
survives the transaction and leaks to the next borrower of a pooled connection — which in
a multi-tenant system is a cross-tenant read waiting to happen. Always SET LOCAL,
inside a transaction.
Whatever the vector store returns, the last step re-selects those ids from the relational system of record with the tenant predicate applied. If the store leaked, the join drops the leak. The vector store is a cache of candidate ids; the database is the source of truth for which tenant may see them.
And it fixes the other drift problem too: a document deleted in Postgres leaves its vectors behind, and a tenant reassignment leaves the old tenant in the payload. Search then returns ids that no longer exist or that the user must not see — and the join catches both.
A subtle class of bug: the tenant is resolved from the session at request start, but a cache key, a background job or an async continuation runs without it. The rule is that tenant id is a required parameter of every retrieval function, never something read from ambient state. Cache keys include it, queues carry it, logs print it.
The two lifecycle questions almost nobody prepares for — and the two an experienced interviewer reaches for once the design questions are answered.
In any SaaS, most tenants are idle most of the time. The shared-table model does not care — an idle tenant is idle rows in a graph that was loaded anyway. A collection per tenant cares a great deal, and Weaviate has the most developed answer.
| State | Where the data is | RAM | Local disk | A query |
|---|---|---|---|---|
| ACTIVE | Loaded | yes | yes | Served |
| INACTIVE | Local disk only | no | yes | Error, or auto-activate |
| OFFLOADED | Cloud object storage | no | no | Error, or auto-activate |
HOT and COLD were renamed ACTIVE and INACTIVE in v1.26. Offloading needs the S3 module. State
changes are eventually consistent across a cluster, so data may not be immediately
available after reactivation. Backups include only ACTIVE tenants. And
auto_tenant_activation reloads a tenant on first read or write, at the price of a
cold-start latency on that request.
That last one is the same cold-cache trade as document 12, at tenant granularity: the cheapest idle tenant is the one whose first request is slow.
| Engine | How it promotes | Downtime |
|---|---|---|
| Qdrant | Tenant promotion from the fallback shard to a new dedicated shard, using shard transfer | None — reads and writes continue during the move |
| Weaviate | The dynamic index converts the tenant’s flat index to HNSW at the threshold | None; async indexing builds the graph in the background. One-way |
| pgvector | You do it: CREATE INDEX CONCURRENTLY for a partial
index, or detach and attach into a dedicated partition |
Concurrent build avoids write locks; moving rows between partitions is a copy |
| Milvus | Move to a manual partition or collection — an application-level re-insert | Application-managed |
| Pinecone | Not needed — namespaces never shared a graph | — |
“Delete everything about customer Z” arrives from legal with a deadline. The isolation model decides whether that is a one-second DROP or a compaction you have to schedule and then prove.
| Engine | Delete tenant Z | When the bytes are actually gone |
|---|---|---|
| pgvector | DELETE ... WHERE tenant_id = 'Z', or
DROP TABLE for a partition |
After VACUUM removes the dead tuples and the HNSW index’s dead entries. A partition drop is immediate |
| Qdrant | Delete points by filter, or delete the shard key | Filter delete: after the optimizer rewrites segments. Shard delete: immediate |
| Weaviate | tenants.remove([Z]) |
Immediate — the shard is deleted |
| Milvus | Delete by expression, or drop the partition / collection | Expression delete: after compaction. Drop: immediate |
| Elasticsearch | Delete-by-query, or delete the tenant’s index | Query delete: after a segment merge. Index delete: immediate |
Primary store rows deleted, physically compacted, count verified. Every replica compacted — RF 3 means three copies of the tombstones. Payload and bitmap entries for Z removed. Outbox and reconciliation queues drained. Application and result caches purged of Z’s ids. Backups and snapshots: retention window documented, or rebuilt without Z. Logs: Z’s query text and ids under a retention policy. And a signed record of all of it for the request file.
Being able to produce that list is the difference between a plausible answer and one that has survived an audit.
Support staff, analytics, abuse scans and duplicate detection all want to search across tenants, and it is the case the collection-per-tenant model handles worst: Model 1 drops the WHERE and it is free; Model 2 scatters to every bay; Model 3 scatters to ten thousand collections, and Pinecone has no single cross-namespace query at all.
The usual answer is a second, thinner shared table for that use case — often a smaller embedding or a sample — or accepting that the cross-tenant path is an offline job. Do not let a rare admin feature force the whole system into Model 1.
Almost every symptom here presents as “search is bad for some customers”, which is why the diagnosis has to start from selectivity rather than from the complaint.
| Symptom | Likely cause | What to check | Fix |
|---|---|---|---|
| A filtered query returns fewer than k rows | Post-filtering with a tight filter — the classic silent shortfall | Selectivity, and Rows Removed by Filter in EXPLAIN ANALYZE |
Raise ef_search, enable iterative scan, or add the payload index that unlocks brute force |
| A more restrictive filter made the query slower | Working as designed — the walk has to explore further to gather k that pass | Whether the engine has a brute-force fallback and where its threshold sits | Lower the threshold so brute force takes over, or route by the key |
| Filtered queries are slow and no tuning helps | No payload index on the filter field | Declared payload indexes against the fields actually filtered on | Index every field you filter on — and in Qdrant, rebuild the graph afterwards |
| Qdrant filtered search stayed slow after adding a payload index | The index was declared after ingestion, so the payload-aware edges were never built | Whether the collection has been rebuilt since | Force a rebuild — set m to 0 and back. On a large collection this is a full re-index, which is why the order matters on day one |
| pgvector picks the wrong plan after a bulk load | Stale statistics — the planner is exactly as smart as pg_stats | ANALYZE timing against the load; autovacuum’s 10 percent threshold on
a 10M-row table |
ANALYZE after every large load, and put it in the pipeline rather than the runbook |
| ef_search changes leak between requests | Plain SET behind a connection pool |
Whether the session setting is inside a transaction | SET LOCAL, always |
| Iterative scan is enabled and still returns short | The 20,000-tuple cap, or a CTE boundary between the WHERE and the ORDER BY | Query shape and max_scan_tuples |
Below about 5,000 matching rows you want Plan B, which means a B-tree on the filter column |
| Elasticsearch kNN returns fewer than k with a filter | The filter is in bool.filter around the knn query — that is a
post-filter | Where the filter clause sits | Move it inside the knn clause |
| Recall is fine on the dashboard, bad for specific customers | Per-tenant recall variance hidden by an average the whale dominates | Recall per tenant, reported as a distribution | Track p10 across tenants; promote the middle band |
| One tenant’s queries slow everyone down | No performance isolation — there is none inside a shared collection | Per-tenant QPS and walk width | Per-tenant rate limits at the API layer; a dedicated replica for the whale |
| A tenant was deleted and memory did not drop | Tombstones — a filter delete frees nothing until compaction | Deleted-vector count against live count | Trigger compaction and verify with a count. For an erasure request, work the whole checklist |
| Cross-tenant admin search times out | Model 3: it is a scatter across every collection | How many collections the query touches | A thin shared table for the admin path, or make it an offline job |
ArchitectHow does a filtered vector search work?
The first thing I would establish is selectivity, because nothing else matters until you know it. Survivors equals candidates times selectivity, so a walk returning forty candidates under a one-percent filter yields 0.4 survivors on average — and the query does not fail, it just returns fewer rows than asked for.
From there there are three bands. Above roughly twenty percent, run a plain walk and post-filter, over-fetching by one over selectivity. Below roughly a tenth of a percent, ignore the graph entirely: use the payload index to get the exact survivor set and score them all, which is faster and exact. In between, the engine filters during the walk, using a bitmap built per query from the payload index, and the walk grows by about one over selectivity.
The band boundaries are not constants — they are where two cost curves cross, and every engine bakes its own version into a knob: Qdrant’s full_scan_threshold, Weaviate’s flatSearchCutoff, and Lucene deciding live mid-query.
ArchitectA user reports that search returns three results instead of ten. Where do you look?
At the filter’s selectivity first, because this is almost always
post-filtering with a tight predicate. The walk returned its forty candidates without knowing
there was a filter, seven of them failed it, and the application showed what survived. There is
no error and no log line — in pgvector the tell is Rows Removed by Filter: 39
in EXPLAIN ANALYZE, and nobody is looking at query plans when the report says “search
feels thin”.
The escalation is cheapest first: raise ef_search, which is roughly linear and needs no schema change; enable iterative scan if the engine has it; then add a payload index on the filter column, which is what actually unlocks the brute-force plan for tight filters.
And I would treat it as a monitoring gap as much as a bug. A query that returns fewer rows than k is a signal the application can emit, and most systems throw it away.
ArchitectWhy can a more restrictive filter make a query slower?
Because the graph was built for one geometry — embedding similarity — and the filter imposes a second one the graph knows nothing about. A tighter filter means the walk has to explore further to gather k candidates that pass, so work goes up as the result set goes down. Elastic documents this explicitly as a difference from ordinary queries.
Underneath it is the connectivity problem. A tenant’s chunks are scattered across embedding space by topic, so the subgraph of matching nodes is usually disconnected — islands joined only through nodes that fail the filter. A walk that refuses to step on non-matches gets stranded; a walk that ignores the filter wastes almost every step.
The ways out are the three vendors have built: extra payload-aware edges at build time, which is Qdrant’s filterable HNSW; multi-hop traversal through non-matches at query time, which is ACORN; or a fallback to brute force once the walk has visited more nodes than there are survivors, which is what Lucene does.
ArchitectWhen is brute force the right answer for a vector search?
Whenever the survivor set is small enough that scoring it costs less than a graph walk — which on our stack is a few thousand rows. At fifty or five hundred survivors it is not merely acceptable, it is faster than the graph: fifty distances at 1536 dimensions is about 77,000 multiply-adds, against two to eight million for one HNSW walk.
And it has a property nothing else here has: recall is exactly 1.00. Approximation enters a vector search only through the graph, and brute force never touches it.
The precondition is a payload index on the filter column. Without one, finding the survivors means reading ten million payloads, and in pgvector the planner will not even consider the plan. That is why “index every field you filter on” is the highest-leverage move in filtered search — larger, by Qdrant’s own account, than any HNSW tuning.
ArchitectHow would you design multi-tenancy for ten thousand tenants?
Tiered, and the tiers come from the size distribution rather than from a preference. On a realistic stack one tenant owns forty percent of the data, nineteen more own another twenty-eight, a hundred and eighty are around ten thousand rows each, and nearly ten thousand have a couple of hundred rows.
So: the whale and the large tenants get their own graph — a dedicated shard, a partition, or a partial index — and their queries walk unfiltered. Everyone else lives in one shared graph with a tenant payload index. The tail is answered by brute force over a few hundred rows, which is exact and takes microseconds; the middle band gets a filtered walk.
Twenty graphs is nothing to operate. Ten thousand would have been the problem — Qdrant Cloud caps collections at a thousand per cluster, Milvus advises the same, and Elasticsearch’s guidance works out to about 620 shards on a 31 GB heap. A collection is the unit of schema and operations, not the unit of tenancy.
And the promotion line is around ten to twenty thousand rows, because that is where a tenant leaves the brute-force band. Both Qdrant and Weaviate default to exactly that.
ArchitectYour recall dashboard says 0.96 and a customer says search is broken. Both are true. Explain.
The dashboard is showing the whale’s number. In a multi-tenant store recall stops being one number: the tail is 1.00 because it is brute-forced, the whale gets a plain walk at 0.96, and the middle band — tenants of ten thousand to a couple of million rows — can sit anywhere from 0.85 upward depending on how connected their subgraph happens to be.
The whale is forty percent of the queries, so it dominates the average. The medium tier is under two percent of traffic, which means a hundred and eighty tenants can be measurably worse forever without moving the number anyone looks at.
The fix is measurement before mechanism: per-tenant recall against a held-out set, reported as a distribution — p50 and p10 across tenants, not the mean. Then promote the tenants that are failing, because moving a tenant to its own graph turns a filtered walk into an unfiltered one and its recall rises to whatever the graph parameters give.
ArchitectIs a tenant filter enough for security?
No, and I would be firm about that. In the shared-table model the entire boundary is one predicate that application code has to remember on every query, and the code paths multiply — admin tools, batch jobs, a new endpoint, a debugging script. One of them will forget.
There are four layers and the filter is the weakest. Strongest is a tenant-scoped credential, where the store itself refuses other tenants. Then row-level security, where the database adds the predicate and forgetting is impossible. Then the read-time join — the final SELECT against the system of record with the tenant predicate, which is the only layer correct by construction, because if the store leaked, the join drops the leak.
Two details I would raise. RLS gives correctness, not recall — it is inlined
as an ordinary filter, so it is Plan A with a predicate you cannot forget, and it still needs
the iterative scan and the B-tree underneath it. And SET LOCAL, never plain
SET, because behind a connection pool a session setting outlives the request and
becomes a cross-tenant read waiting to happen.
ArchitectLegal asks you to delete a tenant completely. Walk me through it.
The first thing I would say is that the answer depends on the isolation model we chose months earlier. With a collection, tenant or namespace per tenant it is a drop — Weaviate deletes the shard and the objects with it. In a shared graph a delete marks rows dead and the vector bytes stay in the index file, exactly like every other soft delete in this system.
So the work is proving the bytes are gone: physical compaction, verified by a count, on the primary and on every replica — replication factor three means three copies of the tombstones. Then payload and bitmap entries, outbox and reconciliation queues, application and result caches, and backups, where the honest answer is usually a documented retention window rather than a rebuild.
And a signed record of all of it for the request file, because the deliverable is evidence, not a delete statement.
Eng managerYour team wants to give every enterprise customer their own collection. How do you respond?
By asking how many customers we expect and what we are actually buying. For twenty large customers it is a reasonable design and the isolation story is genuinely simpler. For ten thousand it is not a design, it is ten thousand things to create, migrate, back up, monitor and rebuild — and every vendor has published a limit that says so.
Then I would reframe it as a tiering question rather than a yes or no, because the tiered answer usually satisfies what the team actually wants: dedicated graphs for the large customers, a shared graph for the long tail, and a documented promotion threshold. That is also a shape the price list can follow, which tends to end the argument.
What I would insist on is that whatever we choose, the security boundary does not rest on the collection choice alone — there is a read-time join or a scoped credential underneath it either way.
Eng managerA customer complains that search is slow only during business hours. What is your first move?
Look at whether it is their query getting slower or their query waiting. Those have completely different owners. If their own latency is stable but their end-to-end time rises, that is queueing behind someone else — the noisy-neighbour problem — and no store in this space gives a per-tenant CPU share inside a shared collection.
So the fix is not usually in the database. It is per-tenant rate limits and a fair scheduler at the API layer, and for a genuinely large tenant, physical separation — its own replica, its own shard, its own node.
The management point is that this is a product policy, not a platform property. Data isolation is something the store team proves; performance isolation is something the API team enforces. If nobody owns the second one, it does not exist, and the incident recurs every quarter with a different customer name on it.
Eng managerHow do you stop filtered-search bugs from reaching customers?
By making the silent failure loud. The defining property of this whole area is that a tight filter returns three rows instead of ten and nothing complains — so the first change I would make is an application-level signal whenever a query returns fewer than k results, tagged with the tenant and the filter shape. That single metric would have caught most of the incidents in this document.
Second, a per-tenant recall test in CI rather than a global one, reported as a distribution. A global average hides exactly the tenants who are suffering, because the largest tenant dominates it.
Third, a checklist item that survives people leaving: index every field we filter on, ANALYZE after every bulk load, and in Qdrant declare payload indexes before ingestion. Those three are cheap, they are easy to forget, and each of them silently costs recall rather than raising an error.
Does the filter live inside the graph node?
No. The payload lives with the row, the filter arrives with the query, and the bitmap is built per query from the payload index and thrown away afterwards. The walk tests one bit per neighbour — a fraction of a nanosecond against the roughly 500 ns of the distance computation beside it. The filter is effectively free per step; the entire cost is in the extra steps.
Should I pre-filter or post-filter?
The words are used inconsistently, so answer with selectivity instead. Above roughly twenty percent, post-filtering is right and cheap. Below roughly a tenth of a percent, you want the payload index and brute force. In between you want the filter applied during the walk. Note that Weaviate calls all of its filtered search “pre-filtering” because the allow-list is computed first, and Elasticsearch means something quite different by the same word — which is why the number is a better answer than the label.
Why is my payload index not helping?
Three usual causes. It is on a field you do not actually filter on. Or in Qdrant it was declared after ingestion, so the payload-aware edges were never built and the graph needs a rebuild. Or the filter is loose enough that the engine correctly ignored it — a payload index does nothing for a forty-percent filter, because a plain walk was always going to work.
Can I just raise ef_search until filtered queries work?
Up to a point, and the point arrives fast. At one percent selectivity you need about a thousand candidates for ten results, which is twenty-five times the default work. At 0.05 percent you need twenty thousand, and at 0.0005 percent you need two million — a fifth of the table. Raising ef is the right first move because it is free to test and free to undo; it is not a strategy for tight filters.
How many payload indexes is too many?
The rule is symmetric: index every field you filter on, and index nothing else. Each one is permanently resident memory that could have been vectors — six or seven fields on ten million rows lands near the 3 GB payload line in the reference stack. The growth path is the thing to watch: every new filterable field the product team requests is memory forever, and nobody usually tells them.
Is one tenant per collection ever right?
Yes, for tens of tenants, and increasingly for thousands if the engine has a native lightweight tenant. Weaviate’s tenants, Pinecone’s namespaces and Qdrant’s dedicated shards are all Model 3 with the fixed cost engineered down — which is why the vendor answer moved from “a few hundred” to 100,000 namespaces per index. What is still wrong is ten thousand ordinary collections, which every vendor documents as unsupported in one form or another.
What does one more tenant actually cost?
It depends entirely on which tier they land in, and the counter-intuitive part is that a tail tenant is cheaper per query than the whale: 212 thousand multiply-adds against two to eight million. What the tail costs is not compute — it is the operational and memory overhead if you give each of them a graph. In the shared graph a row is just a row.
Do I need a separate vector database at all?
Under a few million vectors with a handful of filter fields, usually not — pgvector sits next to the tables you are joining against, filters are ordinary columns, and brute force is often fine. The honest framing is: search within an application, pgvector; search as the application, a dedicated engine. pgvector does not break at ten million, it stops being free — a dedicated instance, maintenance_work_mem tuning, builds competing with transactional queries, no forced plan choice, and one graph per table.
What only Postgres can do here?
Joins — vectors plus permissions plus employment status in one statement; Qdrant cannot join. Transactions — insert a document, its chunks and its vectors atomically. And the operational fact that it is already there, with backups, monitoring, access control and people who know it. At scale the common pattern keeps both: Postgres as the system of record, the vector store holding vectors plus a thin payload, and a two-hop query. The vector database proposes; the relational database disposes.
How do the two stores drift, and what do you do about it?
A document deleted in Postgres leaves its vectors behind; a tenant reassignment leaves the old tenant in the payload. Search then returns ids that no longer exist or that the user must not see. Three layers, and mature systems run all three: a transactional outbox so writes propagate reliably, a periodic reconciliation job that compares id sets, and the read-time join, which makes drift harmless at query time even when it exists.
| Engine | Knob | Default |
|---|---|---|
| pgvector | hnsw.ef_search | 40 |
| pgvector | hnsw.iterative_scan | off |
| pgvector | hnsw.max_scan_tuples | 20,000 |
| Qdrant | full_scan_threshold |
10,000 KB ≈ 1,700 vectors at 1536-d |
| Weaviate | flatSearchCutoff | 40,000 objects |
| Weaviate | filterStrategy |
acorn, since v1.34 |
| Elasticsearch | num_candidates |
1.5 × k, max 10,000 |
| Weaviate | flat → HNSW conversion | 10,000 objects |
| Qdrant | tenant promotion | ~20,000 points |
| Milvus | bitmap vs inverted payload index | ~500 distinct values |
“Filtered vector search is one number: selectivity. Survivors equals candidates times selectivity, so above about twenty percent you run a plain walk and post-filter, below about a tenth of a percent you skip the graph entirely and score the survivors from the payload index — which is both faster and exact — and in between the engine filters during the walk using a bitmap, with the walk growing by one over selectivity.
The reason it is hard rather than fiddly is connectivity. The graph was built for embedding similarity and the filter imposes a second geometry it knows nothing about, so the matching subgraph is usually disconnected. The three answers are payload-aware edges at build time, multi-hop traversal at query time, and falling back to brute force once the walk has visited more nodes than there are survivors.
Multi-tenancy is the same problem where the filter is always present and the values are wildly unequal. So the design is tiered: dedicated graphs for the twenty tenants that own two thirds of the data, one shared graph with a tenant payload index for the ten thousand that share the rest, and a promotion threshold around ten to twenty thousand rows — which is exactly where a tenant leaves the brute-force band.
Two things I would raise unprompted. The failure mode is silent: a tight filter returns three rows instead of ten with no error anywhere, which is why a ‘returned fewer than k’ metric is worth more than any tuning. And recall stops being one number — the dashboard shows the largest tenant’s figure, so you track the distribution across tenants or you do not know.”
| Thread from this document | Resolved in |
|---|---|
| Why deletes leave tombstones and what compaction does | 03 · Identity, updates and deletes |
| Access control at ingestion, and the permission model behind the filter | 04 · Access control and freshness |
| The HNSW walk being filtered, and why insert is a search | 09 · Flat, IVF and HNSW |
| ef_search, k and the parameter budget these filters spend | 11 · Parameters and tuning |
| Why a heavy filter breaks product quantisation’s lookup-table amortisation | 12 · Quantisation and capacity |
| Shard keys, hot shards and the fan-out this section inherits | 13 · Sharding and replication |
| Per-tenant recall measurement, and the CI gate that catches drift | 16 · Evaluation and observability |