Embeddings are getting larger — 1024, 1536, 3072 dimensions and up. More dimensions can mean better retrieval, but every dimension is storage you pay for on every vector and compute you spend on every comparison. At scale, that adds up fast: a corpus of tens of millions of high-dimensional vectors is a real storage and latency bill. Dimension reduction is how you claw that back — shrink the vectors so they're cheaper to store and faster to search, ideally without giving up much quality. Three techniques dominate, and they work in completely different ways.
Why size costs you
Two costs scale with dimension:
- Storage. A vector's size is roughly its dimension times its precision. Double the dimensions, double the storage — across every vector in your corpus. At scale this is the dominant cost, and Kognita's dimension-specific vector tables exist precisely because dimension count is a first-order storage decision.
- Search compute. Every similarity comparison is math over all the dimensions. Fewer dimensions means faster comparisons, which means lower query latency and higher throughput.
So reducing dimensions attacks both the storage bill and search speed at once. The whole game is doing it without losing the retrieval quality that made you pick a big embedding in the first place.
PCA: project onto the directions that matter
Principal Component Analysis is the classic post-hoc reduction. You analyze your existing vectors, find the directions along which they vary most, and project them onto the top-K of those directions — discarding the axes that carried little information. A 1536-dim vector becomes, say, 512-dim.
- Pros: works on embeddings you already have, from any model. No special model support needed.
- Cons: it's a lossy, data-dependent transform you compute and maintain. If your data distribution shifts, the projection can go stale. And you have to store the projection and apply it to every query vector too.
PCA is the pragmatic "I have big vectors already and want them smaller" option.
Matryoshka: embeddings that truncate cleanly
Matryoshka representation learning bakes reducibility into the embedding model itself. The model is trained so that the most important information is packed into the earliest dimensions. That means you can simply truncate the vector — keep the first 256 of 1024 dimensions — and still have a valid, useful embedding. Like nested Russian dolls, a smaller version lives inside the larger one.
- Pros: dead simple to use — truncation is free, no projection to compute or maintain, and you can choose the trade-off per use case (short vectors for a fast first pass, full vectors when needed).
- Cons: requires an embedding model that was trained to be Matryoshka. You can't truncate an arbitrary embedding this way and expect it to work — the property has to be built in.
When your embedding model supports it, Matryoshka is often the cleanest option because the reduction is built into the representation.
Binary quantization: from floats to bits
Binary quantization is the aggressive one. Instead of reducing dimension count, it reduces precision — collapsing each dimension from a 32-bit float to a single bit (positive → 1, negative → 0). This shrinks a vector by around 32× and makes comparisons extremely fast (bit operations instead of float math).
- Pros: enormous storage savings and very fast search. At large scale the reduction is dramatic.
- Cons: it's lossy — you're throwing away most of the numeric detail. Accuracy drops. The standard fix is a two-pass approach: search fast over the binary vectors to get a candidate set, then rerank those candidates using the full-precision vectors (or a cross-encoder). You get binary's speed for the coarse pass and float-level accuracy for the final ranking.
Binary quantization shines at massive scale where its storage and speed wins are worth the extra rerank step.
Choosing among them
| PCA | Matryoshka | Binary quantization | |
|---|---|---|---|
| Mechanism | Project to fewer axes | Truncate trained-in dims | Floats → bits |
| Works on any model | Yes | Only Matryoshka-trained | Yes |
| Effort | Compute/maintain projection | Free truncation | Add a rerank pass |
| Best win | Moderate size cut | Clean, flexible size cut | Extreme storage/speed at scale |
A practical order of preference:
- If your embedding model is Matryoshka-trained, use truncation. It's the simplest and most flexible — no extra machinery, tune the trade-off freely.
- At very large scale where storage/latency dominate, use binary quantization with a full-precision rerank pass. The two-pass pattern recovers most of the lost accuracy.
- Reach for PCA when you have big vectors from a non-Matryoshka model and want a moderate reduction without changing your embedding stack.
The unifying principle: never assume you must store and search full-precision, full-dimension vectors just because that's what the model emitted. There's almost always headroom to make your index cheaper and faster — the art is picking the reduction whose particular loss your quality bar can absorb, and (for the aggressive options) pairing a cheap coarse pass with an accurate rerank to get the best of both.