Kognita
All posts
Engineering2026-09-01·8 min read

The real cost of vector storage at scale: estimating your bill

Vector storage looks cheap until you multiply it out. Here's a practical model for forecasting what your embeddings will actually cost — storage, memory, and the re-embedding you forgot to budget for.

Vector storage feels free in a demo. Ten thousand vectors cost nothing, so nobody models it. Then the corpus grows to tens of millions, someone opens the bill, and the vector store is suddenly a top-three line item that nobody forecast. The costs are entirely predictable — you can estimate them on a napkin before they surprise you. Here's the model: the drivers, the math, and the cost people always forget to include.

the real cost of vector storage at scale

The three cost drivers

Your vector bill has three components. Model all three or you'll under-forecast.

1. Raw vector storage. The size of one vector is roughly dimensions × bytes-per-number. A 1024-dim float32 vector is about 1024 × 4 = 4 KB. Multiply by your vector count:

vectors = documents × avg_chunks_per_document
storage = vectors × dimensions × bytes_per_number

A corpus of 1M documents at ~10 chunks each is 10M vectors. At 1024 dims × 4 bytes, that's ~40 GB just for the raw vectors — before any index. Bump to 3072 dims and it triples to ~120 GB. Dimension count is the biggest lever here, which is why choosing the smallest dimension that meets your quality bar pays off across the whole corpus.

2. Index memory. ANN indexes aren't free on top of storage. HNSW in particular builds a graph that typically adds meaningful memory overhead — often on the order of 1.5–3× the raw vector size depending on parameters — and for low-latency search that graph generally wants to live in RAM, which is more expensive than disk. So the memory footprint of a searchable index is materially larger than the raw vector math suggests. This is the line people miss: they estimate storage, not the index's RAM.

3. Embedding compute — and re-embedding. Generating embeddings costs money (per token on a hosted API, or GPU time self-hosted). The initial ingestion is a one-time cost you'll probably remember. The cost you'll forget is re-embedding for freshness: a knowledge base that stays current re-embeds changed content continuously, and if you ever switch embedding models you re-embed the entire corpus. Freshness turns embedding from a one-time to a recurring cost, and a model migration is a big one-off spike.

A worked estimate

Say you have 5M documents, ~8 chunks each = 40M vectors, using a 1024-dim model:

  • Raw storage: 40M × 1024 × 4 bytes ≈ 160 GB.
  • Index memory: with HNSW overhead, budget roughly 240–480 GB of memory-resident index for fast search.
  • Initial embedding: 40M chunks × average chunk token count, priced at your provider's per-token rate — a real but one-time number.
  • Ongoing embedding: if 5% of content changes monthly, you re-embed ~2M vectors/month, indefinitely.

The headline: the searchable footprint (index in RAM) dominates, and the recurring embedding cost is the one that quietly compounds. Neither shows up if you only estimate raw storage.

The levers that reduce the bill

Once you can see the drivers, the optimizations are obvious:

  • Reduce dimensions. The single biggest lever, because it scales the whole corpus. Pick the smallest dimension that clears your quality bar; use Matryoshka truncation if your model supports it.
  • Quantize. Binary or scalar quantization cuts per-vector size dramatically (binary ~32×) — pair it with a full-precision rerank pass to recover accuracy. This attacks both storage and index memory.
  • Chunk deliberately. Vector count is documents × chunks_per_document. Over-chunking (tiny chunks, heavy overlap) multiplies your vector count and therefore every cost above. Right-sized chunks keep the count sane.
  • Don't over-provision scale you don't have. A distributed vector cluster provisioned for a billion vectors when you have 40M is paying for headroom you're not using. Match the infrastructure to actual scale.
  • Budget re-embedding explicitly. Decide your freshness cadence and model it as a recurring line, so it's a choice rather than a surprise.

The takeaway

Vector storage is cheap per vector and expensive per corpus, and the parts that dominate — index memory and recurring re-embedding — are exactly the parts a naive estimate leaves out. The good news is that none of it is mysterious: documents × chunks × dimensions × precision, plus index overhead, plus your freshness cadence, gets you a forecast on a napkin. Do that math before you scale, choose your dimensions and chunking with the bill in mind, and you'll never be the team that discovers their vector store is a top line item only after the invoice arrives.