Migrating to a better embedding model without breaking your product

Switching embedding models invalidates your entire vector index. Here is how to do it without breaking anything.

The problem

Every embedding model maps text to a different vector space. Embeddings from text-embedding-3-small and text-embedding-3-large are not comparable; you cannot mix them in the same index. When you upgrade, you must re-embed every document before queries work correctly again.

The naive approach (don't do this)

1. Drop the old index
2. Re-embed everything with the new model
3. Rebuild the index

During step 2–3 your search is broken. On a large knowledge base that can take hours.

Zero-downtime migration with a shadow index

Step 1: Create a shadow index

Add a second vector column (or table) to your schema for the new model's embeddings. Keep the old index serving traffic.

Step 2: Backfill in the background

Re-embed all existing content into the shadow index. Process in batches, rate-limit to avoid hammering the embedding API.

for _, batch := range batchBy(contents, 100) {
    embeddings := embeddingClient.Embed(newModel, batch)
    repo.InsertShadowEmbeddings(embeddings)
}

Step 3: Keep new writes dual-indexed

During backfill, write every new piece of content to both the old index and the shadow index.

Step 4: Validate

Run your retrieval eval suite against both indexes. Compare recall@k metrics. Only proceed when the new model meets your quality bar.

Step 5: Atomic cutover

Once backfill is complete and validated:

1. Flip the query path to use the shadow index
2. Stop writing to the old index
3. Drop the old index after a cooldown period

How Kognita handles this

Kognita's embedding migration endpoint automates steps 2–5. You configure the target model, set a batch size, and the API handles backfill, dual-write, and cutover. Rollback is a single API call.