Hybrid search explained: combining BM25 and vector similarity

Full-text search and semantic search each have blind spots. Hybrid search closes the gap, and in 2025, it has become the default retrieval strategy for production RAG pipelines.

The problem with picking one

Most teams start with one retrieval method and quickly run into its ceiling.

Vector search feels like the obvious modern choice; it handles synonyms, paraphrasing, and intent naturally. But ask it to find error code E_CONN_TIMEOUT or a product SKU, and it often returns vaguely related results rather than the exact document you need. Dense embeddings compress meaning, and in doing so, they lose precision on rare tokens, identifiers, and proper nouns.

BM25, on the other hand, will nail that exact identifier every time. But phrase your query conversationally, for example "something about connection timeouts in distributed systems," and keyword search may return nothing useful if the document uses different vocabulary.

Hybrid search runs both in parallel and merges the results. The remainder of this post explains exactly how.

What is BM25?

BM25 (Best Match 25) is the probabilistic ranking function underpinning most full-text search systems; Elasticsearch, OpenSearch, and PostgreSQL's tsvector all use variants of it. It builds on TF-IDF with two key improvements: document length normalisation and a term frequency saturation curve.

score(D, Q) = Σ IDF(qᵢ) · f(qᵢ, D) · (k1 + 1)
              ─────────────────────────────────────────────────────
              f(qᵢ, D) + k1 · (1 − b + b · |D| / avgdl)

• f(qᵢ, D): how often query term qᵢ appears in document D
• IDF(qᵢ): inverse document frequency; rare terms score higher
• k1: term frequency saturation (typical default: 1.2); beyond a threshold, extra occurrences add little
• b: length normalisation factor (typical default: 0.75)
• |D| / avgdl: ratio of document length to average document length

BM25 produces sparse vectors: one dimension per vocabulary token, nearly all zeros. This is efficient to index and lightning-fast to query, but it cannot bridge vocabulary gaps; if the document says "latency" and the query says "slowness," BM25 scores zero.

What is vector similarity search?

Vector similarity search encodes text into dense embeddings, which are high-dimensional floating-point vectors produced by a neural model (typically a Transformer). Two pieces of text that mean similar things end up near each other in this space, even if they share no words.

At query time, the query is encoded into the same vector space and the search engine returns documents whose embeddings have the smallest distance (cosine similarity, dot product, or L2 norm) to the query embedding. HNSW is the most common approximate-nearest-neighbour index structure used for this.

Dense search handles:

• Synonyms and paraphrases ("slow" ↔ "high latency")
• Cross-lingual queries
• Conceptual questions where no single keyword captures the intent

Its weaknesses are the mirror image of BM25's strengths: rare proper nouns, version strings, product codes, and negations are all difficult because the embedding collapses them into nearby regions of the space.

Why combine them?

In production RAG systems, hybrid retrieval has been shown to improve recall by 20–30% compared to either method alone. The intuition is simple: if vectors fail for a given query, BM25 provides a safety net, and vice versa.

The score-scale problem

Naively combining BM25 and vector scores is harder than it sounds. BM25 scores are unbounded positive numbers that depend on corpus statistics. Vector similarity scores are typically in [-1, 1] (cosine) or [0, ∞) (dot product). You cannot simply add them: a BM25 score of 12.4 and a cosine similarity of 0.87 are not on the same scale.

Two approaches solve this:

Relative Score Fusion (RSF): normalise each list's scores to [0, 1] using min-max scaling, then take a weighted sum. Simple but sensitive to outliers: one unusually high-scoring result can compress the rest of the list.

Reciprocal Rank Fusion (RRF): ignore scores entirely and work only on rank positions. This is the more robust choice and is what Kognita uses.

How Reciprocal Rank Fusion works

RRF was introduced by Cormack, Clarke, and Buettcher (SIGIR 2009) and has since been adopted by Elasticsearch, OpenSearch, Weaviate, Azure AI Search, and most other major retrieval systems.

The formula is deceptively simple:

rrf_score(d) = Σᵢ  1 / (k + rankᵢ(d))

• rankᵢ(d): the position of document d in list i (1-indexed)
• k: a smoothing constant, typically 60, that prevents the top-ranked document from dominating

A document that appears at rank 1 in the BM25 list and rank 3 in the vector list receives:

1 / (60 + 1) + 1 / (60 + 3) ≈ 0.0164 + 0.0159 = 0.0323

A document at rank 10 in both:

1 / (60 + 10) + 1 / (60 + 10) ≈ 0.0286

Documents that appear consistently high across both ranked lists rise to the top of the merged result. Documents that only appear in one list still contribute but are naturally discounted. This is what makes RRF robust: a single high BM25 score cannot override a poor semantic ranking.

Tuning with alpha

Beyond RRF, the alpha parameter lets you blend the two retrieval signals at a higher level:

The right value depends on your content and query distribution. A product catalogue with precise SKUs benefits from alpha closer to 0.0. A general-purpose knowledge base or FAQ typically performs best around 0.5–0.75. Some production systems now detect the query type at runtime: if the query contains identifiers or version strings, alpha shifts toward BM25; otherwise it shifts toward vector.

When to use each mode

Full-text only (alpha = 0.0)

• Log search and error code lookup
• Code search where exact symbol names matter
• Product SKU and catalogue ID lookup
• Legal document retrieval where verbatim clauses must match

Semantic only (alpha = 1.0)

• FAQ matching where users phrase questions differently every time
• Long-form question answering over well-written prose
• Cross-lingual retrieval where query and document language differ
• Similarity search ("find me more posts like this one")

Hybrid (0.1 < alpha < 0.9)

• General-purpose knowledge base search
• RAG pipelines over mixed-content corpora (docs, tickets, wikis)
• Customer support: users mix precise product names with vague intent
• Any domain where you cannot predict the query distribution in advance

What comes after retrieval

Hybrid search improves recall, getting more of the right documents into the candidate set. But the top-k results still need a final ranking pass for precision. A cross-encoder reranker (a model that jointly encodes the query and each candidate document) can re-score the hybrid candidates with much higher fidelity than either BM25 or embedding similarity alone.

The full production stack looks like:

Query
  ↓
BM25 retrieval ──┐
                 ├─→ RRF merge → top-k candidates → reranker → final results
Vector retrieval ┘

Retrieval is about maximising recall cheaply. Reranking is about maximising precision on a small candidate set. Hybrid search is the right foundation for both.