Naive RAG vs. advanced RAG vs. modular RAG: which should you build?

Almost every RAG system in production is one of three shapes. The industry has taken to calling them naive, advanced, and modular RAG. The names are unfortunate — "naive" sounds like a mistake and "modular" sounds like the goal — but the underlying progression is real, and knowing where your problem sits saves you from building a research project when you needed a search box.

Naive RAG: embed, retrieve, generate

The original pattern, and still the right default for most teams.

1. Chunk your documents and embed each chunk.
2. At query time, embed the query and retrieve the top-k nearest chunks.
3. Stuff those chunks into the prompt and let the model answer.

That's it. No query rewriting, no reranking, no feedback loops. It is three moving parts and you can stand it up in an afternoon.

chunks = retrieve(embed(query), k=5)
answer = llm(f"Context:\n{chunks}\n\nQuestion: {query}")

Naive RAG fails in predictable ways. The query and the answer don't always share vocabulary, so pure vector similarity misses. Retrieval pulls five chunks whether or not any of them are relevant. There's no step that notices the retrieved context is garbage before the model confidently answers from it. But for a well-scoped corpus with clean documents and users who ask direct questions, it is genuinely enough — and every layer you add is latency, cost, and a new thing to debug.

Build this when: your corpus is coherent, your users ask reasonably direct questions, and you haven't yet measured a retrieval problem you can point at.

Advanced RAG: fix the weak steps

Advanced RAG keeps the same pipeline but hardens the two ends: what you retrieve with, and what you keep after retrieval.

Pre-retrieval improvements shape the query before it hits the index:

• Hybrid search — run vector search and keyword (BM25) search together and fuse the results, so exact-term matches aren't lost to semantic drift.
• Query rewriting / expansion — reformulate a terse query into something closer to how the answer is written, or generate a hypothetical answer and search with that (HyDE).

Post-retrieval improvements clean up what came back:

• Reranking — pull a wider candidate set (say top-50) with cheap vector search, then reorder with a cross-encoder that reads query and passage together. This is the single highest-leverage upgrade for most systems.
• Compression / filtering — drop passages that don't actually address the query before they eat context budget.

The pipeline still runs start to finish in one pass. You've just stopped trusting each step blindly. This is where most serious production systems live, and it's what Kognita ships by default: hybrid retrieval with reranking, not raw nearest-neighbor.

Build this when: naive RAG is missing answers you know are in the corpus, or retrieving the right document but ranking it fourth.

Modular RAG: retrieval becomes control flow

Modular RAG stops treating retrieval as a fixed pipeline and makes it a set of components the system can route between, loop over, and skip. Retrieval might happen zero times or five times for a single query.

Patterns that live here:

• Routing — classify the query and send it to the right index, or straight to the model with no retrieval at all.
• Iterative / recursive retrieval — retrieve, let the model reason, decide it needs more, retrieve again with a refined query.
• Self-correction (CRAG, Self-RAG) — grade the retrieved context; if it's weak, fall back to a web search or ask the model to reformulate.
• Agentic retrieval — the model holds search as a tool and decides when to call it, which is exactly what happens when you expose a knowledge base over MCP.

Modular RAG is powerful and genuinely necessary for open-ended agent workflows. It is also where teams drown. Every branch is a new failure mode, every loop is unbounded latency and cost, and evaluation gets much harder because the same query can take different paths on different runs.

Build this when: you have agents that reason over multiple steps, multiple heterogeneous knowledge sources to route between, or a hard requirement to recover gracefully when retrieval comes back empty — and you have the evaluation harness to keep it honest.

The honest recommendation

The progression is not a ladder you're supposed to climb to the top of. It's a menu. Start naive, measure where it breaks, and add the specific advanced component that fixes the specific failure you measured. Reach for modular RAG only when the shape of the problem — agents, routing, self-correction — actually demands control flow, not because it sounds more sophisticated.

The most common mistake we see isn't under-building. It's a team running a five-stage modular pipeline to answer FAQ-style questions that naive RAG with a reranker would have nailed at a tenth of the latency.