MCP vs. RAG: do you need both?

Since the Model Context Protocol took off, a confused question keeps coming up: "Do I still need RAG if I have MCP?" It's the wrong framing. RAG and MCP aren't two answers to one question; they answer different questions. One is about knowledge, the other about capability.

What each one actually is

RAG (Retrieval-Augmented Generation) is a technique for grounding a model in your data. You embed your documents, retrieve the relevant passages for a query, and stuff them into the prompt so the model answers from your facts instead of its training data. RAG changes what the model knows at the moment it answers.

MCP (Model Context Protocol) is a standard interface for connecting a model to tools and data sources. An MCP server exposes capabilities such as "search the knowledge base," "create a ticket," and "query the database," and any MCP-compatible client (Claude, an IDE, an agent framework) can call them. MCP changes what the model can do, and how cleanly it plugs into things.

The key insight: MCP is a transport and integration standard; RAG is a retrieval strategy. They operate at different layers. Asking "MCP or RAG?" is like asking "USB or the file you're copying?" One is the connector, the other is the content moving through it.

Where the confusion comes from

The overlap is real but narrow. One of the most common MCP servers is... a knowledge-base search tool. So people see "MCP does retrieval" and conclude it replaces RAG. What's actually happening:

• RAG is the implementation behind that search tool: the embeddings, hybrid search, and ranking that find the right passages.
• MCP is the interface that exposes that search to a model in a standard way, so any client can call it without custom glue code.

In other words, MCP didn't replace RAG. It gave RAG a standard socket to plug into. A well-built knowledge-base MCP server is a RAG system wearing an MCP connector.

Two ways to feed a model your knowledge

There's a genuine architectural choice here, and it's worth being precise about it.

Prompt-time RAG (retrieval baked into your app). Your application retrieves passages and constructs the prompt before the model ever runs. The model receives context as part of its input and has no say in what it got. Best when: you control the app, retrieval should always happen, and you want tight control over what lands in the context window.

Tool-based retrieval via MCP. You expose search as an MCP tool and let the model decide when to call it, with what query, and whether to call it again. The model drives retrieval as part of its reasoning. Best when: the model should retrieve only sometimes, refine its own queries, or combine search with other actions (look up a doc, then file a ticket).

So do you need both?

For most serious systems: yes, and they sit in a stack rather than competing.

• RAG is your retrieval engine, the thing that reliably finds the right passage from your knowledge base. You need this regardless; it's the substance.
• MCP is how you expose that engine (and other capabilities) to models and clients in a standard, reusable way. You want this the moment more than one client needs your knowledge, or the model needs to combine retrieval with actions.

A concrete shape of a mature system:

Claude / IDE / agent  (MCP client)
        │  calls tools over MCP
        ▼
  MCP server
   ├── search_knowledge_base  → RAG (hybrid search + rerank)
   ├── get_document           → RAG
   └── create_ticket          → your app's API

The model reasons, calls search_knowledge_base when it needs facts (RAG does the actual retrieval), reads the results, maybe calls create_ticket to act on them. RAG supplies the knowledge; MCP supplies the wiring and the capabilities.

The rule of thumb

• If your only need is "answer questions from my docs inside one app I control," start with prompt-time RAG. You may not need MCP yet.
• The moment you want multiple clients to reach your knowledge, or you want the model to decide when to retrieve and to combine retrieval with actions, expose your RAG system as an MCP server.

RAG is what your model knows. MCP is what your model can reach. Most real products end up needing both, not because they're redundant, but because they were never solving the same problem.