A vector database stores embeddings — lists of numbers that represent the meaning of text, images, or audio — and answers one core question fast: given this vector, which stored vectors are closest to it? That "closest to it" is how semantic search works. You embed a query, find the nearest document vectors, and you've retrieved by meaning instead of keywords.
That's the whole idea. The harder question is whether you need a dedicated vector database to do it, or whether the database you already run will do fine. The honest answer for most teams: you probably don't need a new one yet.
What the "vector" part actually requires
Storing vectors is trivial — they're just arrays of floats. The hard part is the search. A brute-force nearest-neighbor scan compares your query against every stored vector, which is fine at ten thousand vectors and unusable at ten million.
So vector databases use approximate nearest neighbor (ANN) indexes — most commonly HNSW, a graph structure that finds nearly the closest vectors without checking all of them. You trade a tiny bit of recall for orders of magnitude more speed. This ANN indexing is the one genuinely specialized capability, and here's the thing: it's no longer exclusive to dedicated vector databases.
The pgvector option
pgvector is a Postgres extension that adds a vector column type and HNSW indexing directly to a database you very likely already run. That changes the calculus completely:
CREATE EXTENSION vector;
ALTER TABLE chunks ADD COLUMN embedding vector(1024);
CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);
SELECT content
FROM chunks
ORDER BY embedding <=> $1 -- <=> is cosine distance
LIMIT 5;
That's semantic search, in your existing database, with your existing backups, your existing access control, and your existing SQL. And critically, your vectors now live next to your relational data. You can filter by org_id, join to a documents table, and constrain by updated_at in the same query that does the nearest-neighbor search. With a separate vector database, that metadata lives in a different system and you're stuck either duplicating it or making two round trips and reconciling.
This is exactly why Kognita stores vectors in Postgres (Neon) rather than a bespoke vector store — the operational and querying advantages of "it's just Postgres" are substantial, and you don't give up ANN performance to get them.
When a dedicated vector database earns its keep
pgvector isn't the answer to everything. A purpose-built vector database (Pinecone, Weaviate, Qdrant, Milvus) is worth it when you hit specific walls:
- Scale past what one Postgres instance handles well. In the hundreds of millions to billions of vectors, dedicated systems are built to shard and distribute the index in ways Postgres wasn't designed for.
- Vector search is the whole product, at extreme QPS. If you're serving very high query volumes where every millisecond of ANN latency is your core SLA, specialized engines are tuned harder for exactly that.
- You want advanced vector features out of the box — built-in reranking, sophisticated hybrid fusion, multi-vector or late-interaction retrieval — without assembling them yourself.
- You want to keep vector load off your primary OLTP database so a heavy query workload can't contend with your transactional traffic.
Notice these are scale-and-specialization arguments, not "you can't do semantic search otherwise" arguments.
A decision guide
| Situation | Recommendation |
|---|---|
| Under ~10M vectors, already run Postgres | pgvector |
| Need to filter/join vectors with relational data | pgvector |
| Small team, want fewer systems to operate | pgvector |
| Hundreds of millions+ of vectors | Dedicated vector DB |
| Very high QPS, vector search is the core product | Dedicated vector DB |
| Want reranking/hybrid/multi-vector built in | Dedicated vector DB (or a managed RAG layer) |
The real question
"Do I need a vector database?" is usually the wrong question. You need vector search. Whether that lives in a new system or in the Postgres you already operate depends almost entirely on scale and on how much your product's identity is "we do vector search."
For the large middle of the market — teams with millions, not billions, of vectors who want semantic search as a feature rather than as the entire product — pgvector is not a compromise. It's the sensible default, and adding a distributed vector database before you've hit its limits is buying operational complexity you haven't earned yet.