Most conversations about adding AI to a product now include the question of which vector database to use. Pinecone, Weaviate, Qdrant, pgvector as a Postgres extension: the comparison articles are everywhere, and they all assume the answer is one of those four. The question that gets skipped is whether a vector database is the right tool at all. For a lot of the AI features businesses actually ask for, it isn’t, and choosing one anyway adds a piece of infrastructure that has to be run, versioned and paid for indefinitely.
This is worth working through before the technology conversation starts, because it changes the shape of the project and the ongoing cost, not just which package gets installed.
What a vector database actually buys you
A vector database stores embeddings: numerical representations of text, images or audio produced by a machine learning model, positioned so that similar meanings sit close together in that numerical space. Searching it means finding the stored items whose embedding is nearest to the embedding of a query, which is how you get results for “holiday policy” that also surface a document titled “annual leave entitlement” even though the two phrases share no words.
That capability is genuinely useful for a specific class of problem: searching unstructured content where the vocabulary varies and the volume is too large to browse. Long document libraries, support ticket archives, meeting transcripts, product catalogues described in inconsistent language. If the search experience you are building is closer to “find me the paragraph that answers this” than “filter these records by field”, semantic search is doing real work.
When keyword search still wins
A large share of what gets pitched as “AI search” is actually a lookup problem with a small, well-structured dataset, and full-text search in whatever database you already run will outperform it on cost, latency and debuggability. PostgreSQL’s built-in text search, or an existing Elasticsearch instance, handles exact matches, filters and sorting far better than a vector index does, because that is not what a vector index is for.
The tell is usually in the query patterns. If users search by order number, customer name, SKU, date range or status, that is structured lookup and belongs in a normal indexed query. If they type a question in their own words and expect an answer synthesised from several sources, that is closer to genuine semantic retrieval. Plenty of features need both: a keyword filter to narrow the set, then semantic ranking within it. Building the vector layer before confirming the keyword layer is insufficient is how projects end up running infrastructure that duplicates work the primary database already does adequately.
The operational cost the demo doesn’t show
A local proof of concept with a few hundred documents in Chroma or pgvector takes an afternoon and looks convincing. Running the same thing in production for years is a different commitment, and the parts that get underestimated are rarely the query latency.
- Re-indexing on model change. Embeddings from one model version are not comparable to embeddings from another. Upgrading the embedding model, which providers do without much notice, means re-processing every stored item or living with degraded relevance until you do.
- Data freshness. Anything that changes after it’s embedded needs a pipeline to detect the change and re-embed it. This is trivial for a few thousand records and a real engineering problem for anything that updates continuously.
- Where the data actually sits. A managed vector database is another vendor holding a copy of your content, often outside the UK or EU by default. For anything containing personal data, that is a data processing relationship to document, not an implementation detail to skip.
- A second thing to operate. Every additional managed service is another dependency to monitor, patch and pay for, and another vendor whose pricing model can change under you. Pinecone and similar services bill per stored dimension and per query; at genuine scale this is not a rounding error on the invoice.
None of this is a reason to avoid semantic search. It’s a reason to size the ongoing commitment before committing, in the same way you would for any other piece of infrastructure that has to be maintained rather than shipped once.
If you do need it: pgvector, a managed service, or a dedicated engine
Once semantic search is genuinely justified, the choice of where to run it is mostly a question of scale and how much you want to operate yourself.
For most products under a few million embedded items, an extension like pgvector inside the PostgreSQL instance you already run is the pragmatic starting point. Embeddings, source records and metadata stay in one database, joins work normally, backups cover everything in one process, and there is no new vendor relationship to set up. The trade-off is that index tuning and scaling are on you as the dataset grows, and very large collections eventually push past what a single Postgres instance handles comfortably.
A managed vector database earns its cost when the collection is large, growth is fast, or the team has no appetite for running database infrastructure themselves. That convenience is priced in, and switching providers later means re-embedding and migrating everything, so it is worth treating as a longer-term commitment than the initial setup suggests.
A dedicated open-source engine such as Qdrant or Weaviate, self-hosted, sits between the two: more operational responsibility than a managed service, more control over data location and cost than either alternative, and generally only worth the extra work once scale or compliance requirements rule out both other options.
The right amount of infrastructure is the smallest amount that solves the actual retrieval problem, not the largest amount the budget can absorb.
A checklist before anyone builds anything
- Have you tried the existing database’s full-text search against real queries, and confirmed it genuinely falls short, rather than assuming it will?
- Is the underlying problem semantic retrieval over unstructured content, or structured lookup that a normal query already handles?
- Who owns re-embedding when the source content changes or the embedding model is upgraded?
- Where will the embedded content physically sit, and does that answer satisfy your data protection obligations if it includes personal data?
- At current and projected volume, does pgvector inside your existing database cover it, or is the scale large enough to justify a dedicated service?
- If you moved provider in two years, what would migrating the embeddings actually involve?
If you can answer all six without hand-waving, you’re ready to pick a database. If the first two are still fuzzy, that’s the conversation to have before any vendor comparison.