What is a vector database? HNSW, IVF, and when to skip one

A vector database is a data store that indexes embeddings and returns the items closest to a query vector, usually through an approximate nearest-neighbor index. To answer what is a vector database in engineering terms, this guide starts with embeddings and nearest-neighbor search, then opens up HNSW and IVF. A runnable, offline test compares exact and approximate search on generated vectors. The last sections give decision rules for choosing a store, and a position on when you should not add one.

#What is a vector database

A vector database stores embeddings with IDs and metadata, builds an index over them, and answers nearest-neighbor queries through an API. An index library such as Faiss does only the search step. The database adds durable storage, filtering, updates, replication, and access control, and those additions cause most of the operational differences between products.

Faiss's guidelines say all of its indexes are stored in RAM, and a library has no notion of tenants or backups. Those are the jobs a database takes on.

RAG is a common reason developers meet one. A RAG system chunks documents, embeds each chunk, and retrieves the closest chunks for each question. If you are deciding whether retrieval is the right approach at all, RAG vs fine-tuning covers that choice first.

#How embeddings turn similarity into distance

An embedding model maps text, images, or code to a fixed-length list of numbers, and nearby vectors mean related content. OpenAI's embeddings guide says small distances suggest high relatedness. Search then means finding the stored vectors closest to the query's vector.

As of November 8, 2026, OpenAI's text-embedding-3-small returns 1,536 dimensions by default and text-embedding-3-large returns 3,072. Those vectors are normalized to length 1, so cosine similarity reduces to a dot product and ranks results the same way as Euclidean distance.

The example later in this article uses inner product on normalized vectors for that reason. If your model does not normalize, check which distance metric your store expects, because a mismatch silently changes rankings.

#Why exact nearest-neighbor search stops scaling

Exact search compares the query with every stored vector, so its cost grows linearly with both vector count and dimension. It gives perfect recall, and it is fine for tens of thousands of vectors. Doubling the corpus doubles the CPU per query, and a single core soon cannot keep up with real traffic.

Elastic's kNN docs say exact search "does not scale well for large datasets." The measurements below show the shape on 768-dimensional generated vectors. They use one CPU core and report CPU time per query, so other jobs on the machine do not distort the result:

Exact CPU time grows about linearly with N. HNSW at efSearch 40 barely moves, because a graph walk visits a small fraction of the vectors. Its recall did drift, to 0.877 at 400,000 vectors, so a larger corpus needs a larger efSearch. Memory is the other limit. One million 1,536-dimensional vectors need 5.7 GiB as raw float32, before any index.

#How HNSW searches a layered graph

HNSW stacks proximity graphs so that each layer holds a smaller random subset of the vectors. A query starts at the top layer, walks greedily toward the target, drops down at each local minimum, and searches widely only at the bottom. Malkov and Yashunin introduced it in 2016, and the name stands for Hierarchical Navigable Small World.

The paper assigns each vector a maximum layer of floor(-ln(unif(0,1)) * mL) and suggests mL = 1/ln(M), which matches a skip list where each layer keeps 1 in M nodes. With M = 16 and one million vectors, expect about 62,500 nodes in layer 1, 3,906 in layer 2, 244 in layer 3, 15 in layer 4, and 1 in layer 5.

Search uses ef = 1 in the upper layers, which is why the walk in Figure 1 follows a single path. At layer 0 it keeps efSearch candidates and returns the best k. The paper reports a reasonable M range of 5 to 48, and link memory of about 60 to 450 bytes per object, excluding the vectors.

Defaults differ across stores, so "HNSW with default settings" means different things:

HNSW's costs are memory and build time. Faiss documents d * 4 + M * 2 * 4 bytes per vector, and pgvector's README says HNSW has slower builds and uses more memory than IVFFlat. Faiss HNSW also cannot remove vectors, so deletes need a store that handles them for you.

#How IVF partitions vectors into cells

IVF clusters the vectors with k-means, stores each vector in the list of its nearest centroid, and at query time scans only the nprobe closest lists. Jégou, Douze, and Schmid's IVFADC paper pairs that inverted file with compressed codes. Raising nprobe improves recall and scans more vectors.

Faiss's guidelines suggest between 4*sqrt(N) and 16*sqrt(N) cells below one million vectors, with 30 to 256 training vectors per cell. IVF needs that training step before it accepts data, which HNSW does not.

A flat IVF list costs 4*d + 8 bytes per vector, according to Faiss's index table. Product quantization replaces the raw vector with ceil(M * nbits / 8) bytes, which is how Jégou's team validated the approach on two billion vectors. The Faiss authors built a nearest-neighbor graph over one billion vectors in under 12 hours on four GPUs. pgvector's README describes IVFFlat as faster to build and lighter on memory than HNSW, with a worse speed-recall trade-off.

#Exact vs HNSW vs IVF on generated vectors

On 200,000 generated vectors, HNSW at efSearch 40 returned 93.6% of the true top 10 and cost about 1% of exact search's CPU time per query. IVF did better on this dataset because its k-means cells line up with the topics the generator created. Treat these numbers as a method, because recall changes with the shape of your data.

"""Compare exact, HNSW, and IVF search on generated vectors. Runs offline."""
import sys
import time
 
import faiss
import numpy as np
 
N = int(sys.argv[1]) if len(sys.argv) > 1 else 200_000
SPREAD = sys.argv[2] if len(sys.argv) > 2 else "0.5"  # or "gaussian"
D, K, QUERIES = 768, 10, 500
 
 
def make_vectors(n, seed):
    rng = np.random.default_rng(seed)
    if SPREAD == "gaussian":  # no structure at all
        x = rng.standard_normal((n, D), dtype=np.float32)
    else:  # 2,000 topics, each varying along 16 hidden directions
        topics = np.random.default_rng(1).standard_normal((2_000, D), dtype=np.float32)
        basis = np.random.default_rng(2).standard_normal((16, D), dtype=np.float32) / 4
        x = topics[rng.integers(0, 2_000, n)]
        x += float(SPREAD) * (rng.standard_normal((n, 16), dtype=np.float32) @ basis)
        x += 0.05 * rng.standard_normal((n, D), dtype=np.float32)
    return x / np.linalg.norm(x, axis=1, keepdims=True)  # cosine = inner product
 
 
def measure(index, queries, truth):
    """Single-thread CPU time per query, one query at a time, plus recall@K."""
    faiss.omp_set_num_threads(1)
    found, start = [], time.process_time()  # CPU time resists noisy neighbors
    for q in queries:
        found.append(index.search(q[None, :], K)[1][0])
    ms = (time.process_time() - start) / len(queries) * 1000
    hits = sum(len(set(f) & set(t)) for f, t in zip(found, truth))
    return hits / (K * len(queries)), ms
 
 
def report(name, recall, ms, build_s, index):
    mb = faiss.serialize_index(index).size / 1e6
    print(f"{name:22} recall@{K}={recall:6.3f} {ms:8.3f} cpu_ms/query  "
          f"build={build_s:6.1f}s size={mb:6.0f} MB", flush=True)
 
 
xb = make_vectors(N, seed=3)
xq = make_vectors(QUERIES, seed=4)
threads = faiss.omp_get_max_threads()
print(f"faiss {faiss.__version__}, numpy {np.__version__}, N={N:,}, d={D}, spread={SPREAD}")
 
flat = faiss.IndexFlatIP(D)
flat.add(xb)
truth = flat.search(xq, K)[1]  # ground truth, computed with all threads
recall, ms = measure(flat, xq, truth)
report("exact (Flat)", recall, ms, 0.0, flat)
 
faiss.omp_set_num_threads(threads)
start = time.perf_counter()
hnsw = faiss.IndexHNSWFlat(D, 16, faiss.METRIC_INNER_PRODUCT)  # M=16
hnsw.hnsw.efConstruction = 64
hnsw.add(xb)
build = time.perf_counter() - start
for ef in (10, 40, 160):
    hnsw.hnsw.efSearch = ef
    recall, ms = measure(hnsw, xq, truth)
    report(f"HNSW efSearch={ef}", recall, ms, build, hnsw)
 
faiss.omp_set_num_threads(threads)
nlist = int(4 * np.sqrt(N))
start = time.perf_counter()
ivf = faiss.IndexIVFFlat(faiss.IndexFlatIP(D), D, nlist, faiss.METRIC_INNER_PRODUCT)
ivf.train(xb)
ivf.add(xb)
build = time.perf_counter() - start
for nprobe in (1, 10, 40):
    ivf.nprobe = nprobe
    recall, ms = measure(ivf, xq, truth)
    report(f"IVF nprobe={nprobe}/{nlist}", recall, ms, build, ivf)

Run with python ann_compare.py 200000 0.5. Tested on Apple M2, 16 GB, Python 3.15.1, NumPy 2.5.4, and faiss-cpu 1.15.1. Index builds used all 8 cores, and query timings used one. Real output:

faiss 1.15.1, numpy 2.5.4, N=200,000, d=768, spread=0.5
exact (Flat)           recall@10= 1.000   24.896 cpu_ms/query  build=   0.0s size=   614 MB
HNSW efSearch=10       recall@10= 0.671    0.128 cpu_ms/query  build=  66.0s size=   643 MB
HNSW efSearch=40       recall@10= 0.936    0.237 cpu_ms/query  build=  66.0s size=   643 MB
HNSW efSearch=160      recall@10= 0.991    0.657 cpu_ms/query  build=  66.0s size=   643 MB
IVF nprobe=1/1788      recall@10= 0.983    0.199 cpu_ms/query  build=  17.3s size=   622 MB
IVF nprobe=10/1788     recall@10= 1.000    0.297 cpu_ms/query  build=  17.3s size=   622 MB
IVF nprobe=40/1788     recall@10= 1.000    0.662 cpu_ms/query  build=  17.3s size=   622 MB

This test exercised Faiss's in-process indexes only. It did not exercise pgvector, Pinecone, Qdrant, Weaviate, Milvus, filters, updates, concurrent queries, or real embeddings. The machine was shared with other jobs, which is why the timings are CPU time.

Changing only the data shape shows why you should not trust someone else's recall numbers:

Structureless Gaussian vectors give an index nothing to exploit, and the bottom row shows it. Tight, cleanly separated clusters flatter IVF in the opposite direction. Your embeddings will land somewhere on this range, and only your own data can say where. Measure retrieval quality with the methods in LLM evaluation once you have a candidate.

#When you do not need a separate vector database

You do not need a separate vector database when each query searches a small scope, or when your vectors fit on a Postgres instance you already run. A second datastore adds a second backup plan, a sync job, and a new way to serve stale data. Make it earn that cost with a measured limit.

Here is the position this article takes, which is judgment built on the numbers above and not a benchmark result:

1. Under about 100,000 vectors per search scope, use exact search. At this size the test above spent about 10 ms of CPU per query on one core. Multi-tenant apps often qualify, because each tenant is its own scope.
2. If you already run Postgres and your vectors number in the low millions, use pgvector with HNSW. You keep ACID compliance, point-in-time recovery, and JOINs, and retrieval joins the same transaction as your rows.
3. Add a dedicated store only after you see a concrete signal, such as those below.

Stay in Postgres until one of these happens:

• The hnsw graph no longer fits into maintenance_work_mem notice appears and you cannot grow memory.
• Filtered queries miss your recall target even with iterative scans.
• Index builds or vacuum start to hurt transactional latency.

Do the storage arithmetic first. pgvector documents 4 * dimensions + 8 bytes per vector, so five million 1,536-dimensional vectors take 28.65 GiB of vector storage before any index. That fits on a large single instance, but plan for the index on top.

If you are adding agents to an existing product, check the database it already runs before you add another.

#Best vector database by workload and team

No product is the best vector database in general, so pick by your workload and the team that will operate it. The table gives a starting point for each common situation, with facts from each project's own docs as of November 8, 2026. It is not a ranking, and the order of rows implies nothing.

Versions current on November 8, 2026 were pgvector 0.8.7, Qdrant 1.20.0, Weaviate 1.40.4, and Milvus 3.0.5.

Team shape decides the rest. If nobody owns database on-call, start with a managed service. If a platform team already runs Kubernetes and object storage, Milvus's moving parts will be familiar. Compare current managed offerings in the platforms directory.

#Filtering and hybrid search change the answer

Filters are where stores differ most, because an approximate index can return too few results when the filter runs after the graph walk. pgvector's README gives a concrete case: with HNSW and the default hnsw.ef_search of 40, a filter matching 10% of rows returns about 4 rows on average.

pgvector's fix, since version 0.8.0, is iterative index scans, which keep scanning until enough rows match or a limit such as hnsw.max_scan_tuples is reached. Qdrant takes another route: its filterable HNSW adds filter-aware edges, which only help if payload indexes exist before ingest.

Hybrid search combines keyword and vector results, and exact identifiers such as error codes often need it. Weaviate fuses BM25F and vector scores, with Relative Score Fusion as the default since v1.24. pgvector's README points to Postgres full-text search combined with reciprocal rank fusion or a cross-encoder. Pinecone's search docs describe a single index that handles full-text and semantic search together.

Add a reranking step after retrieval if the top results are close but misordered. Retrieval quality is also a context problem, covered in context engineering.

#Test a vector database on your own embeddings

Replace make_vectors in the script above with a NumPy array of your real embeddings, then measure recall@10 against exact search at your actual filter selectivity. If pgvector or a flat index meets your recall and latency targets on a copy of your data, postpone adding a vector database and keep one fewer system to run.

1. Export 100,000 real embeddings and 500 real queries, normalized the same way your model produces them.
2. Compute exact top-10 results as ground truth with IndexFlatIP.
3. Tune HNSW efSearch until recall@10 reaches your target, and record CPU time per query.
4. Repeat with a filter that matches the same fraction of rows your production query matches.
5. Load the same vectors into a candidate store, and compare only at matched recall.

Set a recall target before you start, such as 0.95 at your p95 latency budget. Choose the store that reaches it with the least new operational work. New index types and store releases land in the news feed, and the RAG topic page collects the rest of this retrieval series. If you also self-host the embedding model, the vLLM vs SGLang comparison covers the serving side.

#Questions