Deep dive
RAG vs fine tuning: which failures each one can fix
RAG vs fine tuning compared by failure mode: what each changes, who still sells fine tuning, dated costs worked in code, and when to combine them.
12 min readFine-tuning and training
Topic
RAG and retrieval
Giving a model the right documents at the right time. Chunking, embeddings, hybrid search, and how to tell whether retrieval is helping.
Embeddings, vector search, reranking03 articles07 glossary terms06 platforms
Articles
Read on this topic
Deep dives and guides filed under this topic, newest first.
All articlesDeep dive
What is a vector database? HNSW, IVF, and when to skip one
What is a vector database, how HNSW and IVF search work, recall measured on generated vectors, and when pgvector or no database is the right choice.
11 min readRAG and retrieval
Guide
Context engineering: budget and order the context window
Context engineering is choosing what a model reads on each call. Size each section, order it for prompt caching, trim inputs, and test every layout change.
11 min readPrompting and context
Glossary
Terms you will meet
- Chunking
- Chunking is splitting documents into smaller passages before you embed and index them for retrieval. Chunk size sets how much text a single retrieved passage can bring into the prompt.
- Embedding
- An embedding is a fixed-length vector of numbers that represents a piece of text, an image, or code, produced by an embedding model. Items with similar meaning end up close together, which is what vector search relies on.
- HNSWHierarchical Navigable Small World
- HNSW (Hierarchical Navigable Small World) is a graph index for approximate nearest neighbor search that stacks proximity graphs, with fewer points in each higher layer. Queries start at the top layer and descend, which gives logarithmic scaling with dataset size.
- Hybrid search
- Hybrid search combines keyword search, usually BM25, with vector search and merges the two ranked lists into one result. Keyword matching catches exact identifiers that embeddings miss, and vectors catch paraphrases that keywords miss.
- RAGRetrieval-augmented generation
- RAG (retrieval-augmented generation) is a technique that fetches relevant documents at request time and puts them in the prompt so the model answers from them. The term comes from Lewis et al. (2020).
- Reranking
- Reranking is a second retrieval pass that rescores a shortlist of candidates with a slower, more accurate model and keeps the best few. The first pass can favor recall because the reranker sets the final order.
- Vector database
- A vector database is a database that indexes embeddings and returns the nearest neighbors to a query vector, usually with approximate search such as HNSW. It is a common retrieval layer in RAG systems.
Platforms