---
title: "Prefix caching"
type: "glossary term"
url: "https://aifordevs.org/glossary#prefix-caching"
markdown: "https://aifordevs.org/glossary/prefix-caching.md"
publisher: "AI for Devs"
license: "https://aifordevs.org/reuse"
attribution: "Credit AI for Devs and link to https://aifordevs.org/glossary#prefix-caching"
---

# Prefix caching

Prefix caching is a serving-engine technique that reuses KV cache blocks from earlier requests when a new prompt starts with the same tokens. Prefill skips the matched tokens, which lowers time to first token and frees GPU compute.

vLLM hashes each block from its parent's hash, its own tokens, and extras such as LoRA IDs or a cache salt, so a block matches only when its whole prefix matches, and only full blocks are cached. SGLang's RadixAttention keeps prefixes in a radix tree with LRU eviction. Prompt caching is the hosted API feature built on the same idea, with its own pricing.

## Example

With 16-token blocks, a shared 100-token prefix gives 6 full cached blocks, or 96 tokens. The remaining 4 tokens get recomputed.

## Source

[vLLM docs: Automatic Prefix Caching](https://docs.vllm.ai/en/stable/design/prefix_caching/)

## Related terms

- [Prompt caching](https://aifordevs.org/glossary#prompt-caching)
- [KV cache](https://aifordevs.org/glossary#kv-cache)
- [TTFT](https://aifordevs.org/glossary#ttft)
- [PagedAttention](https://aifordevs.org/glossary#paged-attention)

## Read the article

[vLLM vs SGLang, plus Ollama and llama.cpp compared](https://aifordevs.org/articles/vllm-sglang-llama-cpp)

## Cite this term

AI for Devs. "Prefix caching." https://aifordevs.org/glossary#prefix-caching

- Page: https://aifordevs.org/glossary#prefix-caching
- Markdown: https://aifordevs.org/glossary/prefix-caching.md
- Reuse: Quote or summarize with a link to this page. Terms: https://aifordevs.org/reuse
- Publisher: AI for Devs