---
title: "TTFT"
type: "glossary term"
url: "https://aifordevs.org/glossary#ttft"
markdown: "https://aifordevs.org/glossary/ttft.md"
publisher: "AI for Devs"
license: "https://aifordevs.org/reuse"
attribution: "Credit AI for Devs and link to https://aifordevs.org/glossary#ttft"
also_known_as: "Time to first token"
---

# TTFT (Time to first token)

TTFT (time to first token) is the delay between sending a request and receiving the first output token. It covers queueing and prefill, so long prompts raise it and prompt caching lowers it.

TTFT sets how responsive a streaming interface feels, and it says nothing about how fast the rest of the reply arrives. Report it as a percentile under load, because queueing time only shows up when the server is busy.

## Example

Splitwise measured a median TTFT of 185 ms on an A100 and 95 ms on an H100 for Llama 70B on a coding workload with no batching.

## Source

[Patel et al., Splitwise: Efficient generative LLM inference using phase splitting (2023)](https://arxiv.org/abs/2311.18677)

## Related terms

- [Prefill and decode](https://aifordevs.org/glossary#prefill-and-decode)
- [Throughput](https://aifordevs.org/glossary#throughput)
- [Inference](https://aifordevs.org/glossary#inference)
- [Prefix caching](https://aifordevs.org/glossary#prefix-caching)
- [Prompt caching](https://aifordevs.org/glossary#prompt-caching)

## Cite this term

AI for Devs. "TTFT." https://aifordevs.org/glossary#ttft

- Page: https://aifordevs.org/glossary#ttft
- Markdown: https://aifordevs.org/glossary/ttft.md
- Reuse: Quote or summarize with a link to this page. Terms: https://aifordevs.org/reuse
- Publisher: AI for Devs