---
title: "Eval"
type: "glossary term"
url: "https://aifordevs.org/glossary#eval"
markdown: "https://aifordevs.org/glossary/eval.md"
publisher: "AI for Devs"
license: "https://aifordevs.org/reuse"
attribution: "Credit AI for Devs and link to https://aifordevs.org/glossary#eval"
---

# Eval

An eval is a repeatable test of a model or an agent on examples from your task, scored by code, by people, or by another model. Evals show whether a prompt change, a model swap, or a new tool made results better or worse.

Anthropic's vocabulary is a useful starting point: a task has defined inputs and success criteria, a grader scores one aspect of the result, and a trial is one attempt. Outputs vary between runs, so run each task several times and compare pass rates.

## Example

On a 40-task suite, a prompt change that moves passing tasks from 31 to 36 raises the pass rate from 77.5% to 90%.

## Source

[Anthropic, Demystifying evals for AI agents (2026)](https://www.anthropic.com/research/demystifying-evals-for-ai-agents)

## Related terms

- [Golden dataset](https://aifordevs.org/glossary#golden-dataset)
- [LLM-as-a-judge](https://aifordevs.org/glossary#llm-as-a-judge)
- [SWE-bench](https://aifordevs.org/glossary#swe-bench)
- [Hallucination](https://aifordevs.org/glossary#hallucination)
- [Guardrails](https://aifordevs.org/glossary#guardrails)

## Read the article

[LLM evaluation: build an eval suite and gate deploys in CI](https://aifordevs.org/articles/llm-evaluation)

## Cite this term

AI for Devs. "Eval." https://aifordevs.org/glossary#eval

- Page: https://aifordevs.org/glossary#eval
- Markdown: https://aifordevs.org/glossary/eval.md
- Reuse: Quote or summarize with a link to this page. Terms: https://aifordevs.org/reuse
- Publisher: AI for Devs