---
title: "SWE-bench"
type: "glossary term"
url: "https://aifordevs.org/glossary#swe-bench"
markdown: "https://aifordevs.org/glossary/swe-bench.md"
publisher: "AI for Devs"
license: "https://aifordevs.org/reuse"
attribution: "Credit AI for Devs and link to https://aifordevs.org/glossary#swe-bench"
---

# SWE-bench

SWE-bench is a benchmark that asks a model or agent to fix real GitHub issues by editing the repository, then runs the project's tests to check the patch. The original set has 2,294 problems from 12 Python repositories.

SWE-bench Verified is a 500-problem subset that engineers confirmed as solvable. Compare scores only within one dataset and one agent setup, because the code around the model changes the result.

## Example

A score of 70% on Verified means 350 of the 500 patches applied and passed the required tests.

## Source

[Jimenez et al., SWE-bench: Can Language Models Resolve Real-World GitHub Issues? (2023)](https://arxiv.org/abs/2310.06770)

## Related terms

- [Eval](https://aifordevs.org/glossary#eval)
- [Golden dataset](https://aifordevs.org/glossary#golden-dataset)

## Read the article

[Best LLM for coding: November 2026 benchmarks, compared](https://aifordevs.org/articles/best-llm-for-coding)

## Cite this term

AI for Devs. "SWE-bench." https://aifordevs.org/glossary#swe-bench

- Page: https://aifordevs.org/glossary#swe-bench
- Markdown: https://aifordevs.org/glossary/swe-bench.md
- Reuse: Quote or summarize with a link to this page. Terms: https://aifordevs.org/reuse
- Publisher: AI for Devs