---
title: "Small language model"
type: "glossary term"
url: "https://aifordevs.org/glossary#small-language-model"
markdown: "https://aifordevs.org/glossary/small-language-model.md"
publisher: "AI for Devs"
license: "https://aifordevs.org/reuse"
attribution: "Credit AI for Devs and link to https://aifordevs.org/glossary#small-language-model"
---

# Small language model

A small language model is a language model small enough to run on a single consumer GPU, a laptop, or a phone, usually under about 10 billion parameters. It suits narrow tasks, low latency, and keeping data on the device.

Belcak et al. (2025) define the term by device fit: a model that runs on a common consumer device at low enough latency to serve one user. They say they would count most models below 10 billion parameters, and they argue that many agent calls do not need a larger model.

## Example

An 8-billion-parameter model quantized to 4 bits takes about 4 GB, which fits in a laptop's memory with room for the KV cache.

## Source

[Belcak et al., Small Language Models are the Future of Agentic AI (2025)](https://arxiv.org/abs/2506.02153)

## Related terms

- [Quantization](https://aifordevs.org/glossary#quantization)
- [Distillation](https://aifordevs.org/glossary#distillation)
- [GGUF](https://aifordevs.org/glossary#gguf)

## Cite this term

AI for Devs. "Small language model." https://aifordevs.org/glossary#small-language-model

- Page: https://aifordevs.org/glossary#small-language-model
- Markdown: https://aifordevs.org/glossary/small-language-model.md
- Reuse: Quote or summarize with a link to this page. Terms: https://aifordevs.org/reuse
- Publisher: AI for Devs