---
title: "AI gateway"
type: "glossary term"
url: "https://aifordevs.org/glossary#ai-gateway"
markdown: "https://aifordevs.org/glossary/ai-gateway.md"
publisher: "AI for Devs"
license: "https://aifordevs.org/reuse"
attribution: "Credit AI for Devs and link to https://aifordevs.org/glossary#ai-gateway"
---

# AI gateway

An AI gateway is a proxy between your application and one or more model providers that gives you one API for many models. It also adds routing, fallbacks, caching, rate limits, cost tracking, and logging in one place.

A gateway sits in the request path, so it adds a network hop and needs its own uptime plan. It pays off once you call more than one provider, need per-team budgets, or want failover without code changes. A gateway's response cache is separate from provider prompt caching, which discounts repeated prefixes on the provider's side.

## Example

When provider A returns a 429, the gateway retries the same request on provider B, and your application code receives one successful response.

## Related terms

- [Prompt caching](https://aifordevs.org/glossary#prompt-caching)
- [Guardrails](https://aifordevs.org/glossary#guardrails)

## Cite this term

AI for Devs. "AI gateway." https://aifordevs.org/glossary#ai-gateway

- Page: https://aifordevs.org/glossary#ai-gateway
- Markdown: https://aifordevs.org/glossary/ai-gateway.md
- Reuse: Quote or summarize with a link to this page. Terms: https://aifordevs.org/reuse
- Publisher: AI for Devs