Topic

Prompting and context

What goes into the context window and in what order. System prompts, examples, retrieved data, tool results, and memory.

Context engineering, skills, system prompts01 article06 glossary termsNo platforms

Articles

Read on this topic

Deep dives and guides filed under this topic, newest first.

All articles

Guide

Context engineering: budget and order the context window

Context engineering is choosing what a model reads on each call. Size each section, order it for prompt caching, trim inputs, and test every layout change.

Nov 06 202611 min readPrompting and context

Glossary

Terms you will meet

Context engineering: Context engineering is the practice of choosing what goes into a model's context window for each call, and in what order: instructions, examples, retrieved documents, tool results, and memory. You treat the window as a limited budget and spend it on what the next step needs.
Context window: A context window is the maximum number of tokens a model can handle in one request, usually counting the prompt and the output together. Everything the model has to work with for that request has to fit inside it.
Prompt caching: Prompt caching is a hosted API feature that reuses the processed state of a repeated prompt prefix, such as a long system prompt or a document, across calls. Providers that offer it bill cached input tokens at a discount and return the first token sooner.
System prompt: A system prompt is a set of instructions placed before the conversation that sets the model's role, rules, and output format for every turn. Keeping it stable makes it a good candidate for prompt caching.
Temperature: Temperature is a sampling setting that divides the model's logits before they are converted to token probabilities. Lower values make output more deterministic, and higher values make it more varied.
Top-pNucleus sampling: Top-p, or nucleus sampling, is a sampling method that keeps only the smallest set of most likely tokens whose probabilities add up to at least p, then samples from that set. The candidate count adapts to how concentrated the distribution is at each step.