A context window is the maximum amount of text — measured in tokens, not words or characters — that an AI model can hold in view at one time when it generates a response. Everything the model currently “knows” about a conversation, a document, or a codebase has to fit inside that window; anything older or larger simply isn’t there for the model to use.

How It Works

Think of the context window as a model’s working memory rather than its long-term knowledge. A large language model learns general patterns during training, but it doesn’t “remember” your specific conversation the way a person would — it has to be shown the relevant text again in every request. That text includes the system instructions, every prior message in the chat, any documents or images you’ve attached, and, in AI coding or agent tools, the definitions of the tools it can call. The model’s own reply, once generated, becomes part of that same pool for the next turn.

As a conversation grows, each new message adds to the total. Once the running total reaches the model’s limit, something has to give: the oldest turns get dropped, summarized, or otherwise compressed so the conversation can continue — a process AI companies call context management or, in Anthropic’s terminology, compaction. Whatever falls outside the window is no longer visible to the model unless it’s explicitly reintroduced.

Context window sizes are usually described in tokens, where one token is roughly three-quarters of an English word. Early consumer chatbots worked with just a few thousand tokens — enough for a couple of pages of text. Frontier models have grown that dramatically: Claude Sonnet 5, for instance, ships with a 1-million-token context window by default, while several other current Claude models offer 200,000 tokens, according to Anthropic’s developer documentation. A million tokens is roughly 750,000 words — around 3,000 pages — enough to hold an entire codebase or a small library of documents in a single request.

Why It Matters

A bigger context window sounds like a straightforward upgrade, but it comes with real trade-offs. Processing more text takes more computing power, which can slow down responses. More importantly, models don’t use a full context window evenly: research and vendor documentation both describe a pattern often called “context rot” or the “lost in the middle” effect, where information buried in the center of a very long input gets less attention than material near the beginning or end. Simply dumping everything into the window rarely produces better answers than giving the model a smaller, well-chosen set of relevant text — which is one reason techniques like retrieval (fetching just the relevant passages on demand, the idea behind RAG) built on the transformer architectures behind most modern models remain useful even as raw context limits grow.

Context length can also affect price, though not always the way people assume. Some providers have historically charged more once a request crosses a size threshold. Anthropic, by contrast, now bills its 1-million-token-capable models at a flat per-token rate regardless of length — a 900,000-token request costs the same per token as a 9,000-token one, per its pricing documentation. Pricing structures vary by vendor and change often, so it’s worth checking a provider’s current pricing page rather than assuming a rule that applied to an older model still holds.

For everyday users, the context window explains a familiar frustration: a chatbot that seems to “forget” something mentioned earlier in a long conversation likely didn’t forget in the human sense — the message aged out of the window, or got compressed away to make room for newer messages. Starting a fresh conversation, summarizing key points yourself, or using a tool built for long documents are all practical ways to work around the limit rather than fight it.

In the News

Context window size has become a genuine selling point in model releases. When Anthropic launched Claude Sonnet 5, its 1-million-token context window was one of the headline capabilities, letting the model work across much larger codebases and document sets in a single pass than its predecessors could.

FAQ

Is a bigger context window always better?
Not automatically. Larger windows cost more to process and can be slower, and models don’t always use every part of a long input equally well. Careful selection of what goes into the context often beats simply maximizing size.

What happens when a conversation exceeds the context window?
Behavior varies by product. Some tools stop and return an error; many chat interfaces and coding agents automatically drop, summarize, or compress older messages so the conversation can continue.

Is a context window the same as a model’s training data?
No. Training data is the (much larger) body of text a model learned general patterns from, long before you interact with it. The context window is the specific text visible during one particular conversation or request.

Do context window and token limits mean the same thing?
They’re closely related. The context window is measured in tokens, and “token limit” is often used as a synonym for context window size — though some products also cap output length separately from the overall context limit.