How Context Works
AI models don't have memory the way humans do. Between sessions, they remember nothing — every new conversation starts completely blank. But within a single session, the model has access to everything that's been said so far. That accumulated text is called the context.
What the model actually sees
Every time you send a message, the model doesn't just read your latest message. It reads the entire conversation from the beginning — your opening message, its response, your follow-up, its response, and so on — plus any system prompt, plus your current message. All of it, every time.
This means your conversation is constantly growing. Each exchange adds more text to what the model must process.
The context window
Every model has a maximum amount of text it can process at once — the context window. Think of it like a desk. You can spread out a certain number of documents. Once the desk is full, something has to come off to make room for new material.
Context windows are measured in tokens (the text chunks covered in the previous module). A 100,000-token context window can hold roughly 75,000 words — about the length of a typical novel. That sounds large, but it fills up faster than you'd expect when you're pasting in documents, code, or long back-and-forth conversations.
What happens when you fill it up
Different models handle a full context window differently, but the common outcomes are:
Truncation. The oldest parts of the conversation are dropped to make room for new input. The model simply can't see what happened earlier, and may contradict itself or lose track of instructions you gave early on.
What truncation looks like
Dropped
Oldest content — removed first
- Your first question
- Early context you set up
- Initial instructions
- Old back-and-forth
The model cannot see this — as if it never happened.
Kept
Recent content — what the model sees
- Recent exchanges
- Your latest question
- Most recent instructions
- Current context
The model only works with what fits.
Conversation timeline
Degraded coherence. Even before hitting the hard limit, research shows that models struggle to accurately use information buried deep in a long context. They tend to pay more attention to the beginning and end of the context, and less to the middle — sometimes called the "lost in the middle" problem.
How the model weighs your conversation
Beginning
Strong recall
- ·System instructions
- ·Your first message
- ·Key context you set up
Middle
Weaker recall
- ·Details from hours ago
- ·Earlier corrections
- ·Things mentioned once
End
Strong recall
- ·Your latest message
- ·Most recent instructions
- ·What you just asked
Attention level across the conversation
Instruction drift. If your system prompt or early instructions get pushed far from the current exchange, the model may gradually stop following them as precisely.
How instructions weaken as conversations grow
Short conversation
Instructions are close to the current exchange — the model follows them reliably.
Long conversation
After many exchanges, early instructions are buried in the low-attention zone — the model may quietly stop following them.
The practical experience
You may have noticed this without knowing the cause: a long conversation with an AI that starts going sideways. It forgets something you told it. It gives an answer that contradicts something from earlier. It loses the thread of what you were trying to accomplish.
This isn't the model getting tired or bored. It's context filling up.
When a session starts to feel like the AI has "forgotten" what you're doing, the most effective fix is to start a fresh conversation — and if needed, paste in a compact summary of the key context rather than relying on the model to retain it across hundreds of exchanges.