A context window is the fixed-size reading list an LLM uses on every call — understanding it tells you why AI seems to forget things and how to design around that limit.
Picture this: you're reviewing a support ticket for EngineerPrep's AI tutor feature. A learner is working through a Java concurrency lesson. They ask a great question, get a solid answer, then ask a natural follow-up. The AI's reply makes no sense — it's answering a completely different question, as if the conversation started over. You dig into the logs. The HTTP call succeeded. The prompt was sent. The model responded. Everything looks fine on the surface. But the learner is confused, the conversation is broken, and you don't know why. Here's the question this lesson answers: how does an AI model actually 'remember' a conversation — and what happens when that memory runs out?
An LLM (Large Language Model — the kind of AI behind EngineerPrep's tutor, powered by Claude on Amazon Bedrock) does not have memory the way you and I do. It doesn't store anything between calls. Every single time you send it a message, it starts completely fresh. So how does it seem to remember earlier parts of a conversation? Simple: you send the whole conversation every time. Think of it like a whiteboard. Each time you ask the AI a question, you erase the whiteboard and rewrite everything from the very beginning — the system instructions, every previous message, and the new question. The AI reads the whole whiteboard and replies. Then you erase it again for the next turn. The context window is the size of that whiteboard. It's measured in tokens — a token is roughly a word or a short fragment of a word, like a puzzle piece of text.…
Step 1 — The first message. Imagine a horizontal strip, like a film reel. At the start it's almost empty. The learner sends their first message: 'Explain thread safety in Java.' You send the model a whiteboard with two things on it: the system prompt ('You are a Java tutor') and that one message. The strip is maybe 10% full. The model reads the whole thing and replies. --- Step 2 — The conversation grows. The learner asks a follow-up: 'Can you give me an example with synchronized?' Now the whiteboard has four things: the system prompt, message 1, the AI's first reply, and message 2. The strip is 25% full. Still fine. The model reads everything and gives a great answer with a code example. --- Step 3 — Things keep building. After ten back-and-forth exchanges — questions, answers, code snippets, clarifications — the strip is 85% full. The model still has the whole conversation in view.…