You will understand exactly what a context window is, how tokens are counted and why that number surprises you, what happens inside the transformer when the window fills, how to architect around limits without losing meaning, and how EngineerPrep's own lesson pipeline hit every one of these…
Picture this JIRA ticket: EP-2847 — AI Tutor crashes on long resumes . The report is simple: a user pastes a 6-page resume, adds a job description, and asks the AI tutor for a gap analysis. The Spring Boot service calls Amazon Bedrock, Claude returns a 400, the global exception handler swallows it into a generic 500, and the user sees 'Something went wrong.' You pull the logs. The error message from Bedrock is unambiguous: ValidationException: Input is too long for requested model. You count the words in the resume and JD together — roughly 4,200 words. Claude 3 Sonnet's context window is advertised as 200,000 tokens. Four thousand words shouldn't be anywhere near that. So you drop into the Spring AI ChatClient call and add a log line that prints the token count estimate. The number that comes back stops you cold: 11,400 tokens — for 4,200 words. You're not even close to 200k.…
We already saw the symptom. Let's build the mental model. What exactly is a context window? A context window is the maximum number of tokens a transformer model can attend to in a single forward pass — simultaneously, all at once. Not a rolling buffer. Not a stream it reads sequentially. Every token in the window is directly visible to every other token through the attention mechanism. When you send a request to Claude or GPT, the model sees the entire payload as one flat sequence of tokens: system prompt + conversation history + user message + any injected tool results. That sequence must fit within the window or the request is rejected before inference even starts. The number you see advertised — 8k, 32k, 128k, 200k — is measured in tokens , not words or bytes.…
Scene Setup Imagine a long, narrow swimming pool — 8,192 lanes wide for our Claude Instant example. Each lane is one token slot. The pool is empty at the start of every request. Above the pool floats a scoreboard showing two numbers: Used and Remaining . To the right of the pool stands a bouncer — the Bedrock API gateway — whose only job is to reject any request where Used 8,192 before it ever reaches the model. Waiting at the entrance are four groups of people, each carrying a bag labeled with their role. The groups arrive in a fixed order every single time, no exceptions. Animation 1 — The System Prompt Enters The first group — dressed in gray, labeled System Prompt — walks into the pool and fills lanes 1 through 3,700. The scoreboard reads: Used 3,700 / Remaining 4,492. Pause here. Notice that the user hasn't typed a single character yet and already 45% of the pool is full.…
The attention matrix is the ceiling At the heart of every transformer layer is the self-attention operation. For a sequence of n tokens, the model computes an n×n matrix where cell (i,j) represents how much token i should attend to token j. At n=8,192, that's 67 million cells — per layer, per attention head. A 40-layer model with 32 attention heads is computing ~85 billion attention values per forward pass. This is why GPU memory, not compute, is usually the binding constraint for long-context models. The practical implication: context window size is determined at training time and is inseparable from the positional encoding scheme. You cannot take a model trained with 8k-token position encodings and simply ask it to handle 200k tokens at inference time — the positional embeddings don't generalize that far.…