Streaming lets your server send data one small piece at a time so the user sees progress immediately — instead of waiting for everything to finish before seeing anything.
Picture this: a learner opens EngineerPrep and asks the AI tutor to explain recursion. The request hits the Spring Boot backend, which calls Claude on Amazon Bedrock. Claude is a large language model — a system that generates text one token at a time, where a token is roughly a word or word-fragment. Claude starts generating immediately. Tokens are ready within milliseconds. But the learner sees nothing for eight seconds. Then the entire explanation appears at once. Your first instinct is probably: the server is too slow, we need to optimize it . But here is the surprising truth — Claude was producing tokens the whole time. The server was just collecting every token into one big response, waiting until the very last token was ready, and only then sending anything to the browser. The user wasn't waiting for work to finish. They were waiting for the server to stop hoarding the work .…
Think about how a restaurant works. One approach: the kitchen cooks every dish for the whole table, then a waiter carries everything out at once. You sit there hungry, smelling nothing, until the whole order is done. The other approach: each dish comes out the moment it is ready. Your soup arrives first. Then your salad. You are already eating before the main course is cooked. Streaming is the second approach for data. Instead of building the entire response in memory and sending it in one big chunk, your server sends each small piece — called a chunk — the moment it is ready. The client (the browser, or another service) receives and displays each chunk right away. The total amount of work is identical. The time-to-first-byte — how long until the user sees anything — shrinks significantly, often from several seconds to under a second.…
Imagine a timeline with two lanes: the server on top and the browser on the bottom. Time moves left to right. Without streaming: Server: generating word 1 generating word 2 ... generating word 40 SEND ALL 40 WORDS Browser: ......waiting....................................... receives all 40 words displays The browser lane is empty until the very end. The user stares at a blank screen. With streaming: Server: gen word 1 SEND gen word 2 SEND gen word 3 SEND ... Browser: word 1 ✓ word 2 ✓ word 3 ✓ ... Now let's walk through it one step at a time. Step 1 — Connection opens. The browser sends one HTTP request to EngineerPrep's backend. The server accepts it and keeps the connection open. This is normal HTTP — no special protocol needed. Step 2 — First chunk arrives. Claude generates its first token: "Recursion" . The backend immediately writes that token into the open HTTP response.…
Here is a simplified version of how EngineerPrep's AI tutor endpoint streams a response from Claude via Spring AI. We have three pieces: 1. A @Service that calls the LLM and returns a stream. 2. A @RestController that sends each chunk to the client. 3. The return type Flux<String — a Spring reactive type that represents a sequence of values arriving over time (think of it as a conveyor belt of strings). Let's read through it. java // TutorService.java @Service public class TutorService { private final ChatClient chatClient; public TutorService(ChatClient chatClient) { this.chatClient = chatClient; } // Returns a Flux…