Learn what drives LLM API cost in a Spring Boot system, which levers actually reduce it, and how EngineerPrep's Capstead starter tracks spend so you can act before the bill surprises you.
Picture this: it's Monday morning and your team's AWS cost dashboard sends its weekly digest. Last week EngineerPrep's Bedrock spend jumped 40%. Nobody shipped anything obviously expensive. The feature that went out was a small UX improvement — a friendlier loading message on the mock-interview screen. You dig in. The loading message calls the AI tutor to generate an encouraging tip. That tip call sends the full user profile, the full lesson history, and a long system prompt — every single time the loading screen appears. The screen appears a lot. Here is the trap most engineers fall into: they think cost is about what the model does . It isn't. It is almost entirely about how much text you send and receive .…
LLM APIs like Amazon Bedrock charge by the token . A token is roughly three to four characters of English text — think of it as a short syllable. 'EngineerPrep' is about three tokens. A 500-word paragraph is roughly 650–700 tokens. Every API call has two sides: the input (everything you send — your system prompt, conversation history, the user's question) and the output (everything the model writes back). Both sides cost money. Input is typically cheaper per token than output, but input is also usually much larger in practice. Here is the everyday analogy. Imagine you hire a consultant who charges by the word — both for reading your brief and for writing the report. If you hand them a 50-page background document every single meeting, even for a five-minute check-in, you pay for those 50 pages every time.…
Step 1 — The request arrives. A learner submits their answer to an interview question. The Spring MockInterviewService receives the REST call. At this point no tokens have moved anywhere. Step 2 — The context is assembled (this is where cost is born). The service fetches: the system prompt (≈300 tokens), the last 20 messages of conversation history (≈2 000 tokens), the full lesson the question came from (≈1 500 tokens), and the learner's answer (≈120 tokens). Total input: roughly 3 920 tokens before the model sees a single word. Picture a suitcase being packed. Every item you throw in costs money to ship — even if the recipient only needed one of them. Step 3 — The call goes to Bedrock. Capstead wraps the call and stamps it with the feature name ( mock-interview-feedback ), the model ID, and a timestamp. This is the moment the meter starts. Step 4 — The model responds.…
This walkthrough shows a simplified version of EngineerPrep's MockInterviewFeedbackService . We will build it up in three small steps so each lever is visible. Step 1 — The naive version (no optimization). Every call sends full history and the full lesson text. This is what most engineers write first. Step 2 — Add context trimming. We cap history at 5 messages and replace the lesson body with a short summary fetched from the database. Step 3 — Add output capping and Capstead cost tagging. We set maxTokens on the request and let Capstead record the call. Read the inline comments — each one explains why a line exists, not just what it does. Notice: - Constructor injection everywhere — no @Autowired on fields. - LessonSummaryRepository returns a short String instead of a full Lesson entity — only fetch what you need.…