A Model Gateway is a single, consistent interface that sits in front of every AI model your app uses — so the rest of your code never has to care which model is actually running.
Picture the EngineerPrep codebase on a Tuesday morning. You have a LessonAuthorService that calls Claude 3 Sonnet on Amazon Bedrock. It works. Learners are happy. Then the product team opens a JIRA ticket: "Try GPT-4o for lesson generation — might be cheaper." You add an if statement. A new SDK import. A second API key in your config file. Two weeks later, another ticket: "Can we route quiz scoring to a smaller, faster model to save cost?" Another if . Another SDK. The retry logic is now copy-pasted in three places. Now imagine a model provider has an outage. Which service blows up? You have to grep the whole codebase to find out. This is the problem a Model Gateway solves — and once you see it, you can't unsee it.
Think about a power strip. You don't care whether your lamp runs on electricity from a coal plant, a solar panel, or a wind turbine. You plug the lamp into the strip, and the strip handles the rest. The lamp's plug never changes. A Model Gateway is that power strip for AI models. Your application code — the lesson generator, the AI tutor, the mock interview scorer — always talks to one interface. Something like ChatClient or LlmGateway . That interface accepts a prompt and returns a response. Period. Behind that interface, the gateway routes the call to whichever real AI model is configured: Claude on Bedrock today, GPT-4o tomorrow, a local model in your test environment. The rest of your code never knows, and never needs to. In EngineerPrep, this is exactly what Spring AI's ChatClient provides, backed by a configured provider — Amazon Bedrock with Claude.…
Step 1 — Your service makes one call. Imagine LessonAuthorService sitting on the left side of a diagram. It holds a reference to ChatClient — the gateway interface. It calls chatClient.prompt(lessonPrompt).call().content() . That's it. It has no idea what's on the other side. --- Step 2 — The gateway receives the request. In the middle of the diagram is the gateway itself. Think of it as a smart router. It receives the prompt as a plain Java string (or a structured message object). At this moment it can do useful work before forwarding: log the request, check a budget limit via Capstead, or attach a system prompt that applies to all calls. --- Step 3 — The gateway consults configuration. A small config box feeds into the gateway: spring.ai.bedrock.claude.model=claude-3-5-sonnet-20241022 .…