Introduction
If you have ever studied for an AWS certification, you probably have a routine. Mine goes like this. I buy a Udemy course, work through it, and whenever something does not click I open up Claude and talk it through until it does.
It works, but it has gaps. Claude does not know which exam I am studying for unless I tell it. It does not know which questions I got wrong yesterday, or that I keep mixing up the same two services. Every new chat starts from zero, and I end up explaining the same context over and over again.
So I built something to close those gaps for myself. It is called AWS Cert Coach.
Why I built it
Two reasons, really.
The first is selfish. I am studying for the newer Generative AI Developer Professional exam at the moment, and I wanted the questions and the conversation in one place, with something on the other end that already knows the exam.
The second is that I wanted to get properly hands on with Amazon Bedrock AgentCore. Reading about agents is one thing. Building one that people actually use, with real tools and real data behind it, is a different thing entirely, and it is the only way I really learn anything.
What it does
At its core, it is a practice exam site with a tutor sitting next to you.
There are three ways to use it:
- Exam mode is the straight practice exam. Timed, full length and scored at the end, with a breakdown by domain so you can see where you are weak.
- Free mode is where the tutor lives. You get questions one at a time, answer them, and then talk it through. Ask why your answer was wrong, ask for a question on a topic you are struggling with, or ask it to explain a service in plain English.
- Review mode lets you go back over a finished exam with the tutor and pick apart the ones you missed.
The part I am happiest with is memory. The tutor remembers previous sessions, including the misconceptions you keep falling into. If you mixed up two S3 storage classes last week, it knows, and it can come back to that instead of starting from scratch every time.
The question pool also looks after itself. When a topic starts running low, new questions get written in the background from the AWS documentation, and they are checked before they go anywhere near a learner.
The building blocks
I am keeping this high level, but here is roughly what it is made of:
- Amazon Bedrock runs the models. Claude does the tutoring, as well as writing and checking the questions.
- Amazon Bedrock AgentCore is the main event:
- Runtime hosts the tutor agent.
- Gateway turns a handful of Lambda functions into MCP tools the agent can call, and puts them behind one endpoint with the AWS knowledge MCP server.
- Policy sits in front of the Gateway and has the final say on which tool calls go through.
- Memory is what lets the tutor remember you between sessions.
- The AWS Knowledge MCP server gives the tutor and the question writer access to the AWS documentation, so answers come from the docs rather than whatever the model half remembers.
- Strands Agents is the framework the tutor is written in.
- AWS Lambda, Amazon API Gateway and Amazon DynamoDB handle the regular app side of things, like exams, answers and progress.
- Amazon Cognito takes care of sign in.
- Amazon S3 and Amazon CloudFront serve the website.
- Terraform builds all of it.
Nothing exotic. Most of the effort went into how these pieces work together rather than the pieces themselves.
What I learned
Writing good exam questions is hard.
Harder than building the agent, if I am honest. A model will happily write you a question that reads well, sounds confident and has the wrong answer marked as correct. Getting to questions I would trust took far more attempts than I expected, and it is still the part I most want to improve.
My first version of free mode was awful.
Questions repeated, topics I asked for were ignored, and at one point the tutor told me, with total confidence, that a topic did not exist. None of that showed up in testing. It only came out once I started using it to study, so I ended up tuning it on the fly.
Not everything needs to be agentic.
Early on I was asking the model to do jobs that a few lines of code could do better, like picking the next question or keeping track of what you had already seen. It got them wrong just about every time. Moving those jobs into plain code made the whole thing calmer and more predictable.
There is a lot of pressure at the moment to put an agent in every workflow, whether it needs one or not. If the steps are known and the answer has to be right every time, that is a deterministic workflow and a job for code. The agent earns its place where the job needs reasoning, like explaining an answer, working out what the learner actually means, or deciding what to look up. In this project the model is there to reason, explain and teach, not to keep the books.
Wrapping up
It is still a proof of concept. The question writing needs more work, and I want to add a few more exams.
But it does the thing I wanted it to do. When I get a question wrong now, I can ask why, get a straight answer, and know it will bring that topic back around later. That beats starting a fresh chat and explaining myself all over again.
Written by a human, made legible with AI.