Deployment & Scaling · Advanced
Evaluating and Observing GenAI on AWS: Bedrock Evaluations and CloudWatch
Replace 'it seems better' with a number — automatic evaluations, LLM-as-a-judge, human review workforces, RAG-specific scoring and OpenTelemetry traces.
- 50 min lesson
- advanced level
- AWS AI Track
- Narrated presentation
What you’ll learn
- Automatic evaluation with built-in or custom prompt datasets
- LLM-as-a-judge and calibrating the judge against human scores
- Human evaluation work teams and worker instructions
- Separating retrieval quality from generation quality in RAG
- OpenTelemetry traces via AgentCore Observability and CloudWatch
- A frozen golden dataset as the prerequisite for all of it
Inside this lesson
A narrated, slide-by-slide presentation with 1 prediction exercise, 1 knowledge check, 4 videos. You answer as you go, and the platform checks your work.