Hugo & Stefan

Cohort 1

Cohort 2

Cohort 4

Cohort 3

Cohort 5

This is Chapter 1 of our 300-page course reader: yours free. The full reader spans 9 chapters covering the whole AI software development lifecycle, including evaluation, testing, RAG, agents, multimodal systems, and fine-tuning. It's the companion to our live course, taught by Hugo Bowne-Anderson and Stefan Krawczyk, with guest perspectives from practitioners at Google DeepMind, Pydantic, Chroma, and more. In this chapter, you'll learn why most AI projects stall after the demo: and the evaluation-driven development framework that gets them to prod.

Building AI Applications: The Reader

Want to build LLM-powered applications but not sure where to start? This handpicked guide features the best resources on Python fundamentals, LLM evaluation, AI agent design, and scaling generative AI systems. Learn from experts like Chip Huyen, Hamel Husain, Shreya Shankar, and Sebastian Raschka to accelerate your LLM development workflow and avoid common pitfalls. Whether you’re an engineer, researcher, or founder, this guide will help you build more reliable and production-ready AI systems.

🚀 Essential LLM & AI Engineering Resources: A Curated Guide for Engineers & Data Scientists

Want to build smarter AI agents? This handpicked guide features the best resources on AI agent design, planning, tool use, and failure modes. Learn from industry leaders like Anthropic, Chip Huyen, and Hugging Face to level up your LLM-powered agents and avoid common pitfalls. Whether you’re an engineer, researcher, or founder, this guide will help you build more reliable and capable AI agents.

Master AI Agents: A Handpicked Guide to the Best Resources

Build AI-powered software reliably & from first principles. Learn the GenAI software development lifecycle: agents, evals, iteration & more

AI Builder/Educator (6 Million Students)

Hugo Bowne-Anderson

AI & data engineer, consultant, educator of 6+ million students (ex-Yale)

AI Builder with 15 years of experience

Stefan Krawczyk

Agentforce; Ex-Stitch Fix; 13+ Years building / productionizing data, ML, Agents

Building AI Applications for Data Scientists and Software Engineers

Data Analyst

AI Leader

Data Manager

Enteprise Data Architect

Clinician Data Scientist

Student

DS & ML Mentor & Content Creator

AI ML Customer Engineer

IT Director

Consultant

Final cohort: 300+ engineers from Meta, Netflix, Amazon &amp; more have taken this course. Join them for the last run.If you’re a software engineer or data scientist, chances are you’ve built an impressive proof of concept (POC) that showcases the potential of LLMs.But these POCs often fail to scale into reliable, production-ready applications. The result? Endless iteration cycles, unreliable outputs, and frustration as teams struggle in what we call “POC purgatory.”It doesn’t have to be this way.With the right principles, processes, &amp; workflows, you can build LLM applications that work in production, without overcomplicating pipelines or chasing trends. This course focuses on the iterative development, evaluation, and debugging practices that make AI systems production-ready.You'll build with state-of-the-art tools like RAG, fine-tuning, MCP, context engineering, and agents, but the emphasis is on building scalable, robust systems, ensuring you leave with the mindset and skills to adapt as the technology evolves.Learn how to design, test, &amp; deploy generative AI applications, and iterate rapidly.Create AI systems that solve real problems, deliver real value, &amp; scale with confidence.

Master the patterns and processes that underpin successful GenAI app development

Move beyond POCs and build production-ready AI systems

Build State-Of-The-Art AI Agents

Accelerate your development process with best-in-class "Evals" & LLMs Judges

Develop hands-on expertise by building GenAI and LLM apps

Showcase Tangible Results

Data scientists, machine learning engineers who are sick &amp; tired of seeing and building prototypes &amp; want to ship reliable LLM applications

Software engineers who want to learn how to build Generative AI systems and learn the LLM software development lifecycle.

The Agentic Lifecycle with Opik: Trace, Evaluate, Iterate

Abby Morgan

AI Engineer at Comet, ex-Springboard

AI & data engineer, consultant, educator of 3+ million students (ex-Yale)

How to capture every step with Opik's @track decorator: tool use, retrieval, and decisions

Tracing your agent's reasoning

Build evaluation datasets, score agent outputs, and iterate rapidly with side-by-side experiments

Building evaluation pipelines for agents

Code walkthrough building and evaluating a multi-step research agent with Opik and OpenAI

Live end-to-end agentic build

How To Build Your First Agentic Search Application

Doug Turnbull

Led Search at Reddit and Shopify.

Why having an agent help a human search matters in many domains.

Why agentic search matters

Best practices for structuring the agentic search loop

How to structure search and exploration

Leveraging the agent’s deep need to please people to create more relevant search

Hacking in-prompt reinforcement learning

Ship AI-Generated Code You Actually Understand

Eleanor Berger

AI Leader, ex-Microsoft, ex-Google

Isaac Flath

Founder of Kentro Tech

Learn to divide large projects into specific, manageable pieces that you can build and test one at a time.

Break Complex Problems into Small Tasks

Learn exactly what information to give AI tools to get code that fits your specific needs and existing codebase.

Provide Clear Context to Get Better Results

Develop a systematic approach to reading and understanding every piece of code before adding it to your project.

Review AI-Generated Code Before Moving Forward

Making Sense of Millions of Conversations for AI Agents

Hugo Bowne-Anderson, PhD

Ivan Leo

AI engineer at Manus building the future of agents.

Compress millions of interactions without losing critical meaning.

Summarise LLM & Agentic conversation data at scale

Identify recurring failure themes and unmet user needs.

Embed and cluster conversations

Target fixes that actually improve satisfaction and retention.

Separate capability gaps from data gaps

Turn clusters into monitoring systems for continuous improvement.

Build classifiers and feedback loops

Link issues directly to metrics like cost, revenue, and churn.

Tie insights to business KPIs

Making LLM Agents Observable & Debuggable

Podcaster, Educator, DS & ML expert

Claire Longo

AI Researcher at Comet | Mathematician | Startup Advisor | ex-Arize AI 📈

LLM agents fail silently, hallucinate, and drift: learn to catch issues early with output checks, trace logs, & metrics.

How to debug and monitor agent behaviour in real-time

Use humans and LLMs to evaluate outputs with real-world workflows and practical examples you can apply immediately.

Work with human annotations and LLM's-as-a-judge

Give your IDE eyes and ears using Opik MCP to add telemetry and metrics, so you can spot and fix AI issues fast.

Using MCPs to level-up your vibe coding with telemetry

Get hands-on examples that work across LLMs and agent frameworks—apply these methods in your stack right away.

Start building today with open-source cookbooks

Don't Tweak Prompts. Engineer Agents.

Skylar Payne

AI made easy. AI executive for startups. Ex-Google. Ex-LinkedIn.

Go from staring at a wall of tokens to systematically finding and fixing bugs with a repeatable 5-step process.

Master the Core Eval Loop

Break down long, chaotic agent traces into clear, gradable checkpoints to instantly see where things go wrong.

Define Actionable Milestones

Use aggregate data to pinpoint the biggest leaks in your agent’s performance and focus your efforts for maximum impact.

Build Failure Funnels

Ship every new version with confidence by tracking task success, regressions, and costs in one holistic view.

Learn directly from Hugo & Stefan