Learning path: AI Engineer
Last reviewed: 2026-10 · Volatility: medium
TL;DR
This path is build-first. It starts with how models learn, then moves through models and APIs, grounding and memory, agents and the harness, and production. It is the longest path. The harness track is the core: you build a coding agent's parts by hand and end with a tested capstone.
Who it is for: a software engineer who builds AI-powered systems.
Outcome: You can build, test and run a retrieval-backed, tool-using agent, and explain how it fails.
Before you start: Comfortable Python and one backend stack. Basic linear algebra helps. The foundations stage covers what you need.
Length: about 84 hours, across 5 stages. The time is a rough estimate (see Under the hood).
Terms: a large language model (LLM) is a model like those behind chat assistants; the plural is large language models (LLMs); machine learning (ML) is the practice of training models from data; convolutional neural networks (CNNs) are a model type built for images; Extensible Markup Language (XML) is a tag-based text format.
🎯 For the AI engineer
Why it matters — Calling an API is easy. Making the result reliable, safe and affordable is the job.
What it changes in your decisions — You test before you tune. You put checks in code, not in prompts. You design for failure first.
Ask yourself — "If this step returns garbage, what stops it from doing damage?"
Risk if ignored — The demo works. Production loops, leaks or overspends, and nobody can say why.
The path at a glance
Each stage ends with a checkpoint. Move on when you can do what the checkpoint says.
| Stage | Focus | About |
|---|---|---|
| 1 | Foundations | about 14.5 hours |
| 2 | Models and APIs | about 6.5 hours |
| 3 | Grounding and memory | about 6 hours |
| 4 | Agents and the harness (build it) | about 29 hours |
| 5 | Production | about 27.5 hours |
Stage 1: Foundations
Learn how a model is trained and what its data looks like. Everything later builds on it. (about 14.5 hours)
- Machine learning: 1. What machine learning is, and when to use it · 2. Data: features, labels, splits and leakage · 3. Training: loss and gradient descent · 4. Generalization: overfitting and the bias–variance tradeoff · 5. Measuring a model: metrics, thresholds and baselines · 6. Model families: linear models, trees and neural networks · 7. ML in production: drift, skew and the lifecycle · 8. Learning and doing ML with Claude Code
- Tensors: 1. What a tensor is: shape, rank and dtype · 2. Operations and broadcasting · 3. Matrix multiplication: the workhorse · 4. Batching, memory and precision · 5. Automatic differentiation
- CNNs: 1. What computer vision does: the four jobs · 2. How a convolution sees: one small grid, used everywhere · 3. From edges to objects: what depth buys · 4. Transfer learning and labels: start from someone else's model · 5. CNNs vs. vision transformers: picking a backbone · 6. Vision foundation models: CLIP, DINO and SAM · 7. Shipping vision in production
- Transformers: 1. The shape of a transformer · 2. Attention · 3. The block: heads, the feed-forward layer and the residual stream · 4. Position, and the cost of context · 5. Generating text and the KV cache · 6. Variants, and reading a model card
- Fine-tuning and post-training: 1. From base model to assistant: the post-training pipeline · 2. Supervised fine-tuning · 3. LoRA and parameter-efficient tuning · 4. Preference tuning and reward hacking · 5. The dataset: quality, duplicates, contamination and synthetic data · 6. Deciding, costing and checking a fine-tune
- LLMs: 1. What an LLM actually is · 2. The context window · 5. Temperature, sampling & determinism
Ready to move on when: You can explain what a token and a context window are, and how sampling changes the output. You can read a training curve and say whether a model is overfitting.
Stage 2: Models and APIs
Call models well: contracts, streaming, retries, structured output and prompts that hold up. (about 6.5 hours)
- LLMs: 3. Capabilities & the jagged frontier · 6. Choosing a model · 7. Prompting vs. RAG vs. fine-tuning
- APIs & integrations: 1. The request/response contract · 2. Calling an LLM API · 3. Structured output & JSON mode · 4. Webhooks & async patterns · 5. Integrating into existing systems · 6. MCP & standard connectors
- Prompt engineering: 2. The anatomy of a prompt · 4. Structured prompting: XML, delimiters, scaffolds · 5. Few-shot, chain-of-thought, and self-consistency · 6. Prompt chaining and multi-step workflows · 7. Prompting for tools and agents · 9. When prompts fail: the diagnostic playbook · 10. Prompts in production: versioning, testing, and model upgrades
Ready to move on when: You can call a model with retries and a typed output, and you can diagnose a failing prompt.
Stage 3: Grounding and memory
Give the model the right data, in the right shape, at the right time. (about 6 hours)
- RAG & vector databases: 1. When to use RAG, and when not to · 2. Why RAG? · 3. Embeddings & semantic search · 4. Vector databases · 5. Chunking & ingestion · 6. Retrieval quality · 8. Beyond flat RAG: GraphRAG & structured retrieval · 9. Building RAG with Claude Code, pgvector and Pinecone · 10. RAG terms, explained
- AI engineering: RAG: RAG architecture: chunking, embeddings, hybrid search, reranking, and freshness · Retrieval evals: recall, precision, grounding, attribution, and citation quality
- Memory & context: 3. Writing and maintaining memory · 4. Retrieval as memory
- Context engineering: 3. The anatomy of a context pipeline · 6. Evaluating context quality
Ready to move on when: You can build a retrieval pipeline and measure its quality with a test set.
Stage 4: Agents and the harness (build it)
Build the parts of an agent by hand. This is the core of the path. (about 29 hours)
- Tool calling: 2. Tool contracts & reliability · 3. Permissions, blast radius & the trust boundary
- AI engineering: reliable outputs: Function calling reliability, tool contracts, argument validation, and idempotency
- AI agents: 2. Planning, reasoning & reliability across a run
- Agentic workflows: 1. Choosing a workflow pattern · 2. Orchestrating more than one agent · 3. Making a workflow durable, and worth owning
- Harness engineering: Phase 01 — Foundations and the Loop (5 lessons, build it)
- Harness engineering: Phase 02 — Tools (3 lessons, build it)
- Harness engineering: Phase 03 — Context and memory (5 lessons, build it)
- Harness engineering: Phase 04 — Prompts and instructions (3 lessons, build it)
- Harness engineering: Phase 05 — Files and Shell (6 lessons, build it)
- Harness engineering: Phase 06 — Permissions and Security (4 lessons, build it)
Ready to move on when: You have a working loop with tools, a permission gate and a context budget, and tests that fail when you break them.
Stage 5: Production
Plan for failure: extend, test, observe, secure and scale. (about 27.5 hours)
- Harness engineering: Phase 07 — Planning and Subagents (4 lessons, build it)
- Harness engineering: Phase 08 — Extending: MCP, Skills, Retrieval (5 lessons, build it)
- Harness engineering: Phase 09 — Reliability, evals, and ops (5 lessons, build it)
- Harness engineering: Phase 10 — Capstone (1 lessons, build it). Allow about 3 hours: the capstone plus your own evals, traces and cost.
- AI agents: 4. Running an agent in production · 5. Choosing and acceptance-testing an agent
- AI-assisted development: 3. Verification is the product · 4. Control: permissions, modes and hooks · 5. Running at scale: sessions, subagents and fan-out
- AI engineering: inference internals: Prefill vs. decode latency · Continuous batching & paged attention · KV cache management: eviction, reuse, and memory pressure at scale · Prompt caching vs. semantic caching
- AI engineering: reliable outputs: Model routing, graceful fallback logic, and degraded-mode UX · Structured output: validation, repair loops, and fallback chains · Agent guardrails: loop budgets, tool budgets, and termination conditions
- AI engineering: evals & observability: Evals: golden sets, regression tests, adversarial tests, LLM-as-judge, and human evals · LLM observability: traces, spans, tokens, latency, errors, and drift · Cost attribution per feature, workflow, tenant, and user journey — not just per model
- AI engineering: safety & multi-tenancy: Safety engineering: prompt injection defense, data leakage prevention, and permission boundaries · Multi-tenant isolation, cache safety, and cross-user context contamination prevention
- AI security & guardrails: 1. The threat model, and guardrails as architecture · 2. Red-teaming: testing your defenses
- Access control: 1. Authentication, authorization and the access-control model · 2. OAuth 2.0, OpenID Connect and tokens · 8. Access control for AI agents
- System design: 1. Foundations & framework · 2. Numbers, formulas and papers to know · 7. Real-time systems
Ready to move on when: Your capstone passes its tests, and you can show its evals, traces, permission rules and cost.
Skip-ahead rules
- If you know machine learning, skip the foundations stage and start at models and APIs.
- If you only work with hosted models, read the inference lessons as background and skip the deep ones.
- If you want a faster tour, build harness phases 1, 2, 3 and 6 and skim the rest.
Worked example: a week-by-week plan
This example is invented, to show the method.
An AI engineer with nine hours a week plans 8 weeks of work. Each week ends with something you can show. At this pace the full path (about 84 hours) takes about 10 weeks.
| Week | Stage | What you do |
|---|---|---|
| 1 | Stage 2 | Call a model through an API with retries and a typed output. |
| 2 | Stage 3 | Add retrieval over your own documents and measure it with ten questions. |
| 3 | Stage 4 | Build harness phase 1: the loop. Run its asserts. |
| 4 | Stage 4 | Build harness phases 2 and 3: tools and context. |
| 5 | Stage 4 | Build harness phases 4 and 5: prompts, files and shell. |
| 6 | Stage 4 | Build harness phase 6: permissions and security. |
| 7 | Stage 5 | Build harness phases 7 and 8: planning, subagents and MCP. |
| 8 | Stage 5 | Build phase 9 and the capstone. Break the capstone on purpose and watch it fail. |
Tradeoffs
- Build vs. read. The harness lessons ask you to type the code. Skimming saves time and loses the failures you would have hit.
- Depth vs. breadth. The inference lessons go deep on serving. Read them if you run models, skim them if you call hosted ones.
- Frameworks vs. from scratch. Building by hand first makes later framework choices easier to judge.
Failure modes
- Tuning before testing. Build the eval set first. Tuning without one is guessing.
- Checks in prompts. A rule in a prompt is a request. A rule in code is a control.
- Skipping the permission gate. An agent without one is a security incident waiting for a trigger.
- No failure tests. Break your agent on purpose and watch the tests catch it.
Practitioner checklist
- Does every model call have a timeout, a retry rule and a typed output?
- Do I have a test set for retrieval and for the agent?
- Is every tool call behind a permission check in code?
- Is there a budget on steps and cost?
- Can I trace one request from input to answer?
Under the hood: how this path was built
Lesson links come from the track build configs, so a renamed lesson is caught by the link check. Time is the number of lessons times 25 minutes for a reading lesson, times 60 minutes for a harness lesson that you build, and 120 minutes to skim a harness phase. Add time for exercises. Modules marked "coming soon" are planned and not counted.
Related lessons
Sources
Lesson counts and titles come from the track overview pages and build configs in this repository. The time estimates are this course's rule of thumb, not measured results. The week plan is invented.