AI engineering,
from scratch to production.
The engineering discipline underneath production LLM systems — inference, retrieval, evaluation, observability, safety, and cost — taught through the lens of the decisions a Senior or Principal PM has to make.
The modules
Foundations
The mindset shift from “writing prompts” to “engineering systems.”
- harness engineering
- context engineering
- infra not demos
Inference internals
What happens between your request and the tokens that come back.
- prompt vs semantic caching
- kv cache management
- prefill vs decode
- +3 more…
Reliable outputs & tool use
Making models produce things downstream systems can trust.
- structured output
- function calling
- agent guardrails
- +1 more…
RAG & retrieval
Grounding models in your data — and proving they actually used it.
- rag architecture
- retrieval evals
Evals & observability
You cannot operate what you cannot measure.
- evals
- observability
- cost attribution
Safety & multi-tenancy
Keeping tenants, users, and data from leaking into each other.
- safety engineering
- multi tenant isolation
Strategy & tradeoffs
Picking the right tool, and naming the cost of every choice.
- finetune vs icl vs rag
- inference stack tradeoffs
- production failure modes
Threads — follow one concern across the whole stack
How spend and latency hide in the inference layer — and how a cache can become a privacy incident.
From valid JSON to bounded agents: making stochastic output safe for downstream systems.
Why “it feels slow” has two different causes, and which lever fixes which.
Context → retrieval → grounding → evals: where trustworthy answers actually come from.
Fine-tune vs. ICL vs. RAG vs. distillation — the highest-leverage early decision.