LLMs for the product leader
Tokens, the context window, the jagged frontier, prompting, sampling, choosing a model, and the order to reach for prompting, RAG, or fine-tuning.
A large language model is the engine most of this curriculum's family runs on. It sits under the chat product, the coding assistant, the support agent, and the summarizer alike. Yet most product leaders treat it as a black box: text goes in, text comes out, and the details are "an engineering thing." That gap is expensive. The model's shape decides what your product can promise, what it costs, and how it fails — and every one of those decisions is a product decision, not just an engineering one.
This module opens the engine's hood, at the altitude a product leader needs. It teaches what a model actually does with your words, why it can only see a fixed amount of text at once, why its skill is uneven in a way that trips up first-time builders, how you talk to it well, why the same question can get a different answer twice, how to pick the right model for a job, and when prompting stops being enough. It does not repeat the deep engineering mechanics — those live in Inference internals and the strategy tradeoffs module. This module is the map that tells you when to open which of those doors.
The knowledge graph
An LLM is one idea, applied at scale, with consequences that ripple outward. Every lesson in this module hangs off this picture:
One idea, four consequences · the shape of this module
Every lesson hangs off the same mechanism — next-token prediction — read top to bottom as one chain of consequences.
Context window
A fixed amount of text the model can see at once
The jagged frontier
Uneven skill — strong here, weak there, no clean line
Prompting & in-context learning
How you talk to it, and how it learns a task on the spot
Temperature & sampling
Why the same question can get a different answer twice
Choosing a model
Size, cost, latency, and open vs. closed, as one decision
Prompting vs. RAG vs. fine-tuning
Which lever to reach for, and in what order
Read it in three passes. The core idea: an LLM does one thing — predict the next token — and everything else about it follows from that one mechanism. The limits: prediction only works over a fixed window of text, and the skill it produces is uneven in a way that has no clean edge. The decisions: once you understand how to talk to the model and why its answers vary, you can choose the right model for a job, and know which lever — a better prompt, retrieval, or fine-tuning — to reach for first.
The lessons
- What an LLM actually is — tokens, next-token prediction, and the difference between training a model once and running it many times.
- The context window — why a model can only see a fixed amount of text, and why that limit exists at all.
- Capabilities & the jagged frontier — why a model can be brilliant at one task and unreliable at an easier one.
- Prompting & in-context learning — how to talk to a model well, and how it learns a new task without being retrained.
- Temperature, sampling & determinism — the dial behind why the same question can get a different answer twice.
- Choosing a model — size, cost, latency, and open versus closed, as one real decision.
- Prompting vs. RAG vs. fine-tuning — the order to try the levers in, cheapest first.
Each lesson pairs the mechanics with a 🎯 For the product leader briefing — why it matters, the decision it changes, the question to ask your team, and the risk if ignored — plus a diagram. Where a lesson touches deeper mechanics, it links out: to Inference internals for what happens inside a single request, and to Fine-tuning vs. ICL vs. RAG vs. distillation for the engineering-depth version of the final decision.
📌 Close out the module: Recap & real-world examples.