Technical product management — recap & real-world examples
Real-world examples & war stories
The quarter spent proving the obvious. A team skips discovery because "the CEO already knows customers want this." It builds for three months, and launches to single-digit adoption — a result five prototype tests would have predicted for the cost of a week. 🎯 Takeaway: the delivery track is the most expensive possible place to learn an idea is wrong. Discovery exists to make being wrong cheap.
The spec with no non-goals. A one-pager grows fourteen "small additions" between kickoff and code freeze because nothing was ever declared out of scope. The feature ships a quarter late, doing ten things adequately instead of three things well. 🎯 Takeaway: non-goals are the highest-leverage sentences in a PRD — every scope fight you don't have is a week you get back.
The haggled estimate. A PM talks a "six weeks" down to four in planning. The team ships in six anyway, minus tests, plus a production incident. Every estimate thereafter arrives quietly pre-padded. 🎯 Takeaway: negotiate scope, never the number — the number is data, and haggling it just corrupts your instruments.
Knight Capital's 45 minutes (2012). A trading firm deploys new code to only seven of eight servers. A repurposed feature flag activates dead code on the eighth, and with no rehearsed rollback the firm loses ~$440 million in 45 minutes — roughly two-thirds of its market cap. 🎯 Takeaway: deploy/release discipline, flag hygiene, and a rehearsed rollback aren't process overhead. They're the difference between an incident and an obituary.
The A/B test that always won. A growth team celebrates dozens of "significant" wins. The annual retention number doesn't move. The culprits: peeking, post-hoc hypotheses, and segment fishing. 🎯 Takeaway: the point of experimentation is to be hard to fool — a testing culture that never loses isn't testing.
The silent model upgrade. A provider ships a new model version. The team swaps it in as a config change — no eval diff, no staged ramp. Average quality rises, but a template the biggest customer relies on breaks, and the first detector is an angry email. 🎯 Takeaway: a model change is a migration wearing a config change's clothes. Score it, shadow it, ramp it, and keep the old version one re-pin away.
Module recap
| Lesson | The one idea | The question it makes you ask |
|---|---|---|
| The technical PM role | Own outcomes; influence without authority | Which decisions are mine, framed by me, or not mine? |
| Discovery to delivery | Kill ideas cheaply, build the survivors | What's my evidence on valuable / usable / feasible / viable? |
| Specs, PRDs & RFCs | Write the what; read the how | Could the team define "done" without asking me? |
| Prioritization & roadmaps | Order is strategy; roadmaps are bets | What does this yes displace? |
| Working with engineering | Rituals carry intent; trust carries truth | Am I negotiating scope or corrupting estimates? |
| Metrics & experimentation | Measure or you shipped an opinion | Which event proves this worked — and what guards the downside? |
| Launches, rollouts & migrations | Ship gradually; finish the migration | What's the one action that restores yesterday? |
| TPM for AI | Evals are the spec; the flywheel is the moat | Can we tell, with evidence, whether any change helps? |
The through-line: technical product management is a loop, not a lane. It means discover cheaply, specify clearly, prioritize honestly, build together, measure ruthlessly, release gradually, and feed what production teaches you back into the next turn of the loop. AI products don't change the loop. They raise its stakes, because a probabilistic product without evals, feedback capture, and disciplined releases isn't a product — it's a demo with traffic. The craft in this module is what turns the demo into a business.
Walk-away question: "For my current initiative, can I trace the loop end to end — the evidence it's worth building, the spec that defines done, the metric that will judge it, the rollout that protects it, and the feedback that improves it — and which link would snap first?"
Test yourself
-
What makes an estimate of "3 weeks for a form" credible, and what's the right response to it?
Answer
The system around the form (migrations, backfills, old clients, dependent pipelines). The right response is "walk me through it" — decompose the estimate, then trade scope knowingly. (Tech debt & estimation) -
During a SEV1, what comes first: root cause or rollback — and why?
Answer
Rollback (mitigation). Users are paying for every minute of diagnosis. Understand it fully after service is restored. (Incidents & postmortems) -
Why must the experiment hypothesis be written before launch?
Answer
Because post-hoc, any result can be narrated into a win. Pre-registration — including "if wildly successful, what would that look like?" — sets the bar the result has to clear. (Metrics & experimentation) -
What are the three loops of eval-driven development?
Answer
Inner: build ↔ eval (every change scored). Release: eval → staged rollout → production. Outer: production behaviour reshapes the spec. The PM's leverage is keeping the arrows back into the eval suite alive. (TPM for AI products) -
What turns a postmortem from ritual into learning?
Answer
Blamelessness (so it hears the truth) plus action items with owners and dates, tracked like features — and escalation when a repeat incident finds one unshipped. (Incidents & postmortems)