The economics of infrastructure
TL;DR
Every feature has a cost of goods sold hiding in the infrastructure: compute, storage, network egress, third-party APIs, and — for AI features — tokens. Most software costs are invisible at demo scale and decisive at real scale, because they grow along different curves. Some are fixed (a base cluster). Some scale per user. Some scale per action. A few scale worse than linearly — cross-region data, fan-out notifications, long-context model calls. The PM's job isn't to read the cloud bill. It's to know each feature's unit economics: what one incremental user or action costs, which line dominates, and which product decisions — retention windows, real-time vs. batch, quality tiers — are secretly cost decisions.
🎯 For the AI PM
Why it matters — AI moved infrastructure cost from rounding error to headline. A model call can cost a million times more than a database read, and cost now scales with usage of your best feature. Margins are a product decision for the first time in most PMs' careers.
What it changes in your decisions — Price and package with the cost curve in view. Set budgets per feature and tenant. Treat "make it cheaper without making it worse" as a roadmap line with real engineering behind it.
Ask yourself — "What does one more daily-active user cost me on this feature — and does our pricing recover it at p95 usage, not just at the average?"
Risk if ignored — The success disaster: adoption exceeds the plan, every new user deepens the loss, and the emergency isn't technical. It's a repricing you do in public.
Where the money goes
Four cost families · one unit-economics question
Most software costs are invisible at demo scale and decisive at real scale.
The four families behave differently:
- Compute scales with work done — requests, jobs, renders. Autoscaling makes it elastic. That makes it efficient, but it also means a traffic spike or a retry storm is now a billing event.
- Storage scales with what you keep, and it grows monotonically unless someone decides otherwise. Retention is a product decision wearing an infra costume: "keep every version forever" and "keep 30 days" are different businesses.
- Network is the sneaky one. Moving data out of a cloud (egress) or between regions costs real money. Features like "export everything" or "sync across regions" carry costs invisible in the design review.
- Third parties — payments, maps, messaging, models — price per action. This is the easiest cost to attribute, and the most likely to dominate an AI feature's bill.
The curves matter more than the totals
Two features with the same bill today can be different businesses tomorrow. Classify each cost by its curve:
- Fixed — the base cluster, the observability stack. Heavy at small scale, and it amortizes away as you grow. Fine.
- Linear per user/action — most compute and API costs. Sustainable if pricing scales the same way. Dangerous under flat-rate pricing with unbounded usage.
- Super-linear — the ones that end up in postmortems. Fan-out (every message notifies N followers), cross-joins in analytics, and long-running AI sessions where context grows with conversation length, so cost per session grows faster than sessions do. Find these at spec time by asking "what multiplies?"
Two structural notes from the AI era. Model prices per token have fallen steeply and repeatedly, so re-run yesterday's "too expensive" verdicts on a schedule. And caching changes an AI feature's economics more than any other single lever — prompt caching can cut input cost by an order of magnitude when the prompt is engineered for reuse.
The napkin every feature deserves
Before a feature enters the roadmap, four lines:
- Cost per action — the expensive path, priced at p95 (heavy users define your bill, not the median).
- Actions per user per month — from analogous features, honestly.
- Revenue per user per month — what the plan actually recovers.
- The dominant line — which cost family is 80% of the total, because that's the only one worth optimizing.
If the napkin says margin is negative at target adoption, that's not a veto. It's a design constraint: add a usage tier, batch the work, cache the common case, route easy cases to the cheap path, or price the feature as the premium it actually is. The one unacceptable outcome is discovering the napkin's answer on the invoice.
Failure modes
- The success disaster — Flat pricing sits on linearly-scaling costs. Growth arrives, and each new customer deepens the loss.
- Average-cost pricing — Margins get computed at median usage while p95 users — your most engaged! — are individually unprofitable.
- The immortal archive — Storage only ever grows because no one owned a retention decision. The bill compounds silently.
- Invisible egress — A sync/export/multi-region feature gets designed without anyone pricing data movement.
- Optimizing the wrong line — Weeks get spent shaving compute while a third-party API is 80% of the bill (the twin of the latency-budget mistake).
Practitioner checklist
- For my top three features: what does one incremental action cost, and which cost family dominates?
- Is any cost super-linear ("what multiplies?") — and is anything bounding it?
- Does pricing recover cost at p95 usage, or only at the average?
- Who owns retention for each data store this feature writes to?
- For AI features: what's the cache-hit assumption in the cost model, and is it measured or hoped?