User research: the craft of finding out
TL;DR
Product sense is built from user contact, so the craft of getting that contact well is a discipline of its own. The method landscape splits on two axes. Qualitative methods — interviews, usability tests — answer why people do things. Quantitative methods — analytics, surveys at scale — answer how many do it. A second axis separates attitudinal data (what people say) from behavioural data (what they actually do). Two rules govern the choice: match the method to the question, and trust behaviour over statements — users are honest about problems and unreliable about solutions and futures. Interviewing is the core skill: favor past behaviour over hypotheticals, open questions over leading ones, and silence over filling it. Five users per round beats fifty per year.
🎯 For the AI PM
Why it matters — AI products add research questions classic funnels miss: do users trust the output, can they repair a wrong answer, do they understand what the system can't do? None of these appear in analytics until they appear as churn.
What it changes in your decisions — You watch users react to imperfect AI output, by seeding the test with a wrong answer on purpose. You treat edits, retries, and abandonments as research data. You never validate an AI feature only on its best-case demo behaviour.
Ask yourself — "Have I watched a real user hit a wrong answer from this feature — and did they recover, retry, or quietly leave?"
Risk if ignored — A feature validated on what users said in a demo glow, dead in three weeks on what users actually did.
The method map
Two axes, one choice · match the method to the question
Qualitative finds the why. Quantitative finds how widely it holds. Research that stays on one side produces anecdotes or scale without explanation.
Qualitative
Interviews · usability tests · field visits
Quantitative
Analytics · surveys · A/B tests
The pairing is the point. Qualitative research finds the why behind the numbers. Quantitative research tells you which whys are common enough to matter. A funnel drop-off (quant) tells you where. Five interviews (qual) tell you what happened there. A survey (quant again) tells you how widely the explanation holds. Research that stays on one side of the map produces either anecdotes with no scale, or scale with no explanation.
Always weight for the say–do gap. Users who said they'd pay, didn't. Users who said the old design was fine had already stopped using it. Statements are data about perceptions. Behaviour is data about the world.
Interviewing — the core skill
Most PM research is conversations, and most conversations are quietly corrupted by the asker. The craft:
- Past behaviour, not hypotheticals. "Tell me about the last time you booked travel for work" beats "would you use a tool that…?" every single time. People are historians of their own behaviour and terrible futurists.
- Open, then narrow. Start wide ("walk me through it"). Drill into specifics ("what did you do next? what were you feeling there?"). Save your own topics for the end. The structure of the 5 Whys works on humans too — gently.
- Don't lead, don't pitch. "Don't you think X would help?" produces polite agreement, not truth. If you're testing your idea, watch them use it. Don't ask them to bless it.
- Silence is a tool. The best material arrives after the pause you resisted filling.
- Small n, high frequency. Five users per round, with rounds every few weeks, compounds into the pattern → heuristic → intuition engine. Fifty interviews once a year produces a report. Five a month produces product sense.
Usability testing is interviewing's behavioural twin: give a task, watch, and narrate nothing. The moment you explain the interface, the test is over — you won't ship yourself alongside the product to explain it to everyone else.
Jobs-to-be-done: turning interviews into a hireable job
Interviews and usability tests produce a pile of individual stories. Jobs-to-be-done (JTBD) is the framework that turns that pile into something you can build against: a customer didn't buy your product, they hired it to make progress on a job, in a specific circumstance — "when I , I want to , so I can ___." The job stays stable even when the product category around it doesn't; people have hired a milkshake, a newspaper subscription, and a podcast for the same commute-boredom job at different points in time. That stability is the payoff: solutions come and go with the competition, but the job survives your redesign, your pivot, and your competitor's launch.
Surfacing a job takes a different interview than a features interview: ask about the moment of first thought ("what were you doing right before you started looking for a solution?"), the struggle that made the status quo intolerable, and what else the customer considered — including doing nothing, or a workaround with no software in it at all. JTBD interviews fail the same way ordinary interviews fail: they drift into feature requests instead of staying with the circumstance. Anchor every answer back to when this happened and what was true in that moment, not what the customer wishes existed.
A job stated well does two things a persona or feature request can't: it explains why customers switch (their circumstance changed, or a competitor now does the job better in that same circumstance), and it defines your real competitive set — whatever else gets hired for that job, sometimes a spreadsheet, sometimes doing nothing, not just the other products in your category.
Researching AI products
Three questions classic research rarely asks, and AI products live or die on:
- The trust curve. First impressions of AI features are unstable: delight at a good answer, betrayal at a bad one. Research the second session, not just the first. Deliberately show participants an imperfect output, and watch the recovery: do they edit, retry, distrust the feature, or distrust the whole product? (Trust design is downstream of what you learn here.)
- The mental model. Users bring wildly different theories of what the system is ("it's Google," "it's a person," "it knows my account"). Mismatched mental models predict misuse and disappointment better than any usability metric — surface them by asking users to predict what the feature will do before they try it.
- Behavioural signals as always-on research. Edits, retries, rephrasings, and abandonments are a continuous usability study running in production — the same signals your flywheel captures. Sample the traces behind them (the trace-reading ritual) and you have qualitative research at quantitative scale.
Failure modes
- The focus-group future — asking users to predict their own behaviour, then building on the predictions.
- Leading the witness — questions shaped so agreement is the polite answer; research as pitch rehearsal.
- Demo-glow validation — testing only the happy path of an AI feature; users meet the unhappy path in production instead.
- Solution-shaped jobs — writing the job statement as a feature in disguise ("I want a faster search bar") instead of the underlying progress ("I want to feel confident I found the best price"). A real job statement survives a total redesign of the solution; a solution-shaped one doesn't.
- Research theatre — a big annual study that's stale before it's summarized, instead of small continuous contact.
- Cohorts-and-percentages drift — consuming only aggregated findings until users stop being people (the mediated-research trap).
Practitioner checklist
- For my current open question: is it a why or a how many — and does my method match?
- Am I asking about past behaviour, or hypothetical futures?
- When did I last watch a user (not a dashboard) use the product — and was it within the month?
- For AI features: have I tested the wrong-answer experience on purpose?
- Are edits/retries/abandonments instrumented and sampled as research, not just as metrics?
- Can I state the job my product is hired for as "when , I want to , so I can ___" — and do I know what else gets hired for it?