Data & the data model

TL;DR

Underneath every product is a data model: the entities it stores (users, orders, messages) and the relationships between them. That model quietly decides what your product can and can't do. A question the data model can't answer is a feature you can't ship without a migration. Two big distinctions matter for PMs. Transactional stores handle fast, correct writes for the live product; analytical stores handle big reads for reporting and ML. And structured data differs from unstructured data. "Where does this data live, and how is it shaped?" is one of the most productive technical questions you can ask.

🎯 For the AI PM

Why it matters — AI features are made of data: training/fine-tuning sets, the documents you retrieve over, embeddings in a vector store, and the logs you evaluate on. The quality and shape of that data caps the quality of the feature, more than the model choice does.

What it changes in your decisions — Ask what data exists, who's allowed to see it (permissions are part of the model), and whether it's clean and connected enough to power the feature — before you assume a model can do it.

Ask yourself — "Does the data to answer this actually exist, in a shape we can use, and are we allowed to use it?"

Risk if ignored — You commit to an AI feature the data can't support. Or you leak one user's data into another's because the model ignored the permission relationships.

Entities and relationships

A data model is entities (tables) connected by relationships. A tiny commerce example:

Read the crow's-foot notation as "one-to-many": a user places many orders, an order contains many order items, and a product appears in many order items. The relationships are the point. They're what let you answer "what has this user bought?" A question the relationships don't support — say, "which products are viewed together?" — needs new data, not just a new query.

Structured vs. unstructured

• Structured data fits neat rows and columns (a user's email, an order's total) and lives in a relational database you query with SQL. It's precise and easy to aggregate.
• Unstructured data — text, images, audio, documents — doesn't fit a table. It's stored in blob storage or document/vector databases and searched differently. Most AI features live here: you're retrieving over documents, not joining tables.

Knowing which kind your feature needs tells you which storage and which failure modes apply.

Transactional vs. analytical (OLTP vs. OLAP)

The same data often lives in two systems for two jobs:

• Transactional (OLTP) — the live product's database. It handles many small, fast, correct reads and writes, like placing an order or updating a profile. Optimized for now.
• Analytical (OLAP) — a data warehouse where data is copied for big reads: dashboards, metrics, ML training. Optimized for scanning history.

Data flows from transactional to warehouse on a delay of minutes to hours. That lag is why "the number in the dashboard" and "the number in the app" can differ. It's also why an ML feature trained on the warehouse is working from slightly stale reality.

Consistency is a spectrum, not a given

The OLTP-to-warehouse lag above is one instance of a bigger pattern: distributed data doesn't update everywhere at once. A write to one system can take anywhere from milliseconds to minutes to become visible somewhere else that reads it — a replica, a search index, another region. This is eventual consistency, and the tradeoff behind it (the CAP theorem: a distributed system under a network partition must choose between staying available with possibly-stale data, or staying strictly correct by refusing to answer) shows up as ordinary-looking product bugs: a user saves a profile edit and refreshes to see the old version, or a search result lags a database write by a few seconds. Neither is a malfunction — it's the system's actual consistency guarantee, which is a design decision, not an accident. The product question is knowing which parts of your feature need to read their own writes immediately (a payment status) and which can tolerate a short, bounded lag (a follower count) — see System design: core building blocks for the full mechanics of how systems actually implement that tradeoff.

Permissions are part of the model

Who is allowed to see each row is not an afterthought. It's part of the data model — tenant IDs, ACLs, sharing rules. For any feature that surfaces data — search, feeds, and especially AI retrieval — the permission relationships must be enforced at query time. Otherwise you leak data across users. This is the multi-tenant boundary the AI Engineering track covers in depth.

A worked pass: "show sellers their repeat customers"

A marketplace PM asks for a small feature: a "repeat customer" badge on seller dashboards. The data model says no. Orders reference a session for guest checkouts, not a durable customer identity, so the same buyer appears as three unrelated rows. The "small feature" is actually this: introduce a customer-identity entity, decide how to match guests (email? payment fingerprint? consent implications either way), backfill months of orders, and only then count repeats. That's two sprints and one privacy review, because of a modeling decision made two years earlier when guest checkout shipped. The lesson is that a question the data model can't answer is a feature you can't ship without a migration. The time to hear that is at spec time. That's why "where does this data live and how is it shaped?" belongs in every kickoff.

The sequel is the OLTP/OLAP version. The badge ships, and a "top sellers by repeat rate" leaderboard follows. Someone points the leaderboard query at the production database, and checkout latency spikes every hour on the hour. Analytical reads over millions of rows don't belong on the transactional store that's processing live orders. They belong in the warehouse, fed by a pipeline, a few minutes stale and happily so. Same data, two stores, two jobs.

Failure modes

• The data doesn't exist — A feature needs a relationship nobody ever stored, and you discover it mid-build.
• Structured/unstructured mismatch — Documents get forced into tables, or free-text search runs over data that should have been structured.
• Stale-analytics surprises — Warehouse numbers get treated as live, or a model trains on data that lags reality.
• Assumed immediate consistency — A feature promises "your change is saved" without checking whether the read path (a replica, a search index, a cache) can actually see it yet, so users hit a save-then-refresh-and-it's-gone bug that's really a consistency lag, not a data-loss bug.
• Ignoring permissions — Retrieval or aggregation crosses rows a user shouldn't see.

Practitioner checklist

• Can I sketch the entities and relationships my feature reads and writes?
• Does the data to answer my feature's question already exist, in a usable shape?
• Is this structured (relational) or unstructured (documents / vectors) data?
• Am I reading live (transactional) or reporting (analytical) data — and does the freshness match the promise?
• For any read right after a write, do I know whether that path is guaranteed consistent or only eventually so?
• Are permission relationships enforced everywhere this data is surfaced?

Related lessons

• How systems are built
• APIs & contracts
• System design: core building blocks — the CAP theorem and consistency models in full mechanical depth
• Technical sense for AI systems