Knowledge graphs & LLMs

TL;DR

Knowledge graphs and language models fix each other's defining weaknesses, in both directions. Graph → LLM: a model's knowledge is latent, frozen, and unattributable — it will state your customer count fluently and wrongly. A graph supplies explicit, current, citable facts to ground generation, via GraphRAG (retrieve subgraphs, not just text chunks), text-to-query (the model translates a natural-language question into Cypher/SPARQL and the database answers), and graph-shaped memory for agents. LLM → graph: construction was always the bottleneck, and extraction — reading contracts and tickets and drafting entities, relationships, and mappings — is exactly the labor LLMs made an order of magnitude cheaper. The circle closes into a flywheel: the model helps build the graph; the graph keeps the model honest. The discipline holding it together: LLM-drafted facts enter the graph only through the confidence-and-review pipeline, never straight from generation into "truth." A hallucinated fact, once stored, launders itself into every downstream answer with a straight face.

🎯 For the product leader

Why it matters — This intersection is why knowledge graphs left the semantic-web ghetto and landed on product leaders' roadmaps. "Our assistant answers from our facts, with citations, respecting our permissions" is a product claim vector-only RAG makes weakly and a graph makes strongly. Your enterprise buyers can tell the difference in the security review.

What it changes in your decisions — Grounding stops being a model-quality line item ("reduce hallucinations") and becomes an asset decision: what knowledge do we own, how fresh is it, and which answers must be provably sourced? It also reprices graph construction — the pre-LLM cost estimates in your old business case are stale.

Ask yourself — "When our AI feature answers a customer, can it cite the fact, the fact's source, and the reason this user was allowed to see it?"

Risk if ignored — An assistant that hallucinates policy in your brand's voice. Or the quieter failure: an LLM-built graph whose unreviewed errors surface a year later as confidently cited "facts" in front of your biggest account.

Direction one: the graph grounds the model

Plain RAG retrieves text chunks that look similar to the question. That fails structurally on two question types. Multi-hop questions ("which of our customers are exposed to Supplia's recall?") have no single-chunk answer, because the answer is a path across systems. Global questions ("what are the main themes in this quarter's tickets?") need a summary over everything, not any one retrievable passage. Graph-augmented retrieval answers both by retrieving structure:

Three patterns to know by name:

• GraphRAG — resolve the question's entities, pull their neighborhood subgraph, and hand the model facts alongside text. Microsoft's 2024 GraphRAG work added the global-question trick: pre-summarize graph communities so "themes across the corpus" becomes retrievable. The honest caveat: it adds an indexing stage that costs real money and must be kept fresh. Reach for it when multi-hop and global questions are actually on your roadmap, not because the acronym is warm. For the retrieval-engineering side of this same pattern — chunking, indexing, and where GraphRAG beats flat RAG on a cost/quality basis — see Beyond flat RAG: GraphRAG & structured retrieval.
• Text-to-query — the model writes Cypher/SPARQL; the database executes it. The numbers come from the store, not the model's imagination — the right pattern for anything quantitative ("how many contracts renew in Q3?"). Constrain it to the ontology, validate before executing, and eval it like code.
• Hybrid retrieval — in practice you run vectors and graph: vectors for fuzzy recall, the graph for precision, joined by cross-references (chunks link to the entities they mention). The retrieval-eval discipline applies unchanged — you're just adding a second retriever with different failure modes.

Two properties make this pattern enterprise-grade rather than merely accurate. Provenance flows through — claim → fact → source document — for auditable answers. Permissions apply at retrieval — the graph knows who may see which subgraph, so the assistant's knowledge shrinks to the asker's entitlements (the leakage problem, solved at the right layer). For agentic products, the same graph serves as structured memory. Facts an agent learns get written back as triples with provenance — queryable and reviewable — instead of accumulating in an unauditable text scratchpad.

Direction two: the model builds the graph

Construction was always the cost center. Its most expensive station — reading unstructured text and drafting structured facts — is now LLM work: entity and relationship extraction from contracts and tickets, schema-mapping suggestions, even candidate scoring for entity resolution. Teams that shelved knowledge-graph plans in 2019 because "we'd need an NLP team" should re-run that math. This repricing is a big part of why the field revived.

The discipline that keeps it from backfiring:

• Extraction drafts; the pipeline decides. LLM output enters as candidate facts with confidence and source. It flows through the same thresholds and review queues as any extractor's output, and only then becomes knowledge. The graph's whole value is being the thing you don't have to second-guess — one unreviewed hallucination path breaks that contract.
• The ontology is the leash. Free-range extraction invents creative relationship types faster than any committee can review them. Constrain extraction to the ontology's types; route genuinely new patterns to the ontology owner as proposals.
• Sample-audit forever. Extraction accuracy drifts with document mix and model versions. A standing sample audit (measured, per source type) is the smoke detector — and the same golden-set logic you run on any AI feature.

Choosing your grounding posture

The postures stack — most serious deployments end at "hybrid." The sequencing advice is unglamorous: start with vector RAG, measure where it fails (retrieval evals), and let the observed failure types — multi-hop, quantitative, global — justify each graph investment in order.

Failure modes

• Hallucination laundering — LLM-extracted facts written straight to the graph. Future answers cite them with perfect provenance chains ending in a hallucination.
• GraphRAG as fashion — the expensive indexing pipeline built for a workload that was 95% single-hop document Q&A vectors already handled.
• Ungoverned text-to-query — generated queries executed raw. Wrong answers get delivered with database confidence, and it opens a prompt-injection surface into your store (injection applies to query generation too).
• Grounding on a stale graph — the assistant faithfully cites facts eighteen months old. Provenance makes the wrongness look authoritative.
• Two sources of truth — vector index and graph drift apart until they disagree. Answers depend on which retriever won the race.

Practitioner checklist

• For each AI answer surface: which claims must be cited, and does the chain run claim → fact → source without gaps?
• Are retrieval-time permissions enforced, so the assistant knows only what the asker may know?
• Does every LLM-drafted fact pass through confidence scoring and review before becoming graph truth — no exceptions for demos?
• Is text-to-query validated, sandboxed, and evaluated like code?
• Did observed vector-RAG failures (multi-hop, quantitative, global) justify each step up the grounding ladder — with the eval data to show it?
• Is there one freshness story covering both the graph and any derived indexes/summaries?

Related lessons

• RAG architecture — the baseline this lesson extends.
• Beyond flat RAG: GraphRAG & structured retrieval — the retrieval-engineering depth behind the GraphRAG pattern named above.
• Building the graph — the pipeline LLM extraction plugs into.
• Context & memory — the agent-side view of graph-shaped memory.