Tool contracts & reliability

TL;DR

A tool is only as trustworthy as the promise behind it: what it accepts, what it actually does, and whether calling it twice by accident is safe. Writing that promise down as a schema is engineering work, but the shape of the promise — how many tools to expose, how broad each one is, and whether the underlying action can be safely retried — is a product decision, made before any code gets written. Get the shape wrong and no amount of careful engineering underneath it fixes the result: a model that picks the wrong tool, or a retry that quietly repeats a real-world action.

🎯 For the product leader

Why it matters — A tool's contract determines how often the model uses it correctly. A vague or overlapping toolbox produces wrong tool calls no matter how good the model is.

What it changes in your decisions — How many tools a feature exposes, at what granularity, and whether "safe to retry automatically" was a deliberate design choice or an assumption nobody checked.

Ask yourself — "If this tool call times out and we retry it automatically, what's the worst that happens?"

Risk if ignored — A retry after a network blip double-charges a customer or sends a duplicate email — not because the model made a mistake, but because nobody decided whether the underlying action was safe to repeat.

The mental model: every tool call passes four checkpoints

Each checkpoint is a place a product decision hides inside what looks like a purely technical detail. A vague description is a product problem — the model can't tell when to use the tool. A tool that isn't safely repeatable is a product problem — a retry becomes a silent double action. An opaque error is a product problem — it produces a model that gets stuck instead of one that recovers.

The granularity decision: how many tools, how big each one is

A toolbox with dozens of thin, near-identical wrappers around individual API endpoints reliably confuses a model — it picks the plausible-sounding one, not the correct one. The fix isn't a smarter model; it's fewer, better-scoped tools that match how someone would describe the task, not how the API happens to be laid out. book_meeting(person, topic, week) beats making the model chain list_calendars → get_availability → create_event → send_invite across four separate calls it can fumble one of. This is a product call — what capability to expose, and at what altitude — even though implementing the schema itself is engineering. The full craft of tool design, including how to write descriptions a model reads well, is developed in Tools & function calling.

Tool count has a hard cost too. Every definition sits in the context window on every request. Anthropic's docs describe a five-server setup (GitHub, Slack, Sentry, Grafana, Splunk) that can use about 55,000 tokens of definitions before any work starts. The same docs say tool selection accuracy degrades once a model has more than 30 to 50 tools. For large toolboxes, vendors now offer tool search: the model finds and loads only the 3 to 5 tools it needs. Anthropic reports that this cut definition tokens by about 85% and raised MCP evaluation accuracy for Opus 4 from 49% to 74% (vendor figures, not independent tests). Treat "how many tools" as both a quality decision and a cost decision. See Cost optimization for how input tokens add up.

Two more design habits come from Anthropic's tool-writing guidance. First, prefix names by service (github_list_prs, slack_send_message) so tools from different systems do not blur together. Second, return only the fields the model needs, with readable names instead of raw IDs. A concise-versus-detailed response option cut tokens to about a third in Anthropic's own Slack example.

Strict mode: valid shape is not the same as correct

Vendors can now force a tool call to match its schema. Anthropic calls this strict tool use: it constrains the model's output so a field typed as an integer is always an integer, and the tool name is always a real one. That removes a whole class of hallucinated-argument bugs. It does not check meaning. The model can still pick the wrong order ID or the wrong tool. Strict mode reduces retries. It does not replace your own validation, authorization, or tests.

Idempotency: a promise, not an implementation detail

Models retry. Networks fail and retry. Agents re-issue a call that looked like it didn't go through. None of that is a bug — it's normal operation — which means every tool with a real-world side effect needs an answer to one question before launch: is calling this twice safe? set_status(order, shipped) is naturally safe to repeat; increment_balance( +10) is not, and needs to become something like apply_transaction(txn_id, +10), keyed so a repeat is a no-op instead of a second charge. This is worth a product leader's attention specifically because it's invisible until it fails — a tool works perfectly in every demo and only shows the gap the first time a real retry hits it in production. The engineering pattern (idempotency keys, separating reads from writes) is developed in full in Function calling reliability.

Errors are part of the product, not an engineering afterthought

The MCP specification separates two kinds of failure. A protocol error means the request itself was broken, such as an unknown tool. A tool execution error (isError: true) carries a message the model can act on, such as a bad date format or a value out of range. The spec says clients should pass execution errors back to the model so it can correct itself.

A tool that fails with an opaque "error 500" produces a model that either gives up or retries the same broken call. A tool that fails with "date must be YYYY-MM-DD, e.g. 2026-07-01" produces a model that corrects itself and moves on. The difference between those two failure messages is a UX decision as real as any error message a human would see — it just has an unusual reader. Reviewing a feature's tool errors the way you'd review its user-facing error copy catches this before the failure mode shows up as a stuck agent in production.

Failure modes

• Kitchen-sink toolbox — dozens of overlapping, thinly-wrapped tools that the model routinely mispicks between, because no one made the granularity decision on purpose.
• A non-idempotent action nobody flagged — a tool with a real side effect, assumed safe to retry because it was never asked the question, discovered the first time a retry duplicates a real action.
• Errors that don't teach — a tool whose failures give the model nothing to act on, producing repeated identical failed calls instead of a self-correction.
• Tool sprawl across servers — several connected servers expose hundreds of tools, definitions eat the context window, and selection accuracy falls. Nobody owns the total.
• Schema drift from the real system — a tool's contract stops matching what the underlying system actually accepts or returns, and nobody notices until calls start failing unpredictably.

Practitioner checklist

• Was the number and granularity of tools a deliberate product decision, or did it just accumulate one API wrapper at a time?
• For every tool with a real-world side effect: is it safe to call twice, and if not, does it have an idempotency key?
• Do you know how many tokens your tool definitions use per request, and have you tested accuracy as the tool count grows?
• Is strict mode on where the vendor supports it, with validation still in place for meaning (right customer, right order)?
• Do tool error messages give the model something it can act on, not just a failure code?
• Has anyone actually tested what happens when a network retry hits each side-effecting tool?

Related lessons

• What tool calling is — why the model can only ever request a call, never guarantee its own correctness.
• Permissions, blast radius & the trust boundary — the next question once a tool's contract is solid: who's allowed to trigger it, and how far can it reach.
• Function calling reliability — the full engineering depth on contracts, validation, and idempotency.
• Tools & function calling — the tool-design craft behind the granularity decision.