Tool Discovery

What it is

Finding out, at the moment a task needs it, what is actually callable: enumerating the sources currently reachable, reading what each advertises, and reducing that to the small candidate set a model will be shown. The inputs arrive from whatever is connected now — local processes, remote endpoints, catalogues, directories someone else curates — and they change without warning while a session is in progress.

Why it matters

Below a few dozen callable things, discovery is a list and nobody thinks about it. Above a few hundred, it becomes the dominant factor in whether an agent works, because no model can be shown everything and the step that chooses what to show is now making the decision that determines the outcome. Discovery is also the point where things you do not operate get to describe themselves to your model, so it is simultaneously a retrieval problem and a trust boundary.

Finding candidates versus choosing one

Discovery produces a candidate set; it does not pick the tool. Narrowing a thousand advertised offerings to ten is a retrieval decision made by the host in code, measured by whether the right tool is present in the set. Picking one of those ten and filling in its arguments is the model's decision, measured differently and failing differently. Conflating them hides the more damaging error: when the right tool was never in the candidate set, the model does not report a gap — it uses the nearest thing that was offered, or answers without calling anything at all.

Key decisions

• Whether everything is enumerated once at session start or candidates are retrieved per step. Per-step retrieval scales to large surfaces and makes every step depend on a retrieval system being correct and fast.
• What gets indexed for retrieval: names and descriptions only, or synthesised usage examples, past successful invocations, and the arguments they carried. Descriptions alone retrieve badly because users and tool authors use different vocabulary for the same thing.
• Which sources are admissible. An allowlist of origins is the only control that scales; an allowlist of individual tools has to be maintained every time an upstream adds one.
• How long an enumerated surface is cached and what invalidates it — elapsed time, an explicit change notification, or a failed call revealing the cached copy was wrong.
• What happens to a tool that appears mid-session. Admitting it immediately changes the offered set underneath a model that has already planned; deferring it to the next turn is predictable and occasionally wrong.
• Whether discovery failure is visible. A source that times out during enumeration should produce a recorded degradation, not a quietly shorter list.

Failure modes

• The retrieval step returns a plausible near-match instead of the correct tool, and the agent completes the task incorrectly rather than reporting that it could not.
• A description is written to win retrieval — "use this for any question about anything" — and one upstream's entries crowd out everybody else's across the whole fleet.
• Enumeration happens on the critical path of the first turn, and the user waits on the slowest source before seeing any response at all.
• A cached surface outlives a decommissioned tool, so the model keeps selecting something that returns an unknown-tool error on every attempt.
• Two sources advertise the same name and the discovery layer deduplicates by name, silently discarding one of them.
• Retrieval is run over descriptions written by tool authors, so a tool whose description uses internal product vocabulary is invisible to a user's plain-language request.
• A newly connected source is trusted the moment it appears, and its advertised descriptions reach the model without anyone having approved the source.
• The candidate set is evaluated only on precision, so nobody measures how often the correct tool was missing — the one metric that predicts user-visible failure.

Related capabilities

• Tool Registry — the curated catalogue, one possible source among several
• Function Calling — consumes the candidate set as declarations
• MCP Client — enumerates connected servers on this capability's behalf
• MCP Gateway — can narrow the surface before discovery ever sees it
• Tool Permission Management — filters candidates the caller may not use

Applies: Production readiness · Design questions