Tools MCP and External Integration
Function Calling — Stack
Function calling › Stack
Technology landscape as of 2026-10-10. Knowledge cutoff May 2026.
Selection criteria
The mechanism is partly a model property and partly a runtime property, and the two are chosen separately.
- Emission reliability. How often a syntactically valid, schema-conformant invocation is produced on the first attempt, measured on your own declarations rather than a benchmark's.
- Schema coverage. Which schema constructs actually constrain the output. Nested objects, unions, and conditional subschemas are frequently accepted by an API and then ignored by the sampler that is supposed to enforce them.
- Call-mode control. Whether the runtime can force a call, forbid one, pin a specific tool, and express "answer now" — and whether those controls behave consistently across the models you intend to use.
- Parallel emission. Whether multiple calls can be returned in one turn, and whether the interface preserves enough structure to match each result back to its request.
- Streaming semantics. Whether arguments arrive incrementally and whether a partially streamed call can be cancelled before the host commits to executing it.
- Portability of the declaration format. How much work it is to present the same tool to a second model family, since declaration shapes and the enforcement behind them differ.
Options
| Option | Best for | Watch out for |
|---|---|---|
| Native tool-calling APIs from hosted model providers | Nearly all production use; the behaviour is trained in rather than bolted on | Declaration format, strictness guarantees, and parallel-call behaviour differ per provider and change between model releases |
| Constrained decoding libraries (Outlines, Guidance, LM Format Enforcer, llguidance) | Self-hosted models, or guaranteeing conformance rather than hoping for it | Requires logit access, so it rules out most hosted endpoints; grammar compilation adds per-request overhead |
| Grammar-constrained sampling in local inference servers (llama.cpp GBNF, vLLM structured output) | Local and on-premises deployments that need hard output guarantees | Grammar expressiveness is narrower than JSON Schema; unsupported keywords are dropped quietly |
| Generic JSON mode without a schema | Legacy models with no tool support | Guarantees parseable JSON and nothing about its shape; every field check falls to you |
| Agent framework tool abstractions (LangChain, LlamaIndex, Semantic Kernel, Pydantic AI, and similar) | Getting a working loop quickly and swapping providers behind one interface | The framework's defaults for retries, parallelism, and result truncation are rarely what you would choose, and are easy not to notice |
| Prompted pseudo-call conventions parsed with a regular expression | Models with no structured output support at all | Brittle under adversarial or multilingual input; treat as a migration path, not a destination |
Models
Tool-calling competence is not a monotonic function of general capability, and it is the one model property most worth measuring directly before committing.
- Small instruction-tuned models frequently handle a handful of flat-argument tools well and degrade sharply once declarations nest or the candidate set passes a few dozen.
- Reasoning-tuned models are generally better at deciding whether a call is warranted and at recovering from an error result, at the cost of more tokens before the first call.
- Fine-tuned function-calling variants exist for several open-weight families and can beat much larger general models on emission accuracy, while being worse at everything around it.
- Benchmark standings transfer poorly to a specific declaration set. A held-out set of real invocations from your own traffic, scored on both tool choice and argument correctness, is worth more than any published leaderboard position.