Agent Architecture and Orchestration
Agent Routing — Stack
Agent routing › Stack
Technology landscape as of 2026-10-10. Knowledge cutoff May 2026.
Selection criteria
You are buying a classifier that sits in front of everything, so its latency, its explainability, and its behaviour on inputs it has never seen all matter more than its accuracy on the common case.
- Decision latency at the tail. Milliseconds added before the selected agent starts, measured at p99 rather than on average.
- Abstention. Whether the mechanism can return no match. Anything that always returns a nearest candidate will route out-of-scope work into an agent that answers it.
- Data requirement. Whether it needs labelled traffic or exemplars to function, and whether that labelled data exists before launch rather than after.
- Session affinity. Whether a decision can be pinned for the duration of a conversation and deliberately revisited, rather than retaken implicitly on every message.
- Sensitivity to catalogue change. What adding or removing an agent does — a configuration line, a threshold recalibration, or a retraining run.
- Decision records. Whether the chosen identity, the alternatives considered, and the deciding signal are attached to the request for later analysis.
- Entitlement integration. Whether the candidate set can be filtered by what this caller is allowed to reach before the choice is made.
Options
| Option | Best for | Watch out for |
|---|---|---|
| Deterministic rules over structured request fields | Products where the caller already supplies a channel, queue, or task type | Only as good as the structure; unstructured conversational input has nothing to test |
| Keyword and pattern matching at the front door | Narrow, high-confidence shortcuts such as explicit commands | Brittle in the general case and quietly biased toward whatever phrasing the author imagined |
| Intent classifier (fine-tuned encoder) | High volume with a stable taxonomy and labelled historical traffic | Retraining cadence becomes an operational commitment; new agents need new labels before they get traffic |
| Embedding similarity against exemplar sets per agent | Catalogues where example requests are easier to produce than labels | Thresholds drift; two agents with adjacent purposes become indistinguishable in embedding space |
| Model reading the catalogue and choosing | Open-ended assistants with many agents and no labelled data | Latency and cost per request; the choice is sensitive to description wording in ways nobody tests |
| Function-calling over agents exposed as tools | Runtimes where the calling model is already selecting tools | Selection quality is tied to tool-description quality, and the agent set competes with ordinary tools |
| Framework transfer and conditional-edge constructs | Teams already using a graph runtime where the choice is one node | The decision becomes framework-shaped and hard to evaluate outside the running graph |
| Gateway or ingress-level dispatch by header or path | Enforcing one dispatch policy across many services without touching them | Infrastructure-level choices are opaque to product teams and hard to test with real inputs |