Agent Routing — Patterns

Decide on declared facts, not prose

Problem. When the decision reads free-text agent descriptions, the agent whose description is most broadly phrased wins disproportionately, and nobody realises that editing a sentence in documentation changed production traffic distribution.

Approach. Decide against structured declarations — supported task classes, data scopes, modalities, languages, entitlement tiers — published by each agent and validated on publication. Keep prose for humans and forbid the decision path from reading it.

Trade-off. Structured facts cannot express nuance, so genuinely ambiguous requests have no good answer and must fall through to an explicit default. The declarations also need their own verification, because a declared competence that the agent does not actually have is now a routing bug rather than a documentation error.

An explicit nowhere to send it

Problem. Mechanisms that always return the closest candidate have no way to express that nothing fits. Out-of-scope requests land on whichever agent is nearest, which answers them because answering is what it does.

Approach. Make no-match a first-class outcome with its own handler: a clarifying question, a human queue, or a clear refusal. Set the confidence floor from observed misroutes rather than from a round number, and track the no-match rate as a product metric — a rate of zero means the floor is wrong, not that the catalogue is complete.

Trade-off. Users experience a visible refusal rather than an invisible wrong answer, which is better but not free, and a floor set too high turns servable requests into friction. Somebody has to own the no-match queue or it becomes a dead letter box.

Pin for the session, revisit on declared triggers

Problem. Deciding afresh on every message makes conversation incoherent, because the newly chosen agent lacks the context the previous one accumulated. Never revisiting traps a user with an agent that is wrong for the turn they have now reached.

Approach. Fix the choice when a conversation starts and revisit only on named triggers — an explicit user request, the current agent declining, a detected change in task class, or a declared handover point. Record the trigger with the new choice so switching behaviour can be audited.

Trade-off. Pinning means a mid-conversation topic change is served by a suboptimal agent until a trigger fires, and triggers are a tuning surface of their own. Each revisit also implies moving accumulated context across, which is a separate problem that must already be solved.

Shadow the decision before promoting it

Problem. A change to the decision logic — a new agent, a new threshold, a reworded declaration — alters traffic distribution immediately and globally, and the effect is visible only as a quality shift with no deployment to blame.

Approach. Run the candidate decision alongside the live one without acting on it, record both outcomes, and compare distributions and disagreements before promoting. Promote by percentage of traffic, scoped by tenant, with the ability to revert without a deploy.

Trade-off. Shadow evaluation measures disagreement, not correctness: knowing the new logic would have chosen differently on nine percent of traffic says nothing about whether those nine percent would have been better served. Adjudicating a sample is still manual work.