MCP Gateway

What it is

A mediating deployment that terminates connections from many consuming applications on one side and maintains connections to many providing servers on the other, presenting a curated view of the combined offerings and applying organisational policy to everything that crosses it. It is the place where an enterprise answers "which of our teams may reach which external systems, with whose credentials, and what did they actually do" once, rather than in every application separately.

Why it matters

Without a mediation point, every application accumulates its own connection list, its own copies of upstream credentials, and its own idea of what is permitted — and the organisation has no inventory of either. The gateway converts that sprawl into one configuration surface and one audit trail. The cost is that it becomes an in-path dependency for every tool call the organisation makes, so its availability, its catalogue correctness, and its latency budget are now everybody's problem.

Why it is not just a shared MCP server

A server advertises offerings it implements over a system it owns. A gateway implements nothing: it resolves a call to an upstream that does, and its value is entirely in what it does on the way through — selecting which upstreams a given consumer may see, deciding whether this particular call is permitted, attaching the right credential, recording the event, and shaping the aggregate catalogue so it stays small enough to be usable. Judge a server by the surface it designs; judge a gateway by the policy it enforces and the fidelity with which it passes everything else along.

Key decisions

• Whether every consumer sees one merged catalogue or a per-consumer view assembled from an assignment. Merged is simpler to operate and becomes unusable as upstream count grows.
• Where identity terminates. The gateway can authenticate the consumer and then act under its own credentials upstream, or it can broker a credential bound to the original caller. The first is far easier and destroys end-user attribution at the upstream.
• Collision policy for names arriving from different upstreams, and whether the mapping is stable across restarts so recorded calls remain interpretable later.
• How the advertised catalogue reacts to upstream health: offerings withdrawn while an upstream is down, or kept and failed at call time. Withdrawing changes what the model is shown mid-session; keeping produces calls that cannot succeed.
• Whether the gateway inspects and rewrites payloads — redacting arguments, trimming results, scanning for secrets — or passes them through opaquely. Inspection is where most of the value is and all of the latency and breakage.
• Whether non-protocol backends are adapted here, so an existing HTTP API becomes callable without anyone writing a dedicated server for it.
• How configuration changes are released. A catalogue edit silently changes what every connected model can do, which makes it a deployment, not a settings change.

Failure modes

• The gateway is on the path of every tool call in the organisation, and its restart window is an outage for every agent at once.
• The aggregate catalogue grows past the point where a model can select from it, and consumers start requesting narrow subsets — which is the per-consumer view, rebuilt badly.
• An upstream's structured error is flattened into a generic gateway failure, so the model loses the one detail that would have let it correct the call.
• The catalogue is cached for performance, an upstream removes a tool, and calls fail with an unknown-tool error for as long as the cache lives.
• Every upstream credential is held in one process, so a single compromise reaches every connected system at once with no per-system containment.
• Policy is evaluated after dispatch, so a denied call has already had its effect upstream and the denial only suppresses the result.
• Two upstreams expose the same name, the mapping is resolved by load order, and which backend a call reaches changes after a routine redeploy.
• Per-consumer limits are enforced at the gateway while all consumers share one upstream account, so one team's batch exhausts a quota the others were relying on.

Related capabilities

• MCP Server — the upstreams this deployment aggregates
• MCP Client — the consumer that connects here instead of directly
• MCP Authentication — what the gateway terminates and what it forwards
• Tool Rate Limiting — the constraint a mediation point is best placed to apply
• Credential and Secret Management — custody of the upstream credentials held here

Applies: Production readiness · Design questions