Credential and Secret Management

What it is

The custody chain for the material that authenticates an outbound call: where API keys, signing keys, database passwords, and third-party access and refresh tokens are stored, how they reach the process that needs one at the moment it needs it, how narrowly each is scoped, how often each is replaced, and how an individual one is revoked without a coordinated outage. In an agent system it also covers per-end-user tokens held on behalf of people who authorised a connection weeks ago and are not present when it is used.

Why it matters

Secrets in an agent system are exposed to a hazard ordinary services do not have: a process whose next action is chosen by a model reading text from outside. Anything the executing context can read — environment, mounted files, a configuration object — is reachable by a tool that was talked into printing it, and anything that reaches the model's context reaches the transcript store, the trace backend, and potentially the user. The per-user token problem compounds it: a system holding refresh tokens for thousands of people's mailboxes is a more attractive target than any single mailbox, and it holds them long after consent was given.

Key decisions

• Delivery to the process. Injected once at start into the environment, fetched per use from a broker, or never held at all because a sidecar or proxy attaches the credential at the edge. Only the last keeps the secret out of the agent's reachable memory.
• One shared service identity per integration, or a credential per end user. Shared is far simpler and makes per-person attribution, revocation, and least privilege impossible.
• Lifetime. Short-lived credentials derived from a workload identity remove most of the rotation problem and require an issuing authority on the path of every call.
• Scope. Whether the upstream can issue something narrower than full account access, and what compensating control exists when it cannot — which is the common case.
• Where per-user tokens live, under what encryption, and what their presence implies for retention, data residency, and breach notification.
• How rotation happens without a deploy: overlapping validity, a version pointer, or a push to every holder. The third guarantees that one missed holder becomes an incident.
• What revocation of a single user's grant actually does to sessions, caches, and queued work already holding a derived credential.

Failure modes

• A secret is printed by a tool — an error message, a debug dump, a command echo — becomes part of a result, enters the model's context, and is written to a trace store with a long retention period and a broader access list than the secret store ever had.
• One shared key serves every user of an integration, so the upstream's logs show a single caller and no individual access can be attributed or withdrawn.
• A refresh token is stored in a session record alongside ordinary application data, with no encryption distinct from the database's own and no separate access path.
• Rotation is attempted, several services are still holding the previous value from an environment set at boot, and the rollback restores the compromised credential.
• A long-lived credential is issued to the execution environment because the narrower option required work, and model-generated code inside that environment can read it.
• A model-authored script enumerates its own environment as a debugging step and the output is returned as a tool result.
• Secret scanning runs on the default branch only, so the value committed on a feature branch is never flagged and remains in history.
• The upstream offers only account-wide keys, so a tool that needed read access to one folder holds credentials for the entire account, and nothing records that this was a known exception.
• Revocation removes a stored token while an in-flight agent run still holds a derived credential, which continues working until it expires on its own schedule.

Related capabilities

• MCP Authentication — issues and scopes much of what is stored here
• Tool Permission Management — decides whether a call happens; this proves who is making it
• Tool Execution Sandbox — the boundary secrets must stay outside of
• MCP Gateway — a common custodian when upstream credentials are centralised
• Webhook and Event Connectors — signing keys whose rotation breaks verification silently

Applies: Production readiness · Design questions