Model Versioning

What it is

Knowing exactly which model weights and which serving configuration produced a given output, and controlling when that changes. In practice it covers pinning to dated snapshots rather than moving aliases, tracking provider deprecation calendars, recording the resolved version on every stored response, and running a deliberate migration when a version is withdrawn.

Why it matters

A model identifier that resolves to different weights next month is an undeclared dependency upgrade applied to production without a release. Outputs shift, prompts that were carefully tuned regress, and a regression reported by a customer cannot be reproduced because the thing that produced it no longer exists. Regulated and audited workflows have a stronger version of the same problem: they must be able to say which system made a decision on a given date, and "the latest model" is not an answer.

Key decisions

• Whether production pins a dated snapshot or follows a moving alias. Pinning gives reproducibility and an explicit upgrade; following gives automatic improvements and undeclared behaviour changes on the provider's schedule.
• Where the resolved version is recorded — ideally on the stored response, the trace span, and the evaluation result, because the three are reconciled during an incident.
• How long a withdrawn version must remain usable, and what the plan is when the provider's deprecation window is shorter than your migration capacity.
• What a version upgrade is gated on: a replayed evaluation set, a shadow comparison on recent traffic, a canary percentage, or a sign-off.
• For self-hosted weights, what identifies a version — a repository revision, a file digest of the weights, the quantisation applied, and the serving engine build, since the same weights behave differently under different runtimes.

Failure modes

• A prompt is tuned against a floating alias, the alias moves, and output format drifts just enough to break a downstream parser without failing any health check.
• An evaluation suite's results are stored without the model version, so the historical scores cannot be compared and the record is worthless as a regression baseline.
• A deprecation email is filed and forgotten; the endpoint starts returning errors on the cutoff date and the migration happens as an incident.
• Development and production pin different snapshots, so a behaviour reproduced locally is not the behaviour being debugged.
• Self-hosted weights are re-quantised during a deployment change and the version label stays the same, making a real behaviour change invisible to everyone reading the label.
• Fine-tuned or adapted models are versioned by training run but not by base model, so a base update silently changes what the adapter sits on top of.

Related capabilities

• Model Capability Registry — holds the deprecation dates a version policy depends on
• Model Benchmarking — supplies the comparison that gates an upgrade
• Multi-model Support — makes running old and new versions side by side possible
• Local and Self-hosted Inference — where weight digests and engine builds become part of the version
• Inference Configuration — the other half of what must be pinned for an output to be reproducible

Applies: Production readiness · Design questions