About this collection

About this collection

AI Production Engineering Handbook

1 of 3 in this book · 3 min read

A capability inventory for production AI applications, agent systems, and enterprise platforms. Each capability's README.md states neutral, durable guidance — concept, practices, pitfalls, and production checklist — that does not depend on any particular vendor or tool. Opinionated, dated technology, framework, and model guidance is quarantined in that capability's stack/ subfolder, so a reader can trust the README.md on its own and treat stack/ as a recommendation that ages. Legacy technology and model guidance in stack/ reflects knowledge through May 2026. Source-verified updates beyond that baseline state their review date and scope explicitly; an updated chapter does not imply that the rest of the landscape has been refreshed.

Scope: 16 domains, one folder per capability. Each capability folder holds a README.md (capability guidance), stack/README.md (technology landscape), patterns/README.md (solution shapes), and references/README.md (pointers to specifications and canonical sources). These documents are implementation guides, not assertions that every capability is mandatory.

How to use

  1. Start with high-risk user journeys and actual authorization boundaries.
  2. Open a domain folder and read the capability folders that matter for your product.
  3. Turn the practices into test cases and architecture decisions; consult stack/ only after the capability itself is understood.
  4. Track your current implementation, evidence, owner, and release gate separately.

For an agent system's conceptual map, start with Agent Harness Engineering, Agent Graph Engineering, and Loop Engineering. They connect runtime components, execution structure, and feedback across runs to the specialized guidance already present in the handbook.

Assess a project for production

The handbook also supplies evaluation criteria: each capability is a top-level criterion, and its guidance informs concrete acceptance checks for the project's architecture and deployment. Production readiness defines applicability, evidence levels, unresolved findings, and qualified_for_profile versus not_qualified outcomes. Use the assessment workflow and report template to compare docs, source, and observed results.

Create the criterion matrix with tools.project_assessment. It starts unassessed and snapshots the current capability guidance. Adding a capability adds a criterion; editing existing guidance queues review of its acceptance checks. The diff operation preserves prior reports and identifies affected criteria. These tools organize evaluation; they do not inspect a target project or establish production readiness on their own.

Domains

Recommended implementation sequence

Core: security boundaries, execution reliability, evaluation, logging, and release safety. Use-case dependent: retrieval, long-term memory, MCP, multi-agent orchestration, and custom inference. Scale-up: platform self-service, advanced optimization, and automated governance.

Important principles

  • A correct schema does not prove a correct business decision.
  • Deterministic access control cannot be replaced by prompt instructions or model judges.
  • Side-effectful tool calls must address idempotency and partial failure.
  • Evaluate quality on representative human-reviewed examples and real task outcomes.
  • Prefer the simplest architecture that meets reliability and security requirements.
  • Knowledge-base conversations should proactively expose relevant missing foundations and organize them into concepts, relationships, decisions, and evaluation criteria. Apply the knowledge conversation principle, even when the user does not yet know which missing concept to ask about.

Inventory

See CAPABILITY_INDEX.csv for a flat inventory.

The production readiness checklist and design questions that apply to every capability in the handbook are stated once in _shared/ rather than repeated across all 246 capabilities.