Evaluation & Quality Engineering · Evaluation and Quality Engineering

Scope: measurement of application quality and failure modes.

Capabilities: 18

From measurements to project qualification

Every capability across the handbook can supply an assessment criterion for a target application. Use the production readiness contract to classify applicability, derive concrete acceptance checks, and connect findings to docs, source paths, and observed results. The criteria evolve with capability guidance; each project assessment keeps its own KB snapshot and release profile.

Evaluation Gates determines how trustworthy evaluation outcomes affect release policy. Project qualification also accounts for implementation evidence, missing operational proof, and scope-specific exclusions across the other domains.

#	Capability
1	Offline Evaluation
2	Online Evaluation
3	Golden Datasets
4	LLM-as-a-Judge
5	Decision Model Evaluation
6	Deterministic Evaluation
7	Human Evaluation
8	RAG Evaluation
9	Agent Trajectory Evaluation
10	Tool Selection Evaluation
11	Task Completion Evaluation
12	Regression Evaluation
13	Adversarial Testing
14	Safety Evaluation
15	Multi-turn Evaluation
16	Evaluation Dataset Versioning
17	Confidence Calibration
18	Evaluation Gates