Agent Harness Engineering — Patterns

Boundary-first failure diagnosis

Problem. A ticket assistant produces an unsupported draft, and replacing the model changes the symptom without identifying whether it received the necessary project evidence.

Approach. Retain identifiers for the intake, selected context, proposed call, execution result, and verification verdict. Replay the same task against recorded tool observations. Find the earliest boundary where the required information or behavior disappeared, then vary that component while holding the others fixed. A missing permission grant is an intake or dispatch defect; it is not a reasoning problem for another model to solve.

Trade-off. Reproduction requires storing enough evidence without turning every trace into a permanent copy of sensitive source data. Recorded observations also cannot reproduce all timing and concurrency faults.

Externalize the working record

Problem. A long task resumes in a fresh context and repeats setup, forgets an unfinished step, or reports completion from a partial artifact.

Approach. Keep a task record with acceptance criteria, completed and pending items, artifact revisions, failed checks, and external effect receipts. Derive each new context from that record and relevant source material. On startup, validate workspace health before claiming another item; on exit, record a continuation that another process can interpret. Only a checker authorized to evaluate acceptance may mark the item verified.

Trade-off. The record can become stale independently of the workspace. Its update protocol must represent uncertain outcomes and associate observations with artifact revisions.

Change the harness under a fixed task suite

Problem. A context limit and a tool timeout are changed together, making it impossible to tell which adjustment recovered failed tasks or increased cost.

Approach. Run a representative suite with recorded inputs and controlled dependencies, change one harness policy, and compare successful-task cost, failure classifications, and critical trajectory segments. Exercise recovery as well as the ordinary completion path. Release the accepted configuration as a revision that can be selected or rolled back.

Trade-off. Controlled dependencies improve attribution but conceal changes in the real services. Complement the suite with a small monitored deployment using the same revision.