Execution Loop — Patterns

Shape observations before they re-enter

Problem. Tool results vary from a boolean to a megabyte of HTML. Appending them raw means the next call's cost, and the model's attention, are decided by whichever tool answered rather than by the task.

Approach. Put every tool result through a normalising step before it joins the context: extract the fields that matter, cap the length with a defined truncation rule, and attach a reference to the full payload stored elsewhere so the model can ask for more deliberately. Record both the shaped and raw forms, so a later investigation is not limited to what the model saw.

Trade-off. Shaping is lossy by construction, and the field that was dropped is sometimes the one that mattered. Each tool needs its own shaping rule, which is ongoing work and a new place for bugs that manifest as inexplicable model behaviour.

Errors as typed observations, not prose

Problem. An error appended as a sentence is just more text. The model reads it as narrative, produces an apology, and frequently reissues the same failing call, now with the apology occupying context on every subsequent turn.

Approach. Return failures in the same structured form as successful results, with a machine-readable code, a short description, and an explicit statement of what would make a retry worthwhile. Let the loop itself decide retryability from the code — silently retrying transient faults without involving the model, and surfacing only what the model can act on.

Trade-off. Codes have to be maintained per tool, and a miscategorised error either wastes retries on something permanent or gives up on something transient. Hiding retried failures from the model also hides latency the model might otherwise have accounted for.

The step log is the source, the prompt is derived

Problem. When the context window is the only record, trimming it destroys the audit trail, and a crash loses the run entirely. The prompt doubles as storage, badly.

Approach. Append every iteration — model request, response, tool calls, results, counters — to a durable log keyed by run. Build each prompt by projecting from that log under a policy, so changing the policy changes nothing historical and a run can be reconstructed or resumed from the log alone.

Trade-off. A write on the critical path of every iteration, and a projection policy that is now a tested component rather than an implicit append. Storage grows quickly for long runs and needs a retention plan that does not quietly undo the audit property.

Count everything per iteration

Problem. Run-level totals arrive too late to act on. By the time a run has cost ten dollars, the twenty iterations that caused it are finished and indistinguishable from each other.

Approach. Maintain per-iteration counters — input and output tokens, wall clock, tool calls, cumulative spend — check them against ceilings before each call rather than after, and emit them as structured events. Let the loop refuse the next iteration when a remaining budget cannot fund a useful one.

Trade-off. Pre-call checks require estimating the cost of a call before making it, which is approximate and conservative. Conservative estimation stops runs slightly early, which is usually preferable to discovering the overrun on an invoice.