Parallel Execution — Patterns

Scoped branches with mandatory cancellation

Problem. Branches launched and forgotten outlive the work that spawned them. A run that has already failed keeps paying for eleven outstanding calls, and their late results arrive to write into state that has moved on.

Approach. Launch every branch inside a scope that cannot be exited while any branch is outstanding: the scope either waits for all of them or cancels them, and a failure cancels its siblings as part of unwinding. Carry a cancellation signal into every call, including through client libraries, and assert in tests that cancellation actually ends the branch.

Trade-off. Cancellation requires cooperation at every layer, and a single uncooperative dependency reduces the guarantee to an intention. Fail-fast cancellation also discards sibling work that was nearly complete, which is sometimes a worse outcome than waiting.

Declared read and write sets

Problem. Independence is almost always assumed. Two branches that were obviously unrelated when written turn out to update the same record, and the resulting corruption appears rarely and is nearly impossible to reproduce.

Approach. Make each unit declare the state it will read and the state it will write before launch. Compute overlaps and treat a write-write collision as a defect in whatever produced the units, rejected before execution rather than detected afterwards. Run concurrently only what the declarations prove to be disjoint.

Trade-off. Declarations can be wrong or incomplete, and a branch that writes something undeclared is now violating a contract rather than merely racing. Over-declaration serialises work unnecessarily, so the declarations need to be as narrow as honesty permits.

Reassemble by position, not by arrival

Problem. Collecting results as they complete makes the output depend on timing. The same inputs produce different outputs across runs, which breaks caching, evaluation, and any investigation that relies on reproducing a result.

Approach. Give every branch an index when it is launched, write results into a slot at that index, and assemble strictly by index once the scope closes. Where a branch fails or is cancelled, leave an explicit absence marker in its slot rather than compacting the collection.

Trade-off. The combination step waits for the scope to close rather than streaming results as they arrive, which costs perceived latency for interactive output. Explicit absence markers also push the handling of partial results onto whatever consumes them.

Fan out against the tail, not the mean

Problem. Wide fan-out makes the slowest branch the latency of the whole operation. Average branch latency becomes irrelevant, and the probability of encountering at least one slow branch rises with every branch added.

Approach. Set a deadline for the group rather than per branch, return what has completed when it expires with the incomplete branches explicitly marked, and consider issuing a duplicate request for a branch that passes a latency threshold where the work is idempotent and cheap. Share a concurrency allowance across the whole system rather than per call site, so independent fan-outs cannot collectively overwhelm one downstream dependency.

Trade-off. Duplicate requests raise load to lower latency, which is the wrong trade when the downstream service is the constraint. Group deadlines also mean some branches are routinely discarded after being paid for, and the consumer has to handle incompleteness as a normal outcome rather than an error.