Workflow Execution and Reliability

Workflow Execution and Reliability

Scheduling and Event Triggers

14 of 69 in this book · 3 min read

What it is

Turning a time condition or external event into an admitted unit of workflow work. A trigger states when work becomes eligible; admission decides whether this occurrence should start, join, defer, or skip an execution. Scheduling and Event Triggers therefore includes occurrence identity, overlap, missed occurrences, ordering, and ownership of the schedule itself.

An outer agent loop uses these wakeups to inspect its backlog. A wakeup is not evidence that there is useful work, and a dispatched agent is not evidence that the work completed.

Admission contract

Record the trigger definition and revision, tenant or project scope, scheduled occurrence or event identity, observed time, admitted execution, and reason for deferral or rejection. A redelivered event should map to the same occurrence. Where an event changes an existing work item, check the source revision so an older event cannot overwrite newer work.

Separate nominal time from actual dispatch time. Choose an explicit timezone for calendar rules, a policy for clock or daylight-saving changes, and an acceptable discovery delay. Intervals and local calendar schedules answer different questions.

Overlap and missed work

Situation Policy to choose explicitly
Previous execution still active Skip, buffer, allow bounded overlap, or replace under a cancellation contract
Scheduler unavailable for several occurrences Skip, coalesce into one current scan, or backfill each eligible occurrence
Same event delivered repeatedly Deduplicate admission and reconcile any existing execution
Events arrive out of source order Compare revisions, fetch current state, or route to reconciliation
Nothing eligible after a wakeup Record no-work and avoid launching an unnecessary agent

Cancellation or lease expiry does not undo a worker's external effects. Replacing an execution requires deciding who can still commit and how uncertain writes are resolved.

Example: incident discovery for an outer loop

An authorized hourly scan checks one project's new incidents since its committed cursor. It inserts stable incident identities into a work table, advances the cursor only under the chosen completeness contract, and dispatches bounded workers for eligible items. A webhook can also admit an incident to that same table. Shared identity rules prevent the scan and webhook from creating two business items for one report.

After an outage, a current-state scan may be sufficient if only unresolved incidents matter. If every historical change must be processed, retain events or a replayable source and backfill with a bounded policy. Repeated full scans should not be mistaken for event replay.

Key decisions

  • Whether time, event delivery, or explicit request is the authoritative admission source.
  • What uniquely identifies an occurrence and how long deduplication records remain valid.
  • How catchup, overlap, and source revisions interact with queue capacity.
  • Who may create, alter, pause, resume, or delete the trigger and which authority runs its work.
  • How discovery lag and no-work outcomes are measured independently of run success.
  • Whether notification depends on a meaningful change or on every scheduled occurrence.

Failure modes

  • Overlapping scans claim the same item because schedule identity is mistaken for item identity.
  • Missed occurrences are replayed without a limit and overwhelm the external service.
  • A cursor advances before discoveries are stored, losing work on a crash.
  • An obsolete event restores an incident to an earlier state.
  • Disabling a trigger is presented as stopping already dispatched executions.
  • A calendar rule silently uses the worker machine's timezone rather than the intended one.
  • An idle backlog still launches a model on each wakeup and consumes its budget.

Related capabilities


Applies: Production readiness · Design questions