Durable agent execution

The established workflow-engine pattern transfers directly. Orchestration holds deterministic control flow only; everything non-deterministic is wrapped as an activity, with model calls explicitly named as non-deterministic. A recorded history then lets a workflow pause at a failure for seconds or for years and resume holding no compute — demonstrated live by killing an API mid-run and watching the workflow complete on restore with no code changes. The lighter variant annotates ordinary code with what to save versus what to recompute, and that variant is already carrying production traffic for several well-known agent products.

Agents are idle almost all of the time — waiting on a human, on CI, on a downstream service. One estimate puts it at “99.999% of the time”, and the container platforms everyone reaches for have no warm pools and no suspend/resume. The answer being built multiplexes actors onto pre-warmed runners from golden snapshots: wake in a couple of hundred milliseconds, suspend within a second of going idle, with a proxy triggering wake-up on inbound traffic. The same lever appears at training scale, where microVM pause-between-turns frees compute during long thinking turns.

This is the negative result that makes durability more than an efficiency story. There are no reliable error codes, no liveness signal, no convergence detection — and an agent’s own account of what it did is not faithful to its actual computation, because the emitted tokens clash with the computation that produced them. Self-report cannot substitute for a check. Supervision therefore has to rest on externally checkable signals, and a durable journal is the most direct source of those. Open-ended, runtime-chosen instruction sets make it worse: they break zero-trust security, which needs interaction patterns known in advance, and they make compensation and rollback intractable.

Control planes sized for a few long jobs break when agents launch hundreds of minutes-to-days jobs, and serving stacks built on chatbot-era assumptions — one model, static execution, independent requests, homogeneous accelerators — are invalidated by agentic traffic at roughly 10 to 100 times the computation per workflow. The responses range from warm GPU pod pools and checkpoint/restore up the stack to rack-scale pooled memory that decouples an out-of-memory event on one side from stalling the whole pipeline.

Also: Temporal; Nexus; Agent Substrate (open source, pre-production, targeting end of fall 2026); gVisor and microVM runtime classes; Chakra (MLCommons); AstraSim; NIXL; LMCache; Nemotron; NVFlow.

Assay’s desk loops are a poor-man’s durable execution. Coordination state lives in durable claim records on the forge; work state lives in repository artifacts — the brief, the branch, the draft pull request, the generated board; and recovery means re-reading the repository from its main branch. That is deliberate, and it was reasoned through first-hand: adopting a durable-execution server as the engine was assessed and declined, because it would move coordination state off the un-forgeable forge substrate that is the whole point, and would buy crash-recovery machinery for a problem an artifact-durable design mostly does not have. The failure-handling vocabulary and several specific mechanics were kept.

The artifact-durable design genuinely lacks a save. A dead session leaves recovery to repository forensics plus a claims sweep; long waits on human review or CI hold no formal state; and there is no per-step journal, so reconstructing what a desk actually did is a manual read of pull-request history. Durable execution would turn “session died, resync from claims” into “workflow resumed”, and would deliver per-step observability as a byproduct rather than as a separate build. The golden-snapshot model points further out — desks as cold-startable actors woken by webhook traffic, which would change the economics of running more desks at all. Checkpoint/restore is the smaller version worth stealing outright, since worker recovery today is a branch plus a brief and nothing else.

Desk liveness is inferred from claims and pull-request activity because there is no honest self-report channel, which is why stuck-claim detection keeps needing bespoke sweepers. It is also convergent evidence for Assay’s core bet: gating on externally checkable evidence rows rather than on an agent’s account of what it did is the right design precisely because the account is unfaithful. One adjacent principle is worth keeping too — the single-environment invariant rhymes with the rule that the same verify rows and the same tooling apply from authoring through verification, with no environment drift between stages.

• Whether the pre-warmed agent substrate ships on its stated timeline, and whether wake latency and idle suspension hold outside its authors’ own numbers.
• Whether any agent runtime proposes a real error-code or liveness contract — the missing primitive behind stuck-claim detection here.
• Whether load-testing the claim substrate against hundreds of concurrent briefs, rather than the current trickle, exposes the same control-plane bottleneck LinkedIn hit.