Parallel fan-out versus long horizons

Point many agents at independent slices of one repository — lint sweeps, coverage targeting, mechanical refactors, format conversions — with the human acting as reviewer and architect rather than implementer. That converts calendar time into parallel machine time, and the reported multiples are large: years of lint debt in under an hour, a half-week of coverage work in three hours, an eight-year platform rewritten to feature parity in three weeks. The constraints are prosaic. Provider concurrency limits are real (one speaker’s account was banned for a ten-agent run and reinstated half a day later), and so is human working memory.

The organizing rule that emerged is that parallelism should come from a pool of interchangeable agents claiming discrete tasks under an explicit work-in-progress constraint. Fine-grained role-play agents were found to “devolve into enterprise politics”, burning tokens arguing on their own channels; the synthesis that replaced them is coarse-grained categories that genuinely differ in tools and connectivity, times task-scaled pools within each. Kanban is the natural fit precisely because it constrains parallelism. The human-side version of the same cap: run many agent tasks, but only one complex one at a time.

Long-horizon failure has two structural drivers independent of task difficulty. The first is per-step error compounding: 95% per-step accuracy decays below 60% over ten steps, and models scoring 88–97% on isolated API calls score 44–73% once those same calls are chained into dependent twenty-step workflows. The tasks do not get harder — dependency structure alone costs 30 to 50 points. The second is the inability to hold repository-wide or goal-wide invariants while closing local obligations: on a formal-verification benchmark of real repositories, the best agent fully proves 27 of 43 and in some cannot prove a single specification, with the bottleneck observed as local obligations closed while global invariants slip.

Granular steps, a test and an exception handler per step, local recovery on failure — and, ahead of the code, an explicit research artifact and task plan so the expensive mapping phase is done once and replayed cheaply. Goal drift is the field’s name for the failure mode this suppresses: subtask optimization quietly diverging from the real goal. The honest state of the art, from one of the largest deployments in the room: “agents are great, but on long-running tasks they are actually not that great”.

Also: LLM Compiler (UC Berkeley); API Flow Bench; Verena; Vero; Lean; the D3 framework; Paper Compute’s “steros” runtime; Meta’s codemod infrastructure.

The worker desk keeps a standing pool of parallel workers, each in its own worktree behind a draft pull request, refilling a slot the moment its worker finishes rather than waving and stopping — which is the task-scaled pool with a Kanban-style WIP cap, arrived at independently. The claim substrate is durable claim records on the forge, so a claim survives the session that took it, and the one-brief-one-branch-one-pull-request rule is the discrete-task-claiming discipline stated as policy. Assay learned the concurrency constraint the hard way as well: fan-out beyond roughly five concurrent agents on a single credential trips provider secondary rate limits, which is why the pool is capped rather than maximized. The account ban is the same lesson from outside.

The brief is a deliberately short-horizon verifiable unit, and the phase plan is the glue — which is exactly what the 1/n signal-decay argument for long-horizon training predicts you should do when you cannot train the long horizon. What is new is the quantitative backing: the 30-to-50-point chaining drop and the 95%-to-under-60% arithmetic are the first hard numbers Assay can cite for why brief granularity matters, and goal drift is the formal name for what tight brief scope suppresses. The local-versus-global gap is the argument for why project-level checks — skill linting, the generated board, cross-repository guards — must exist alongside per-brief verify rows: a per-brief gate cannot see a repository invariant by construction.

The Discover / Plan / Deliver framing is convergent: Discover is intake and a brief’s context assembly, Plan is the brief with its verify rows, Deliver is the worker desk, and human sign-off is the gated merge. Worth noting the source is assertion-heavy and data-light, so it corroborates the shape and not the numbers. Two things are missing. First, there is no equivalent of the two-phase economics — mapping a path once and replaying it cheaply — which is the obvious answer for repetitive cross-repository sweeps that currently re-derive the same plan per worker. Second, taping and distilling every sweep session afterward is not yet discipline here; sweep sessions leave pull requests and nothing else.

• Whether any group publishes long-horizon numbers that improve with harness work rather than degrading with chain length — the Wells Fargo admission is the honest state of the art.
• Whether the map-once-replay-cheaply pattern generalizes past UI automation to code work.
• Whether anyone measures goal drift directly, rather than inferring it from end-state failure.