CI was not the bottleneck

We were asking the wrong question, and we nearly spent a migration finding out. The question on the table was whether to move our CI off its current provider — the pipelines felt slow, the way pipelines always feel slow when delivery is behind. Before anyone priced runners, we ran the measurement a migration decision should start with: where do the hours sit between a change entering the queue and that change reaching an independently accepted result? The answer was uncomfortable in two directions at once. The pipeline was fine. Everything after it was not.

The lesson: a faster job with a slower acceptance path does not satisfy the goal. Measure the dwell before you buy the runner.

Map the dwell before the demo

The instrument is not a dashboard purchase. It is a list of hand-offs with timestamps on both ends. We took the delivery path apart into edges — intake to dispatch (work accepted to work assigned), dispatch to worker (assigned to started), PR opened to first review, merge to verified — and computed a median dwell per edge over a trailing 28-day window, from the committed daily reports:

Read that table the way we had to. A pull request meets its first reviewer in about twelve minutes. Then the merged result waits six days, at the median, for independent verification to close — p90 is eleven. The human merge gate is not the constraint either: the cycle-time report puts ready→merged at a twelve-minute median over 100 merged PRs in seven days. And the arrival queue dwarfs everything else: intake→dispatch at thirty-six days median is a prioritization queue, not an execution one. Each edge is a different cohort closing inside the same window; the medians describe their own edge, not one change's trip end to end.

The same week, we sampled the 100 most recent CI runs per repository and recomputed what a run costs from the public run metadata (gh run view <run-id> --json jobs, then earliest created_at to latest completed_at). Across both samples the median run lasted about a minute and none waited to start — the created-to-started median rounds to zero. The main pipeline, job by job:

Five minutes and change, job starts inside half a minute. One sampled window, not a queueing study — but the slowest of the two hundred runs finished in twelve minutes, and nothing on the pipeline side explains six days of dwell. Had we migrated providers to fix the felt slowness, we would have bought a faster version of the stage that already finishes before lunch, and the six days would have been waiting on the other side of it, unchanged.

Where the hours sit

The merge→verified edge has a forensic explanation already on the record. An October 7 pass over 68 verification-session transcripts found the desk spending roughly 86% of its own tool calls on plumbing — tokens, worktrees, claims, landing, lint — re-dispatching 166 distinct briefs about 1,100 times, while about 70% of the queue's verification tables failed as written over intact work: the tables assume Linux, the desk host runs macOS. The pass judged the verification itself, done by sub-agents, mostly correct. The models were not the problem. The plumbing was.

Quality sits beside throughput here, not under it: the rework report counts 14 of 38 delivered briefs carrying at least one rework event in 28 days, 73 verify-fail events in all — and a failed verification is not automatically a code defect.

The gate that held

The same week produced the artifact this method runs on. Two sampled pull-request runs each failed one test shard — build-test-shard (desk-2) — and in both, the aggregate job refused to go green, failing at the step named for the condition it did not find: All build-test shards passed. That is what a non-ceremonial gate looks like from the outside. We know which of ours are non-ceremonial because the gate telemetry says so: the CI-red gate fired 381 times in the window and blocked 66, a mutation-tested catch-rate; the telemetry records three other gate classes as ceremonial-or-untested — fires 0, never mutation-proven. A gate inventory that cannot tell ceremonial from load-bearing is decoration. Ours can, because we measure it.

What we decided instead

No migration — the current-state evidence does not justify one. What the review commissions instead, in priority order: repair the measurement's own completeness first, because the same day's collection summary reported every capture succeeded while its instrument log exited nonzero with role configuration unset — a green summary concealing a broken instrument, so instrument health now has to survive into the summary. Then finish one qualified verification path end to end before any competing redesign. Then, and only then, a bounded comparison: the current provider stays the trigger; one representative workload; frozen source revisions; passing, deliberately failing, and cancelled cases; a switching threshold predeclared with the owner before the first benchmark runs. A candidate that loses required coverage, changes what a failure means, hides unavailable inputs, or weakens credential isolation is rejected regardless of its stopwatch.

The second act, as hypothesis

Once execution is agent-callable — bounded experiments, disposable environments, results an agent can inspect and reproduce — the pipeline stops being a cost line and becomes part of what the fleet can do. That is a product hypothesis, not a result. It earns a number when the first demonstrator — qualifying a release against a disposable adopter environment — produces one.

Run your own map

Nothing here needs our tooling. Pick the four or five hand-offs in your delivery path. Timestamp both ends of each from whatever tracker and CI history you already have. Median per edge, trailing month. Count only edges that closed inside the window — work still sitting in an edge at window close stays out of the median, and two readers using different cohort rules will compute different maps, so state yours. And the discipline that matters most: predeclare what would have to be true for a migration to be worth it before the vendor demos start. If your map shows the hours inside the pipeline, buy the runner. If it shows them in the acceptance path, you have a verification investment to make, and a faster pipeline would only have hidden it better.

Disclaimer

One system's numbers, convenience samples, windows that do not add up to one end-to-end trip. The limits that change how to read the piece:

• The dwell medians belong to different cohorts and must not be summed.
• Dispatch→worker rests on n=3 — a coverage gap, not a finding.
• The cycle-time report excludes pending pull requests, so it says nothing about work currently stuck mid-review.
• The two failed PR runs are unclassified; calling them flakes would be a claim we cannot support.
• None of this proves runner speed never matters — it proves it did not explain our dwell.
• The capabilities discussion is hypothesis, not result.

Three weaknesses named here were open as of late October 2026: a scheduled witness-audit run remains disabled pending an isolated runner; the measurement-completeness repairs are unimplemented; the verification-path fixes are in flight, three briefs implemented with premerge execution and cutovers still ahead.