Concept register · Concept 15 of 64 · Theme: verification at scale Reviewed 2026-09-01
assay · concepts · verification-at-scale
Verification as the scarce resource
When agents generate and execute cheaply and in parallel, the binding constraint on a team stops being production and becomes checking. Raising generation capacity while validation capacity stays flat raises volume without raising knowledge — an unverified artifact is not an asset.
established · assay: built on this
12 independent sources · sighted at DevCon London 2026 and the Agentic AI Summit 2026 · last reviewed 2026-09-01
§1What it is
Checking is the bottleneck
Human time reallocates from having ideas and writing code to verifying results and controlling the process. The people reporting this are not theorists: a security-research team running models end to end says verification is now where most of the time goes, and a survey of biomedical work found roughly 70% of papers irreproducible by others and 50% by their own authors — AI making science slower by inflating paper volume against flat validation capacity. Generation alone produces no new knowledge.
Verifier’s law
Which tasks fall to agents first is predicted by how cheap and how fast their verification is, not by how hard the task is. Coding won because unit tests supplied a verifiable reward that let reinforcement learning work; anything simulatable is verifiable and therefore solvable with unlimited generated data. Self-driving and credit underwriting lag because the ground-truth signal arrives years later, and one bank now starts its prioritization from verifiability rather than from model capability.
Verification is a budget, not a constant
Because checking is the scarce input, it is also a budget: different claims deserve different amounts of proof, and a well-built loop chooses the cheapest sufficient check rather than the strongest available one. The worked forms are adaptive compute allocation — trading cheap retrieval against simulation runs that take days — and tiered cascades that escalate only when the cheap tier cannot answer. Cost-accuracy positioning follows: the best score is often a budget base model with orchestration supplying the accuracy.
§2Sightings
Agentic AI Summit 2026 · 12 sightings
#136Security research end to endMilad Nasr, Anthropic
#080AI for scientific discoveryMengdi Wang, Princeton
#152Trust as the binding constraintScott Clark, Distributional
#063A year of evolutionary coding agentsIon Stoica, UC Berkeley
#145The human as process controlEno Reyes, Factory AI
#043Agentic AI in finance & legalChandhok, Circle; Shafiq, Wells Fargo
#037Why coding went firstIgor Babuschkin, River AI
#034Simulation as the pivot pillarRichard Socher, Recursive
#038Where RL does and does not biteChi, Socher, Cubuk, mod. Babuschkin
#160Adaptive compute allocationRose Yu, UC San Diego
#146Compute at the edgeRahul Bakshi, Amazon
#159Cost-accuracy ParetoAndrew Schoen et al., presented by Mingyu Kang
§3Where Assay stands
The assumption the whole methodology rests on
Assay gates on verify rows rather than on output volume, and this is the cleanest external argument for that choice yet recorded. The lifecycle makes merged ≠ done precisely because merging is production and verification is the scarce step; the verifier is a separate desk role, staffed by someone who did not implement the work, for the same reason. Verifier’s law is also the theoretical justification for the brief scoping rule: briefs are cut small enough that verification is cheap, and cheap verification is exactly the property that decides what can be automated at all. Shipped.
Verifiability as an intake test
The two preconditions named on the summit panel — a clear yes/no signal, and all relevant variables fitting in context — are usable as an intake triage rule today: a proposed brief that has neither needs redesign before dispatch, not a worker. Assay’s intake currently triages on disposition, not on verifiability. Designed, unbuilt.
Cost tiering is the open gap
Verify rows are flat. A cheap lint, a full build and a hand-run manual evidence row all carry the same weight, and the loop has no way to pick the cheapest sufficient proof; nothing budgets which candidates get expensive verification. Any pass that generates candidates faster than a human can adjudicate them will drown the reviewer without such a gate. Tiering is not designed — this scan is the first written statement of the need. The internal application is uncomfortable and worth stating plainly: counting briefs shipped is the paper-volume metric, and counting verified claims is the knowledge metric.
§4Watch
- A published cost model or tiering scheme for verification — someone naming a policy for which claims get cheap versus expensive proof, with numbers on what tiering saved.
- A counter-case: a team that scaled generation and validation together and can show the throughput number, which would move this from “verification is the bottleneck” to “verification is the thing that got automated.”
- Any sign the field has stopped naming checking as the constraint — the only thing that would move this concept toward fading, and nothing here suggests it.