Concept register · Concept 43 of 64 · Theme: humans move up the stack Reviewed 2026-09-01
assay · concepts · humans-move-up-stack
Humans own intent and judgment
When implementation becomes cheap and repeatedly rewritable, the human contribution concentrates at two ends: saying what good means before the work starts, and judging what came back. Specification is the job, judgment is the scarce step, and the human gate belongs at the act step rather than the think step.
established · assay: built on it
18 independent sources · sighted at DevCon London 2026 and the Agentic AI Summit 2026 · last reviewed 2026-09-01
§1What it is
Specification is the job
The human’s remaining work is surfacing the choices and non-functional requirements that define good work locally, and delivering them in a form the model can learn in context. A clearly specified task is the one thing models reliably solve; under-specification is the named cause of disappointing agentic outcomes. Empowering an agent is like onboarding a capable colleague who does not yet know what good looks like here.
Judgment is the scarce step
When a model returns thirty plausible answers to one research question, each worth months of work, the bottleneck moves to deciding which deserves attention. Taste resists automation hardest — novelty, elegance, longevity are exactly the qualities a good reviewer judges and nobody has automated. Hiring has already shifted away from coding tests toward problem decomposition, prompting skill, and the ability to detect confidently-wrong output.
Gate the act, not the think
Reads are free, writes are gated, somebody clicks approve — with identity attached to the click. Reviewing the agent’s intermediate reasoning is waste; reviewing its plan before it executes is not. The efficiency case here is not only governance: a dedicated plan-review phase converged to the same quality in roughly 35 iterations instead of 175, one of the largest measured token savings in either scan.
§2Sightings
DevCon London 2026 · 4 sightings
#13Skills EverywhereJohn Groetzinger, Cisco
#18How AI-First Dev Teams Build Collective IntelligenceEdouard Maleix
#04When Our PM Started Writing CodeTammuz Dubnov, AutonomyAI
#19From Blind Spots to Merged PRsMay Walter, Hud
Agentic AI Summit 2026 · 16 sightings
#063The Limits of AI Coding AgentsIon Stoica, UC Berkeley / Databricks
#056Fireside chatAli Ghodsi, Databricks & Andy Konwinski, Laude
#076Workshop: From Assistants to AI EmployeesAnushka, Ema
#070Workshop: Infrastructure You Can Talk ToJeff, SUSE
#012Harness EngineeringRyan Lopopolo
#014Panel: Future of Software EngineeringGraveley, Lopopolo, Steinberger, Catasta
#021Panel: Agentic AI Foundational CapabilitiesTworek, Vinyals, Song, Zaremba, Chen, Steinhardt
#022Robot Foundation ModelsSergey Levine, UC Berkeley / Physical Intelligence
#105The Rate Limiter on AI Adoption Is OrganizationalSunita Verma, Ironclad
#027Panel: Agentic AI in Capital MarketsWecker, Two Sigma; Allen, D.E. Shaw; Nazari, Susquehanna; Deng
#018Recursive Self Improvement… of What?Oriol Vinyals, Google DeepMind
#002Omniscient AgentsAlex Graveley, Flying Object
#154Using Agents to Build Production AI SystemsJun Yang, NVIDIA
#159Solving Quantum Sci Problems with SMARTAndrew Schoen et al., presented by Mingyu Kang
#143Looping for Model OptimizationJohn Liu, AWS Trainium
#108The Art & Science of Benchmarking AgentsVincent Sunn Chen, Snorkel
§3Where Assay stands
Shipped — this is the concept the architecture is built on
The brief is the specification artifact; its Verify rows are the evaluation authored before the work; the human-gated merge is the act-step gate; and the desk roles never gate the agent’s intermediate thinking. Ema’s and SUSE’s articulations are crisper than Assay’s own docs, and identity-stamped approvals in infrastructure operations are outside confirmation that this governance shape is converging across domains — the same design arrived at independently. The 175-to-35 result is the first quantified argument for authoring briefs at all: planning before executing is not process overhead, it is a 5x efficiency measure.
Three gaps the evidence names
Intake has no readiness test for specification quality — a check that a brief unable to state its non-functional requirements is not yet dispatchable is designed and unbuilt, and it is exactly what the harness-engineering evidence describes. Waves are static once authored; the verifier-feeds-replanner structure suggests verification outcomes should be able to split or merge pending briefs, which is an extension of the dreaming pass into plan maintenance rather than a new mechanism. And the sycophancy point cuts at the review loop: the reviewing App’s verdict is only worth something if it forms independently of the worker’s self-report — which is the design, but is not measured.
Where judgment comes from
The capital-markets panel raises the open question worth carrying: judgment is traditionally built by making your own mistakes, so where does the next generation’s come from if the machine does the experimenting? Read against Assay’s lifecycle, that argues the human merge gate should be staffed deliberately as the place judgment is acquired, not treated as pure cost to be minimized.
§4Watch
- Whether anyone reproduces the plan-review efficiency result outside NVIDIA’s own stack; a second 5x-class number would make plan review a required stage rather than a good practice.
- Whether a specification-readiness gate appears in any harness — a check that refuses to dispatch an under-specified task, rather than a linter on the resulting code.
- Whether the eval frontier’s move toward deliberately under-specified tasks undercuts the specify-first doctrine, or simply measures a different capability.