Verification must scale faster than generation

It is too vague to organize thinking, too ambiguous to communicate intent, and not repeatable — version-controlling a prompt does not get you back to the same result. The durable artifact is the specification: a precise description of wanted behavior, under version control, with the code treated as generated output. One speaker pushed the analogy to its end: this is what a compiler is for, and nobody reads compiler output anymore — the investment moves to spec quality and verification strength.

Tests generated from the implementation only confirm that the code does what the code does — they reinforce the wrongness. Stating the goals is the part that never gets delegated. Two anti-gaming controls followed: hold back test values the agent has never seen, and run verification in a pipeline the agent does not control.

A productized spec-reviewer talk gave the working architecture: separate the planner that extracts requirements from the verifier that checks them; fan out one sub-agent per requirement and collect verdicts, because a sequential verifier degrades by the fifth item; ground the verifier in the base branch rather than the diff — the diff is biased toward the solution someone already chose. Left ungrounded, verification agents hallucinate requirements nobody asked for; anything implemented but unsolicited should be flagged as invented scope.

The control taxonomy that organized the track: guides feed forward, sensors feed back, and both come in inferential (a model judging — GPU, probabilistic) and computational (static analysis, tests — CPU, deterministic) forms. The placement rule: never let a pipeline's green/red state depend on the semantic interpretation of a model. Inferential review belongs near the human; deterministic checks belong in CI. Real oracles beat mocks; flakes are removed, not tolerated, because a flaky red teaches everyone to ignore red.

Assay's central claim — the agent that writes the code never gets to say it works — is this concept, held since before the conference stated it. A brief's definition of done and Verify table are the executable specification; verification runs in CI the agent does not control; the resulting evidence is landed by a separate verifier role, not the implementer. The strongest external talk on the subject read like an independent derivation of the position.

For properties that must hold regardless of any test author's imagination, Assay uses closed-form checks — invariant math beside the on-ledger assertions, not runtime assertions sprinkled through the code. Duplication is redundancy; distinct checks at distinct trust boundaries are depth.

CI in an Assay repository is deterministic — lint, tests, board generation. Model-judged verdicts gate readiness, never the merge itself: a human merges. The conference's rule ("no inferential sensors in CI") is therefore already satisfied — and should be stated in the specification as an invariant rather than left as a happy property of the current design.

The track added a third axis to risk assessment — will you even notice if this is wrong? Evidence rows and closed-form invariants are exactly that: investments made before merge in being able to notice. Assay's brief format is gaining an explicit detectability line so the question is answered per work unit, in writing.

Four imports sharpen the verifier without new infrastructure: one verification sub-agent per claim, with the desk as orchestrator; base-branch grounding for spec-fidelity checks (derive what should exist before looking at what does); invented-scope flagging (verified work no brief asked for is reported, not passed); and held-back spec tests derived from the brief alone, after the worker has finished.

8 of 67 talks shown — the ones that reach the most concepts in this theme. Every sighting, per talk, is on the concept pages above.