Provenance and attestation

Treat the model as a teammate who has to convince you. As with a person, ask for the staging log or the screenshot — and then make it mechanical: computer and browser use, an agent desktop the human can look into, a headless display recording a reproduction video. In practice the artifact that carries the most weight is a unique deploy preview per change, because that is where the human re-enters the loop. One team runs dozens of agent browser sessions per pull request against the deployed preview, checking data integrity, subscription flow and sign-on onboarding, which doubles as regression cover for critical journeys.

Attestation has a harder sibling. For claims about artifacts, the dominant failure mode is not forged content but true content in a false context — half of flagged manipulated-media cases have unmodified pixels — and no detector catches that. The durable answer is a multi-signal reconstruction of the artifact’s history and chain of custody, fusing reverse search, provenance metadata, watermarks and detectors to answer context rather than to issue a real/fake verdict.

Two constraints complete the picture. In high-stakes summarization the dangerous failure is what the summary leaves out, so recall is the safety property — which inverts the usual precision-first instinct for anything that reviews or condenses. And a working demo is a hypothesis, not validation: most automations fail, only evidence from a real controlled trial separates the ones that do not, and roughly two randomized trials of generative-AI tools exist across an entire clinical field.

Also: Backstory, C2PA, SynthID, the Habermas machine, Polis.

Attestation is what Assay was built around, and it is shipped: risk-bearing Verify rows require a per-row date and runner table, the verify role is staffed by someone who did not implement the work, and an agent’s assertion is explicitly not evidence. Nothing in these reports asks Assay to change that; they supply independent argument for it.

Evidence rows are text- and CI-shaped. There is no visual-evidence row type, which is the wrong shape for briefs whose deliverable is a rendered artifact rather than a passing command — a site, a deck, an explainer. The proposal: a visual-evidence row type for interface-tagged briefs, built on headless-browser and render machinery Assay already runs, with any brief whose deliverable is a rendered artifact declaring its preview surface in the Definition of Done. Precedent exists in Assay’s own practice, where render-producing work already gates on the rendered still, so this generalizes an existing habit rather than inventing one. Designed here, unbuilt.

The manipulated-media finding transfers more directly than it first appears. A verification culture can over-index on the analog of pixel-checking — did the tests pass? — while the context attacks are the dangerous ones: a pull request whose code is fine but whose claim is wrong, an evidence row citing a real run of the wrong command. “Context over content” is a usable review heuristic and has no current expression in Assay’s reviewer body. Recall-as-the-safety-property argues for recall-oriented reviewer and verifier prompts — a missed finding is worse than a false positive — which is close to the inverse of how a reviewer instructed to minimize noise behaves. Not stated. The last item is an attitude rather than a mechanism and is the most useful thing here: the explicit prior that most agent automations fail, and only evidence separates the transformative few. Assay already records honest framing as a rule — shipped, designed-not-built, deliberately rejected — and the two-trial number is the external calibration for how thin the field’s real evidence base is.

• A preview or attestation format a reviewer agent can consume rather than only a human — today’s screenshots and preview URLs are human-facing, which caps how much of the review can be fanned out.
• Anyone reporting a recall-versus-precision tuning result for review agents.
• The randomized-trial count moving: two across a whole field is the number that most constrains what anyone, including this register, can honestly claim.