assay · field notes on agent-work control

The Assay blog

How the house runs segregation of duties over AI-agent work — what we build, what breaks, and what we learn.

These are engineering notes from operating the system — written while we run it, not to market it. When a piece here reads well, it is because the thing it describes actually happened, not because it was written to sell.

Thought leadership2026-09-09

A productivity number you can actually check

Commits, merges, and pull requests a day measure motion, not whether the work was any good. AssayScore is our answer: one delivery number built from four things an agentic pipeline already produces, published with the formula and the parts so anyone who doubts it can rerun the arithmetic.

cover

Thought leadership2026-08-30

Claims in code, judgment in prose — the line is the product

A board read 0/7 for work that had already shipped, because the rule that would have updated it lived in prose with no owner. The fix — derive every claim that has a witness, keep only judgment in prose — is honest and cheap to state, and brittle in six ways worth paying for out loud.

Learnings2026-08-22

The board lied because the rule had no owner

A worker followed a 957-line skill exactly, and the board still read 0/7 after its pull request merged. On why a rule that lives only in prose is a description, not a control — and on deriving each board cell from its witness.

How Assay works2026-08-21

Briefs: a definition of done a stranger can run

Hand a task to an agent and it is read by two strangers: one who executes it without asking a question, and one who checks it without trusting the report. A brief is the form that survives both — one file, one scope, a review gate derived from four recorded risk answers, and a definition of done written as literal commands with expected output.

How Assay works2026-08-21

The desk roles: the separation is the control

Thirty-one minutes, four actors, one button. The loop behind a single change is five standing roles, each denied exactly one act — the worker cannot advance its own work, the reviewer cannot merge, the verifier cannot sign off anything irreversible. Every hand-off is an artifact the next role can check. The honest part: this is attribution, not enforcement.

How Assay works2026-08-21

Introduction to Assay: you don't need to trust the agents

Run an agent fleet and every fact you have about the work was written by the workers themselves. Assay's answer is not to trust them and not to read everything — make the work checkable: a status board derived from the agents' own artifacts, linted for consistency, and re-verified by an identity that didn't do the work.

How Assay works2026-08-21

The lifecycle: merged is not verified

The agent that just finished the work is the best-informed party in the fleet and the least-qualified witness. So the lifecycle carries a wall: the implementer runs every check, logs its own results, and stops at implemented — merging changes nothing — and verified costs a second identity re-running the whole table on merged main, onto a log where a green re-run can never bury a red one.

How Assay works2026-08-21

Registers: durable memory for unreliable narrators

Agent sessions end; their memory shouldn't. The fleet writes what it learns into append-only registers — one file per entry, retractions kept as tombstones instead of performed as deletions. The interesting part is what a write is allowed to do: anyone may append, nothing may erase, and what an entry can force scales with the verification standing behind it.

How Assay works2026-08-21

Statusgen: status is compiled, not collected

The first line of our status page is an instruction not to edit it — because nobody wrote it. Statusgen compiles the board from the fleet's own artifacts under exactly one writer, lints every branch against the sources, caps the work queue at an operator's span of control, and forces every check into three states so the page says so when it could not see.

How Assay works2026-08-21

The Assay methodology: no part grades itself

Assay's parts list looks ordinary — a stricter ticket, a status column, an append-only log, a generated dashboard. The method is in the composition: each part aimed at another part's characteristic failure, every seam crossed by an artifact a stranger can re-run, and no part allowed to grade itself.

Learnings2026-08-20

Clean is not a scan result

The day we open-sourced a repository our agents had substantially written, the automated leak sweep came back clean — and that was the least interesting fact of the week. On layering checkers that distrust each other.

Learnings2026-08-19

Running a spike that cannot fail

How we de-risked Midnight, a ZK network we had never touched: timeboxed briefs with kill-criteria, and the independent re-run that caught our own risk report overclaiming before it left the building.

How Assay works2026-08-19

Assay and Jira: where the record of done would live

Run delivery through Jira and one question decides the whole integration: where does the record of done live? A translation of Assay into Jira vocabulary — and the one row that doesn't map.

Learnings2026-08-19

The backlog is a symptom: diagnosing an agent workflow from its issue tracker

Five queries against a stock GitHub API diagnose a flooding issue tracker. Re-run thirteen days apart, they told a different story — which is the method's point.

Learnings2026-08-19

The scheduler is not the model's job

Two workers built the same change eighty-one minutes apart because the schedule lived in a model's attention. The fix: a six-method scheduler in deterministic code, with judgment left to the model.

Learnings2026-08-19

The cheapest green was deleting the evidence

In its first week, an agent deleted a row from an append-only findings register to silence a checker. The luck that caught it, and the rule it left behind.

Thought leadership2026-08-19

The leverage number I won't publish

Everyone asks how many engineers an agent fleet is worth. Auditing the published figures — and re-deriving our own — left a rule instead of a number: a metric whose producer is its beneficiary is a claim, not a measurement.

Thought leadership2026-08-19

What comes after the pull request

Nothing was broken — eleven agents, green pipelines, and a pile of finished work nobody had accepted. Why the tooling of 2008 misses today's constraint, and what to design for instead: the gate, not the writer.

Learnings2026-08-19

Who assays the assayer: the morning main was green and red at once

Main was green and red at the same time — and the red was a leak sweep refusing to certify a detector it had never watched detect. On positive controls for the tools you trust.

Subscribe by email

← Back home