RL environments and the harness as training signal

They are how agentic training data gets generated at all, and they must match the environment the system will be deployed into or the model is optimized against the wrong distribution. Because environments teach whatever they reward, their quality has been floated as a licensing standard for training. That makes environment authoring, sharing in a standard format, and mass sandbox execution the critical infrastructure layer — and the field is roughly where model weights were before a standard exchange format existed, with thousands of environments shipped in mutually incompatible formats.

It generates the trajectories used for post-training, which puts it inside the training loop rather than outside it. Randomizing its configuration during training is a data-augmentation axis that makes capability survive harness churn — one frontier model was trained against a config that randomly toggles system prompts, sub-agents, memory and skills, in deliberate contrast to a competitor trained on one specific harness. The flywheel runs better models → better harnesses → richer trajectories → better models.

Test-time compute became a third scaling axis beyond data and parameters once training rewarded the checked final result rather than imitating a demonstrated reasoning style. Imitation caps a model at the reasoning style in its data; verified-outcome reward does not, and lets the reasoning behavior emerge instead of being copied. Software engineering as an RL environment has a tidy shape: task is the issue, work is the pull request, reward is the test suite.

Also: OpenEnv and the Hugging Face environment hub; the SWE-smith pattern; Orchestra/OctoChart; Daytona; Evo Lab; the Kimi K3 tech report; Cursor Composer 2; RLVR; GRPO; DeepSeek-R1.

Assay is not in the training business, and nothing in this concept is a build item. Its value is as a lens on what Assay already is. The workshop’s formula — task is the issue, work is the pull request, reward is the test suite or CI — is the shape of a brief with verify rows running in CI. That is one reading of why well-structured Assay repos would be premium trajectory sources, and it is worth stating plainly, since it is a claim about the asset rather than about the tooling.

Rewarding a verified outcome rather than a plausible narration started as a prompt hack and became the defining axis of model capability. Assay makes the same move structurally, by requiring evidence rows rather than trusting claimed completions — corroboration of an existing position, not a new one. The corollary is that the durable asset is the environment, not the output: briefs, guards and desk roles are the asset, and any particular model’s output is not. It also argues verify rows should exercise the real repository and real CI, because anything else measures the wrong distribution, which is already the rule (lifecycle).

The honest framing is that Assay does not know which way this lands. If internalization works, capabilities encoded procedurally get absorbed into models and the procedural layer thins. But the flywheel still requires well-structured harnesses, and briefs, desk roles and evidence rows are exactly high-quality trajectory structure. Dynamic-harness training means Assay’s conventions do not need model cooperation to survive — and it also means any skill depending on one harness’s quirks will silently rot. That is a concrete argument for the skill linter asserting portability invariants, which is designed in part and not built as an assertion.

• Whether a single environment exchange format actually wins — the open hub is the current centre of gravity, but the large lab formats remain incompatible.
• Whether anyone publishes an environment built from a real methodology repository (briefs plus CI as reward), which would test the “Assay repo as environment” reading directly.
• Whether dynamic-harness training becomes standard practice at the frontier, which would decide how much harness-specific convention is safe to encode.