Self-improvement and RL

With pretraining text abundant, the scarce input to capability is no longer data or algorithms but environments — the tools, resets, verifiers and reward surface an agent acts in. The reward that unlocked the last step of capability was the verified outcome, not the plausible narration, and rewarding it started as a prompt hack and became the defining axis.

Stripped of the mythology, recursive self-improvement is a loop in which each improvement drives the next and the target is mostly the outer system — environment, harness, tools, memory — rather than the weights. The binding constraints are the outer two phases, and regression control is the engineering answer: prove a proposed edit does not degrade already-verified behavior before it lands.

Assay is a systems-oriented improvement loop with a deliberate human ratchet: it improves its own skills, guards and desk roles, but a human gates every merge. The evidence cuts both ways — the gate-free results compound faster, and the coherence threshold (current agents lose the plot on long horizons) is the precise statement of why the gate sits exactly there. It is a better articulation of an existing rule than the methodology managed itself.

Requiring evidence rows rather than trusting claimed completions is the same move that defined the RL axis, made structural. The corollary lands too: the durable asset is the environment — briefs, guards, desk roles — not any particular model’s output, which argues for keeping verify rows exercising real repositories and real CI. The honest open question is which way internalization cuts: procedural layers may thin as models absorb them, which is a concrete argument for the skill linter asserting portability invariants.

The transcript-mining pass remains designed and unbuilt, and this lane adds three requirements to it: regression control (re-run affected verify rows to prove an edit does not degrade verified behavior), smallest durable change as the optimization objective, and compounding versus negative transfer measured over successive rounds rather than judged on one round’s plausibility.

8 of 15 talks shown — the ones that reach the most concepts in this theme. Every sighting, per talk, is on the concept pages above.