Recursive self-improvement in practice

Two things. The environment and harness have become hard to disentangle from the agent, so “self-improvement” mostly means letting the agent modify that environment rather than its weights. And improving the harness, prompts, tools and memory from measurable non-gradient feedback is far cheaper than retraining, which puts systems-layer recursive improvement inside ordinary engineering budgets rather than frontier-lab ones.

The loop decomposes into ideation, implementation, experimentation and evaluation — and only the middle two are automated. Evaluation binds because indirect benchmarks invite overfitting while meta-benchmarks are expensive and out of distribution, which is what puts a third tier above both: grading how an improvement was achieved. Ideation binds on taste. Beneath both sits a self-coherence threshold below which a system loses the plot on long horizons and a human has to step in.

The honest diagnosis of the current state is that improving agents is vibe-based: inspect a failure by hand, ask a coding agent to patch it, ship with no evidence the change worked and no visibility into what it broke elsewhere. The answer is regression control — turn real failure signals into replayable environments, find the smallest durable change, optimize under in-loop regression checks across model, harness and memory layers, and land it as a human-reviewable pull request with the reasons attached.

Also: continual Terminal-Bench; Meta-Harness; GEPA; Darwin-Gödel machines; Sakana-style coding RSI; harness training on self-generated data.

Assay is a systems-oriented improvement loop with a deliberate human ratchet. By the taxonomy above it is partial at most: it improves its own skills, guards and desk roles, but a human gates every merge. That gate is the whole design position, and the evidence cuts both ways on it — the gate-free results are the strongest argument that removing it compounds faster, and the best specimen of why it exists, since benchmarks-as-targets is not the same thing as judgment under consequence (desk roles).

The coherence threshold is a precise statement of why the human gate sits on merges: current agents lose the plot on long horizons, and the merge gate is where a human re-establishes coherence. That is a better articulation of an existing rule than Assay has managed itself. Skill-bundle supply-chain controls are the real control surface — an agent that can rewrite its own methods makes parity checks and the review gate on skill edits the thing standing between a fleet and self-modification drift; shipped, and this evidence raises its importance. And “better does not always mean better” is a calibration warning for any metric approaching its ceiling: a flattening number may mean the measure has saturated rather than that progress stopped.

The transcript-mining pass remains designed and unbuilt, and the industrialized external analogue offers three specific steals. Regression control: the pass must re-run affected verify rows to prove a proposed edit does not degrade already-verified behavior — without that it is exactly the vibe-based patching diagnosed above. Smallest durable change as the optimization objective, rather than the largest defensible rewrite. Compounding versus negative transfer as the acceptance test for any self-improvement loop, measured over successive rounds rather than judged on a single round’s plausibility. One competitive warning is worth recording: if that class of tooling matures, Assay’s differentiator narrows to governance and spec discipline rather than the improvement loop itself. The verify-row corpus is the measurable feedback signal Assay would need if it ever auto-tuned its own machinery — it exists, and nothing reads it that way today (registers).

• Whether any independent party reproduces the automatic-SOTA claims; the results are self-reported and are the load-bearing evidence for gate-free systems-layer self-improvement.
• Whether “compounds versus plateaus over successive rounds” becomes the standard way self-improvement results are reported — it is currently one team’s framing, and it is the right one.
• The first credible account of an impactful true recursive self-improvement loop; the field’s own consensus is that none exists yet, which is why this stays at corroborated.