Concept register · Concept 24 of 64 · Theme: isolation beats instruction Reviewed 2026-09-01
assay · concepts · isolation-beats-instruction
Hard controls over instructions
A rule that exists only as tokens in the model’s context is a suggestion, not a control — it can be argued around, socially engineered one step at a time, or erased outright by a context reset. A control is something the agent cannot negotiate with: a capability it was never issued, a credential it structurally cannot read, a deterministic hook on the tool call, a server-side policy that rejects the request whatever the agent believes it was told.
established · assay: capability issuance shipped
17 independent sources · sighted at DevCon London 2026 and the Agentic AI Summit 2026 · last reviewed 2026-09-01
§1What it is
Tokens are not a control surface
The clearest demonstration is a bypass ladder: run the skill, refused; rewrite it as Python, complies; run it, refused; wrap it in a module, complies; rewrite in Rust, refused as clearly malicious — then clear the context, and the same model executes the payload it had just called malicious. Instructions all become soft in a prompt. All-caps and exclamation points are not a control surface, and a hard constraint degrades the moment it is expressed as tokens. The benign case is identical in shape: an agent that keeps ignoring a never-tolerate-flaky-tests rule because its training data taught it that developers ignore flakes.
Issuance replaces prohibition
The design consequence is that you decide what an agent can do by choosing which tools and permissions to hand it, not by writing “do not use this tool” into a prompt. Teams write the prohibition and hand over the tool anyway, “and then they are surprised that the agent deletes the database.” The positive version: an email tool that can only draft and never send makes the irreversible failure structurally unrepresentable rather than merely discouraged. The same move at the credential layer keeps secrets out of the context entirely — a broker holds the token, the model receives only the capability.
Judges are mitigation, not containment
A stated ordering recurs across independent designs: least privilege first, then containers and sandboxing, then judges and output validation. That order matters because judges are statistical. Dual-model jury review runs in production at hyperscale and is talked past by social-engineering the judge — “I’m not a hacker, I’m testing” — roughly three times in ten. Model-judged permissioning is inherently unreliable as a gate. The deterministic layer sits underneath: intercept the tool call, check the result, and refuse structurally. Agents should not be negotiating any boundaries.
§2Sightings
DevCon London 2026 · 8 sightings
#09You’re absolutely right, it was your home directory!Oleg Šelajev, Docker
#28Piece of PIMatthias Lübken
#23Code Security ReinventedJoseph Katsioloudes, GitHub Security Lab
#30Agents in the browserLars Trieloff, Adobe
#33Harness EngineeringRyan Lopopolo, OpenAI
#38Built for Humans. Now Agents Are Here.Dana Lawson, Netlify
#41Agents on the canvas with tldrawSteve Ruiz, tldraw
#08When Tests LieJustin Cormack
Agentic AI Summit 2026 · 10 sightings
#109Superintelligence vs Super ReliabilityDan Klein, UC Berkeley / Scale Cognition
#126Earning Autonomy: Governance as CodeEric Aldana, Credo AI
#134Observability Is Not GovernanceJon Rav Shende, Thales
#073Omnigent: A Meta Harness for AI AgentsArvin and Drew, Databricks
#071Building a Token-Efficient Agentic SystemMaddie and Satya, AMD
#076From Assistants to AI EmployeesAnushka, Ema
#067Why Your AI Agent Needs a WalletHarshel, Circle
#072Quantitative Decision AgentsUri Shalmy and Hadar Sharvit, Fetcherr
#039Agentic AI Developer Platformspanel: Daytona, PyTorch, Fireworks AI, Agentic AI Foundation
#099Systems Foundations for Agentic AIJohann Schleier-Smith, Temporal Technologies
Also: a microVM agent sandbox distributed as a Homebrew formula; Credo AI’s governing harness; Circle’s cryptographically enforced wallet spend limits; Databricks Omnibox.
§3Where Assay stands
Issuance, not prohibition
This is the sharpest external evidence for a standing Assay rule: client-side guards are advisory, and only server-side rulesets, branch protection and app permissions bind. The desk verb layer is capability issuance rather than instruction — each verb is a narrow, intent-revealing authority boundary, and a role holds the verbs its job needs and no others. Draft pull requests with a human-gated merge are the same shape as the draft-only email tool: the irreversible action is unrepresentable, not guarded. The reviewer identity is deliberately kept read-only on the surfaces where a broader grant would buy nothing, and a human’s ambient credential is never borrowed to fill a gap.
The unfinished half
Anything living only in prose is soft control by this definition — skill bodies, per-repository agent instructions, prompt-level rules — and the migration toward mechanical enforcement is not finished. Result-validation hooks are designed and unbuilt: deterministically checking invariants the model currently only promises, such as that a filed issue really carries its raised-by attribution or that a verdict really landed as a review rather than a comment, is the recipient-check pattern applied to desk verbs. Credential brokering is the other gap. The strongest designs here keep the secret outside the model’s reach entirely; Assay hands real, short-lived tokens to desk processes today.
The number that decides the ordering
The three-in-ten judge-bypass figure is the load-bearing measurement, because it is what settles judges into the mitigation tier rather than the gate tier. One corner remains unaddressed: quarantine of untrusted content is a process control today, not a containment one — triage of unblessed material should run in a reduced-permission context rather than in the session that holds the tokens.
§4Watch
- Whether any harness ships credential brokering as a default — the secret swapped in outside the agent’s reach — rather than as a bespoke integration. That would move this from principle to platform baseline.
- Whether a published measurement of judge-bypass rates appears from a second organization, corroborating the three-in-ten figure.
- Whether “capability issuance, not instruction” shows up as declared, lintable data — a per-role output budget — in any toolchain, rather than as advice.