Model tiering and routing

The routing decision is made on task shape, not on convenience. Messy extraction, ambiguity and judgment get the heavy model; mechanical checks, navigation and bulk generation get a cheap or local one. What makes this survivable is the layer in between: an abstraction that keeps the application from knowing which tier answered, so a model can be swapped, blended or retired without the stack falling apart. Deterministic code is a legitimate routing destination alongside the models — the cheapest correct answer is often not a model at all.

Two guardrails recur. A capability floor: a weak model may be delegated to, but never put in the driver's seat of a loop that holds authority — you never want somebody stupid to be powerful. And budget policy as governance: spend is managed by session-aware soft checkpoints and downgrade gates rather than hard stops, so the run degrades rather than dying. Privacy joins cost and latency as a routing input once local models are in the mix: some queries are routed by where they are allowed to go, not by how hard they are.

The strongest results are not single-model at all. A measured mix beats every constituent, and a junior-generates / senior-decides ladder beats frontier-only at a fraction of the cost. Alongside it sits adaptive test-time control: how much compute a step gets — thinking tokens, retries, which workflow to run — becomes a runtime decision under an explicit budget instead of a constant frozen into a prompt. Because pricing, drift and new models move continuously, the mix is re-optimized against production traffic rather than chosen once.

Also: the Lemonade semantic router, FrugalGPT, Berkeley RouteLLM and Plano as routing prior art; Mercury and its Augment Code deployment; QLoRA, Unsloth, Qwen 4B, GLM-class models and Gemma from the fine-tuning talk.

"Can you swap models without the stack falling apart" is the same requirement Assay met by keeping desk verbs and skills harness-portable rather than binding them to one vendor. On the same evidence base Assay has declined fine-tuning as an investment: skills rot slower, port across harnesses, and review as diffs. Two smaller carries land here too — cache-aware stickiness argues against swapping models mid-session inside a desk loop, and "plug in your own eval, not the public leaderboard" restates the position that external benchmarks do not measure desk outcomes.

Which desk roles may run on a cheap model is decided ad hoc today. The frontier supplies a crisp rule worth adopting: mechanical loops — intake sweeps, board regeneration, documentation fixups — may delegate down; anything holding authority may not. Verify verdicts, review verdicts and any flip of a status row sit above the floor. That is a project-layer value rather than something a neutral role definition should carry, which is precisely why it has drifted.

Briefs already carry effort estimates, but no desk declares a token or session budget, no soft checkpoint asks before spending the next tranche, and Assay’s own reporting ignores cost entirely — so desk economics are currently ungovernable. Cascades are closer than they look: worker → reviewer → human is the junior/associate/partner ladder, but it was designed for correctness, not cost, and has never been measured as one. The standing caution applies directly — continuous cost-optimization erodes exactly the checks that never show up in the quality metric, so cost-optimizing the guardrails away is the failure mode to pre-commit against.

• Whether anyone publishes a measured capability floor — the task classes at which a cheaper tier starts failing a specific authority-bearing role — rather than the qualitative "don't let the weak model drive" rule.
• Whether budget policy shows up as a harness primitive (soft checkpoint, downgrade gate, per-session cap) in a shipped agent framework rather than a workshop demo.
• Whether continuously re-optimized model portfolios report a guardrail regression alongside the cost win — the failure mode nobody has yet measured.