Pyx: judging a pull request for automatic merge

A pull request has an approval and a green CI summary. Should software merge it while nobody is watching?

That question includes several others. Is this kind of change allowed through unattended? Does the approval cover the current revision? Did the relevant checks finish? Has a reviewer already found a problem? Is the automation authorized to act at all?

Assay coordinates software delivery by agents under human governance. A bounded request carries requirements and acceptance checks; implementation, review and verification have distinct responsibilities. Pyx brings risk evidence to the decision about how much scrutiny a change requires. The useful rule is simple: define what automation may do before asking a score whether it should proceed.

This also separates Pyx from AssayScore. AssayScore summarizes recorded delivery performance across a reporting window. Pyx concerns the scrutiny and eligibility of individual changes. A productive week cannot authorize a particular PR, and a low merge-risk reading cannot establish customer value.

Start with a change small enough to describe

Consider an illustrative PR that corrects a dated engineering research note. An operator has chosen that narrow document category for a potential automatic lane. A worker agent opens the PR, with a linked work item. The file is inside the allowed area, a reviewer has approved the current revision, and its reported checks have completed acceptably.

Compare it with an equally small change to an agent instruction file. Both may contain only a few lines. The second changes how future work is performed and is outside the permitted category. A favorable score cannot rescue it.

The first PR has only passed the scope test. It still needs the remaining evidence and an authorized action path. This article follows that distinction through the implemented evaluator and the proposed learning system.

Category establishes eligibility

The public command deskautolane implements the automatic-lane evaluation. An operator must explicitly configure the allowed repository areas and policy values. An absent configuration leaves the lane closed.

Admission uses the PR’s hosting-platform author identity, linked work-item trailer, changed paths and surface classification. A surface classification identifies a repository area’s role and sensitivity. The evaluator checks every changed path, including the old path of a renamed file. It excludes stream briefs, risk-classed paths and files that instruct agents or govern the lane itself. A human-authored PR does not qualify for this worker-App lane merely because its content looks routine.

The admission label also has provenance: only the designated reviewer App’s application of that label counts. An author cannot admit their own PR by applying the same text. Scope is recomputed as the PR changes. Admission and configuration implementation.

These restrictions give a reviewer’s judgment a bounded consequence. A mistaken approval cannot, through this lane, grant permission to change its own governing rules. They do not prove that every permitted document edit is harmless; documents can affect builds, published claims and later decisions.

The implemented score counts reasons to stop

The current lane uses a small deterministic score. Each named signal contributes one point when it fires. It does not estimate a probability of a defect.

The ejection line in the initial house policy is zero: a scored reason above that line prevents automatic progression. A clean zero means these signals did not fire on readable inputs. It does not mean “zero risk,” and it does not supply category admission. Score implementation.

Some signals can describe the same history. A request for changes on an older revision can fire both review rework and push after request. Two points are two named conditions, not two independent defects or twice the probability of failure. Reasons belong beside the number.

Model attestation is similarly limited. It records what the dispatcher says it launched and checks who supplied that record. It does not establish that a stronger model produced correct work, or that two role identities have independent credential custody.

Waiting and ejection have different consequences

Return to the research-note PR earlier in its life, while CI is still running. The evaluator cannot establish a green result. Pending checks contribute the unreadable signal; when unreadable evidence is the only problem, the command returns a could-not-check result without permanently ejecting the PR.

A successfully read record can instead establish that a required model attestation is absent or conflicting. That fires the separate missing-model-attestation signal and can cause substantive ejection. Failure to read the attestation source is uncertainty; reading it and finding no valid attestation is a different result.

Now suppose a reviewer requests a substantive correction. That is positive evidence of rework. In an enacted lane, the evaluator removes the admission label, routes the PR into the human queue and records the reasons. Later approval does not automatically readmit the same PR. The ejection is intended to persist for that PR.

This distinction prevents a routine delay in CI from being treated like a confirmed review problem. It also prevents a confirmed problem from disappearing behind a later green headline. If a substantive reason and an unreadable input occur together, the substantive reason still matters. Command-level handling.

Here is the resulting illustrative decision record after the correction:

The merge decision needs fresh evidence

A PR can change after review. The final evaluation therefore checks the current revision, rechecks admission and score, checks for earlier ejection, requires approval by the reviewer App at the current head, examines check results and mergeability, and verifies lane state and human activation. It reads the head again before the action boundary. A moved head refuses the operation.

“Green” has a precise implementation meaning. The current check reducer considers the latest run for each check name and latest status for each context. Completed check conclusions of success, neutral or skipped are accepted; status contexts must report success. Empty, pending or truncated results do not establish green. A newer passing rerun supersedes an older failure of that same check.

That is a rule over the checks reported to the evaluator. It does not prove that every test which ought to exist was triggered. Positive coverage of required checks and validation of the combined target branch remain essential engineering concerns. GitHub’s merge queue addresses the latter by checking changes against the latest target branch and preceding queued changes. A per-PR risk reading cannot detect every incompatibility between separately acceptable PRs. Check reduction, GitHub merge-queue behavior.

Where the implementation stops today

As checked on 10 November 2026, the public code contains category evaluation, the six-signal score, check reduction, provenance checks, ejection handling, activation checks and a merge dry run. The final merge-writing operation is explicitly absent. Even if every condition holds, a non-dry-run request reaches a refusal explaining that nothing was merged. Current command and delivery boundary, final refusal.

The house activation record is also unsigned. Approval of the design and its initial values is separate from permission to operate it. Neither the command’s name nor a successful dry run establishes an unattended merge service.

This is the narrower implementation within the broader Pyx design. There is no single shipped Pyx probability or trained model that currently replaces these predicates. Projects evaluating adoption should inspect the exact version they consume and its enablement state.

Judge the lane after it acts

The intended control continues after a merge. A lane can repeatedly admit a class of changes whose problems its checks miss. That calls for stopping the lane and investigating its scope or verification.

The initial house policy specifies a daily cap of three lane merges per repository and a first-pass-yield floor of 90%, evaluated over a trailing 28-day window once there are at least ten lane merges. These are bounded rollout choices, not universal safety constants or achieved results. With fewer than ten observations, the yield check is explicitly an early state; other conditions and the cap must bound the trial. Missing or unreadable yield input puts the lane on hold.

First-pass yield is intended to be the fraction of lane merges that needed no requested rework, had no linked finding and had no verification-failure transition. The available collector’s calculation is partial: it checks for requests for changes and linked findings, but lacks the verification-failure history required by the full metric definition. A reported ratio must retain that limitation. The intended sticky lane hold, main-failure response and human re-arming behavior require completion and verification as a whole; the existence of a yield evaluator alone does not establish those operational controls.

Keep lane health separate from individual performance. A review finding can be a useful catch. Turning “never needs rework” into a personal target would encourage the wrong behavior.

How outcomes can teach Pyx to direct scrutiny

The proposed learning path* begins with a record of each evaluated revision: the inputs available at that time, category decision, reasons, action and later outcome. Outcome collection then asks whether the main branch became unhealthy, which changes were plausible causes, what repaired it and whether an independent audit supports that attribution.

A collector called repmetrics builds candidate links between CI failure episodes and merged changes. An episode is a transition from a healthy workflow to a failing one. Several intervening changes can make the culprit ambiguous. That ambiguity belongs in the record; the most recent merge is not automatically the cause.

Recent committed captures contain real episode records. The audited-label register and backtest results are still pending in the inspected records. Collected candidates cannot stand in for validated training labels.

The design proceeds from transparent rankings to an expert scorecard and then, if the data supports it, a learned model. Candidate inputs include change size, touched-area history, recent churn, review/check evidence and attested model context. Each additional input has to be available at decision time and earn its complexity.

Evaluate on later changes than those used for training. Compare the richer model with a simple lines-added-only logistic model. Report how many real incidents it finds within a limited review-effort budget, alongside incident prevalence, false alarms and the raw counts. A model that ranks well can still produce unreliable probabilities. The Pyx design therefore treats risk rankings as attention signals until calibration supports any stronger interpretation.

Assay’s repository-quality analyzer, qualgen, already contains a separate learned change-risk library with temporal checks and a fallback when data is insufficient. Its implementation is useful groundwork; it does not establish integration into this lane or satisfaction of Pyx’s audited-label benchmark. Learned-risk library.

Other organizations use risk models under different authority policies. Meta describes using its Diff Risk Score to permit lower-risk changes during periods that previously had broad freezes. That reported use demonstrates a possible application; it does not validate our model, data or permission boundary. Pyx’s current design permits risk to increase scrutiny without widening automatic authority. Meta’s account.

The final test is the intervention: did additional scrutiny find consequential problems at an acceptable cost, and did later outcomes improve? Record reasons for human overrides and investigate noisy signals. Any change to the permitted scope, thresholds or operating authority remains a separately reviewed policy decision.

Put one decision on record

Before adopting automatic merge, take a candidate PR and write down its permitted category, exact revision, approval, check coverage and unresolved risk reasons. Record whether the result is waiting, human review or eligibility to continue. Then record who authorized the action path and what can stop it afterward.

You can do this with ordinary repository records and CI today. Pyx’s value will be in making that decision consistent and inspectable, then improving the allocation of review effort with audited outcomes. A useful score explains a decision. The authority to merge must remain explicit.

Disclaimer

This article describes inspected source and committed records as of 10 November 2026; it does not certify live deployment. The carried PR is illustrative. The automatic merge-writing step and house activation are incomplete. First-pass-yield collection has the stated coverage gap; audited Pyx model results and operational benefit have not been established. The broader learning loop is proposed.