# The fix shipped. Did it help customers?

Field guide — spoken transcript.

The fix shipped. Did it help customers?

Your team has merged a fix. Customer reports have halved. Can you conclude the repair helped customers? This field guide follows an illustrative duplicate-order bug through that decision. Assay coordinates software delivery by agents under human governance. We'll use its delivery and code-history evidence alongside measurements supplied by the application team. You'll see why an encouraging report can still leave the customer problem unresolved, and what evidence to request next.

A customer retries checkout and gets two orders. The team repairs the retry handling. In our fictional example, there were twenty reported duplicates before the repair and ten afterward. The repair record is marked complete. Calling the repair successful would let the team move on. But those two counts do not yet tell us whether checkout became more reliable.

First, establish what complete means. Statusgen is Assay's command-line tool for generating the work-status board and delivery reports. It reads bounded work requests, called briefs, their status changes and linked pull-request evidence. In this example, the repair passed verification on the main branch. That supports the delivery record. It does not establish that customers received the fix.

The application team must connect the repair to a released version and identify which checkout attempts used it. That deployment and product observation comes from the team's own systems. Assay does not automatically install customer telemetry. If the version or exposure record is missing, the outcome is unknown. Request that record before describing the fix as a customer improvement.

Now add the missing denominator. In comparable windows, the old version had twenty reported duplicates across ten thousand checkout attempts. The repaired version had ten across five thousand. Both are one reported duplicate per five hundred attempts. Reports halved, but exposure halved too. This example gives us no improvement in the observed reported-event rate. It does not prove the repair had no effect: reporting can change, and these are small samples.

So we cannot close the investigation on the strength of falling report counts. Preserve the released version, time window, attempts and reports together. Check whether reporting coverage and the checkout populations are comparable. Then inspect a recent report from the repaired version: did it reproduce the original retry failure, or reveal a different problem? The next action is a targeted investigation, not another celebration of delivery completion.

Qualgen is Assay's local command-line analyzer of repository quality history. With Git history and a supplied issue and pull-request snapshot, it can link the fixing change to a defect report and trace changed lines to candidate introducing changes. Use that history to choose what to inspect. Reproduce the reported failure and examine the requirement and implementation. Line ancestry supplies a lead; it does not prove the cause.

Whether an internal check or a customer finds the bug, the component owner triages a linked issue. Application teams supply their support-to-tracker connection. Our internal code workflow retains a reviewed claim explaining how the repair resolves the issue. Keep that issue connected to the delivery record, the candidate defect explanation and the exposed customer outcome. A closed repair record and an open outcome investigation can coexist without contradicting each other.

Change one input. Suppose the repaired version still handled ten thousand comparable attempts, with ten reports. The observed rate would halve. That would be encouraging evidence worth checking against reporting coverage and sample uncertainty. It still would not be causal proof by itself. Define the outcome and acceptance criteria before judging the repair, then use the evidence to decide whether to close, investigate further or revise the fix.

For your next completed repair, ask three connected questions. Was the change verified? Which users received it? What happened for those exposed users? Statusgen supports the delivery question. Qualgen helps investigate code history. Your application supplies customer outcomes. Keep their limits visible and link their records. That is how delivery metrics help you decide where work is still needed. The article covers collection details and the proposed learning cycle.

For more Assay explainers, like and subscribe. You'll find the complete guide at Assay dot finance.