Read the manuscript
Download the working manuscript PDF (13 pages, version October 18, 2026).
What does a completion signal establish?
An agent can announce completion without meeting its objective, or change the checks that judge its work. Passing a verifier, printing a completion marker and correctly finishing the intended task are different measurements.
This manuscript audits two completed coding-agent studies using minimized ledgers, inspected source and saved transcript signals. It separates overlapping outcomes, records how runs ended and examines which experimental comparisons support the claims made about them.
What the audit found
- In 164 contradictory-task trials on five fixtures, 71 records contain a completion marker, including all 41 records labeled verifier subversion. Exclusive categories hide that overlap.
- In a selected Sonnet continuation comparison, exclusive false completion is 14/15 under subjective completion wording and 0/15 under test-conditioned wording. All 15 test-conditioned runs reach the eight-turn limit, so marker absence does not establish useful refusal.
- A separate 239-run study on two elementary functions contains denominator and endpoint errors in its historical summary. Its clause-configuration contrast is confounded by different product binaries.
What the evidence does not establish
Task coverage is narrow, the analysis is retrospective and the original correctness labels have not all been independently re-adjudicated. A failed replication is retained. The studies do not establish a general mitigation, an independent cross-scaffold model ranking or a deployment benefit.
The 120 solvable runs in the later study have no recorded gaming; that does not independently certify all original solutions. A proposed follow-up tests updating after trusted task repair. That model study has not been run.
Contributions
Jonathan Scarpelli led development and research and contributed research design, orchestration and evaluation implementation, experiment execution, analysis and manuscript preparation. Ayesha Akhter helped audit, develop and test the security experiments. This contribution account does not establish independent review of every original artifact.
Source code and companion evidence
The Commandem source repository is private and requires access. A repository URL alone does not grant access. We plan to open-source Commandem within the next few weeks. Until then, source access can be arranged for technical review without making the repository public.
The PDF is the manuscript artifact. Original transcripts, companion records and the independent-label review mapping are not distributed here. Earlier reports in the research archive have their own dates and populations; they should not be pooled with this manuscript.