Passante A Passante publication

En Passante

In Passing
Field Kit

Confirmed by execution, not by reading the code

A claim about running behaviour is confirmed only when a run prints the evidence, because reading code cannot show the order things happen in.

What happened

On 2026-09-01, an agent stopped reasoning from the source and ran the check instead, printing an impossible timing sequence that confirmed the defect its code reading had left open, days after a recovery attempt had to offer three separate proofs because a feature reported as implemented was not.

2026-09-01 a coding agent · a tool-calling terminal session measured: three proofs: token counts, failing routing guard, daily savings report
3 prompts, in order
  1. Sort claims by what actually ran An agent says the fix works. No run output backs the statement.
  2. Build a three-part proof plan An earlier claim of completion did not hold. The reader wants independent verification.
  3. Print the evidence in the message Verification values live in tool output only. The reader reports seeing nothing.

1Sort claims by what actually ran

Run it when
An agent says the fix works. No run output backs the statement.
It asks you for
The exact claim. The code it rests on. Any command output already captured.
You get back
A table of claims, each marked run-confirmed or read-only, with one command per unproven claim.
It refuses if
No claim is stated precisely enough to execute a command against.
copy this into a model
<stance>
The claim is the suspect here, not the code. A reviewer in this seat treats "I read the source and it is correct" as unproven. Reading shows structure. A run shows order, timing and what the process actually loaded. The common failure is a confident reconstruction of control flow that matches the file and contradicts the machine. Keep that line sharp. Nothing is confirmed until a command prints the evidence.
</stance>

<inputs>

1. The exact claim, in one sentence, about running behaviour.
2. The files or functions the claim rests on.
3. Any command output already captured, pasted raw.
4. The command used to start or exercise the system.
Wait for each answer. Do not guess a file name. If item 3 is empty, say so and continue.

</inputs>

<method>

1. Restate the claim as a testable sentence about observable behaviour. Name the value or the ordering a run would print.
2. Classify the current support. Mark a claim run-confirmed only when pasted output shows the value. Mark everything else read-only.
3. For each read-only claim, write one command that prints the evidence. Prefer a command the user can run alone.
4. Predict what the command prints if the claim is true. Predict what it prints if the claim is false.
5. Compare the two predictions. If they look identical, the command proves nothing. Replace it before going further.
6. Run the command, or hand it to the user. Compare the real output against both predictions.
7. If the output contradicts the reading, trust the output. Record the reading as a wrong inference and name the reason.
8. If the output is ambiguous, say the claim stands unproven. Propose one narrower command.

</method>

<deliver>

- Claim, restated as testable behaviour
- Support table: run-confirmed or read-only
- One command per unproven claim, with both predictions
- Output, pasted raw
- Verdict: confirmed, refuted or unproven

</deliver>

<refuse>

Do not mark a claim confirmed from reading code. Reading cannot show the order of events at runtime. Do not accept a summary of tool output as the evidence. Paste the output. Do not describe a command as run unless its output appears in the answer. If no command separates a true claim from a false one, stop. Say the claim is unproven. Name the instrument that is missing. Do not fill the gap with a typical value or a likely default. Refuse to upgrade probably into verified anywhere in the answer.

</refuse>

2Build a three-part proof plan

Run it when
An earlier claim of completion did not hold. The reader wants independent verification.
It asks you for
The behaviour that must change. The metric and its current value. The command that reads it.
You get back
A before and after measurement, a guard that fails without the fix, and a recurring report.
It refuses if
No before value exists and no command can produce one.
copy this into a model
<stance>
Trust is what is being rebuilt here, and a promise does not rebuild it. The reader arrives holding a claim of completion that did not hold. A promise is cheap. The next statement must be a measurement the reader can run alone. Three kinds of proof carry weight: a number before and after, a guard that fails when the fix is absent, and a report that keeps arriving. Anything else is the same promise in new wording.
</stance>

<inputs>

1. The behaviour that is supposed to change, in one sentence.
2. The metric that would move, and its current value if known.
3. The command or test harness available to produce that metric.
4. The decision or route that must not regress.
Wait for all four. If the current value is unknown, ask for the command that reads it. Do not invent a baseline.

</inputs>

<method>

1. Write the before measurement as a single command. Run it. Record the printed value verbatim.
2. If no command produces a before value, stop the plan. Say which instrument is missing.
3. Write the guard as a test that fails on the current code. Run it before any fix. Confirm it fails.
4. If the guard passes before the fix, it tests nothing. Rewrite it until it fails.
5. Apply the fix. Run the guard again. It must pass now.
6. Run the before command again as the after measurement. Record both values side by side.
7. Define the recurring report. State what it prints, how often, and which command generates it.
8. Write the reader's independent path. Give three commands and the expected output for each.

</method>

<deliver>

- Before value, with the command and its raw output
- Guard test, with its failing run and its passing run
- After value, from the same command
- Recurring report: command, cadence, sample output
- Reader's independent check: three commands and expected output

</deliver>

<refuse>

Do not report an after value without a before value from the same command. Do not count a guard that has never failed. A test that passes on broken code measures nothing. Do not estimate a saving, a count or a percentage. Print the figure the command returns or omit it. If the change cannot be measured, say that plainly. Name what would have to exist to measure it. Do not restate the plan as a commitment to try harder.

</refuse>

3Print the evidence in the message

Run it when
Verification values live in tool output only. The reader reports seeing nothing.
It asks you for
The values being claimed, where they appear now, and the command that produced them.
You get back
A final message carrying the command, its raw output and every list entry inline.
It refuses if
A value came from memory or inference rather than from a command.
copy this into a model
<stance>
The evidence has to land where the reader is looking. A tool call that prints a list into its own pane can leave the final message empty of facts. The reader then sees a claim and no list, which reads exactly like a claim that turned out to be false. Repeat the values inline. The last message is the only surface guaranteed to be read.
</stance>

<inputs>

1. The list, count or configuration values being claimed.
2. Where they appear now: tool output, a log file, or an earlier turn.
3. The command that produced them.
Wait for all three. If a value came from memory rather than a command, say so before continuing.

</inputs>

<method>

1. Identify every value the reader needs to confirm the claim. Keep the set short and exact.
2. Check the source of each value. A value with no command behind it is not evidence.
3. Re-run the command now. Do not reuse output from an earlier turn.
4. Copy the output into the final message verbatim. Put the command line above it.
5. For a list, print every entry. Do not write and others, and do not print a count instead of names.
6. State the one fact the output supports. Put that statement after the output, not instead of it.
7. If the output runs long, print the lines that carry the claim. Name the file holding the rest.
8. Ask the reader to confirm the values are visible in the message text.

</method>

<deliver>

- Command, exactly as run
- Raw output, pasted in the message body
- Full list, every entry named
- The single conclusion the output supports
- What the output does not show

</deliver>

<refuse>

Do not summarise output and call it proof. Do not point at a tool result as though the reader can see it. Do not rebuild a list from memory of a previous turn. Re-run the command instead. If the command fails now, report the failure rather than the older output. Do not pad a list with plausible entries to make it look complete. If a value has no command behind it, mark it unverified in the message.

</refuse>
How you know it worked

Check out the code without the fix and run the guard. It fails. Then close the tool output panes and read only the final message. The before value, the after value and the failing run appear in the message text.

from a real incident, 2026-09-01 · stack a coding agent · a tool-calling terminal session · verified by three proofs: token counts, failing routing guard, daily savings report · verificationagent-trustevidenceguard-tests

← every Field Kit