Passante A Passante publication

En Passante

In Passing
Field Kit

The monitor read the file it was watching

A check that derives its expected values from the system under test, or that asserts only presence rather than flow, cannot fail and therefore proves nothing.

What happened

Notes dated 2026-08-26 record two monitors that reported green for months: one confirmed a model was merely present by reading the same configuration it was supposed to police, and another confirmed an executable existed while no text actually flowed through the router.

2026-08-26 config-driven router · scheduled health check measured: check exit status against an intentionally corrupted config
3 prompts, in order
  1. Prove this check can fail A monitor has been green as long as you can remember and you have never seen it red.
  2. Pin expectations as hard-coded constants The check reads its expected values from the same configuration it is meant to police.
  3. Replace existence checks with canary flow The monitor confirms a binary, process or service is there, and nothing more.

1Prove this check can fail

Run it when
A monitor has been green as long as you can remember and you have never seen it red.
It asks you for
The check's source, what it reads at runtime, and the last time it failed for a real reason.
You get back
A verdict on whether the check is falsifiable, plus the exact corruption that should turn it red.
It refuses if
The check's runtime inputs cannot be enumerated, or no one can describe a failing condition.
copy this into a model
<stance>
Audit monitors that have been green for months while the system behind them was dead. The persuasive wrong answer is familiar. The check loads the live configuration, compares it against itself, and prints a pass. That is a tautology wearing a pass badge. The same mistake has a second shape. A check confirms a binary or a process exists, and calls that health. Treat every check as guilty until someone has watched it fail.
</stance>

<inputs>
Ask for, and wait for, each of these before any analysis:
1. The full source of the check, not a summary of it.
2. Every file, environment variable, endpoint or service the check reads while running.
3. The last time this check failed, and whether that failure was real.
4. What the check is supposed to protect, stated as a sentence about behaviour.
If item 1 or item 2 is missing, stop and say which one. Do not reconstruct the check from its name.
</inputs>

<method>

1. List every value the check compares. For each, say where the expected side comes from and where the actual side comes from.
2. Flag any comparison where both sides trace back to the same file or service. Those comparisons are circular and cannot fail. Name them explicitly.
3. Flag any assertion whose subject is existence: a path exists, a process is listed, a key is present. Mark each as a presence check, not a function check.
4. Write the smallest corruption that should make the check go red. Be concrete: which key, which value, which file.
5. State whether that corruption would be detected by the current code. If the source does not settle it, say so and name the line you cannot resolve.

</method>

<deliver>

- Falsifiable: yes or no, one line.
- Circular comparisons: each one, with both sources named.
- Presence-only assertions: listed verbatim.
- The corruption drill: exact edit, expected exit code behaviour.
- Unresolved: lines whose behaviour the source does not reveal.

</deliver>

<refuse>
Do not declare the check sound because it is well written or widely used. Do not accept a long green history as evidence of health. If every comparison is circular, say the check cannot fail. Stop there. Do not suggest cosmetic improvements. If the inputs are not fully enumerated, refuse to issue a verdict. Name the input that is unaccounted for.
</refuse>

2Pin expectations as hard-coded constants

Run it when
The check reads its expected values from the same configuration it is meant to police.
It asks you for
The current configuration, the intended design it encodes, and who owns legitimate changes to it.
You get back
A rewritten check with expectations as literals, and the drift-handling rule for intentional changes.
It refuses if
The intended design cannot be stated independently of the current file contents.
copy this into a model
<stance>
Break circular monitoring. The expensive failure behind this prompt is a routing table that silently collapsed. A guard sat beside it reporting agreement, because it asked the corrupted table what correct looked like. One objection sounds responsible: hard-coding expectations is brittle and duplicative. Brittleness is the feature here. A check that follows the config can never notice the config moved.
</stance>

<inputs>
Request these and wait:
1. The configuration or routing definition in question.
2. The intended design in words, stated by a human, not read back from the file.
3. The list of values that are genuinely allowed to change without review, if any.
4. Where the check runs and what it does on failure.
If item 2 arrives as a restatement of item 1, stop and ask for the intent. A constant cannot be pinned to the file that is under suspicion.
</inputs>

<method>

1. Separate the configuration into values that encode design decisions and values that are operational noise. Only the first group gets pinned.
2. For each design value, write the expected form as a literal inside the check. Name the constant after the decision it protects, not after the key it reads.
3. Make the check compare the live value against the literal. Fail loudly on any deviation, including an unexpected extra entry or a missing one. Say which constant failed and what was found instead.
4. Add the intentional-change path. When a design value legitimately changes, the constant changes in the same review. Write that as a comment above the constant block.
5. State the corruption drill for this check. Edit one pinned value in a scratch copy of the config. Run the check. Confirm it fails and names the value.
6. If the intended design is ambiguous on some key, leave it unpinned. List it as unprotected rather than guessing a value.

</method>

<deliver>

- Pinned constants: each with the decision it protects.
- Unpinned and why: operational values, plus any ambiguous keys.
- Rewritten check: full source, failure messages included.
- Change protocol: how a legitimate design change updates both sides.
- Corruption drill: exact edit and expected failure text.

</deliver>

<refuse>
Do not read any expected value out of the monitored configuration, even once, even as a default. Do not pin a value you were not told the intent for. If the intended design cannot be stated independently, say the check cannot be made falsifiable yet. Name the decision that needs a human answer. Do not soften a failure into a warning.
</refuse>

3Replace existence checks with canary flow

Run it when
The monitor confirms a binary, process or service is there, and nothing more.
It asks you for
The pipeline's entry point, one safe input, and the observable output that proves it processed.
You get back
A canary probe that sends real input end to end and asserts on the output.
It refuses if
No output can be observed, or no input is safe to send against the live path.
copy this into a model
<stance>
A status board can stay green for months while nothing moves through the pipeline. In one recorded case the check asked only whether the executable existed. Presence is not function. The persuasive wrong answer is a probe that hits a health endpoint and reads success. The endpoint answers before any work happens. Believe a pipeline only when input went in one end and the expected output came out the other.
</stance>

<inputs>
Ask for these and wait for each:
1. The pipeline or router being monitored, and its real entry point.
2. One input that is safe to send repeatedly and distinguishable from live traffic.
3. The output or side effect that appears only if processing completed.
4. Anything the probe must not touch: real queues, user data, billing, outbound sends.
If item 3 is missing, stop. Without an observable output there is nothing to assert on.
</inputs>

<method>

1. Write out the stages the input must cross. For each stage, say what evidence of passage exists.
2. Pick the furthest downstream observable that is still safe to read. That is the assertion point. Anything closer to the entry point is presence, not flow.
3. Build the probe. Inject the canary input and wait on the observable with a bounded timeout. Assert on content, not just arrival. An empty or zero-length result is a failure, not a pass.
4. Make the failure specific. The output should say which stage the canary reached and where it stopped.
5. Run the negative case. Disable or misroute one stage in a non-production copy. Run the probe and confirm it goes red. A probe that stays green through a deliberate break is not a probe.
6. If the canary cannot be distinguished from live traffic, stop and say so. Same if it cannot be cleaned up. Do not pollute real data.

</method>

<deliver>

- Flow map: stages and the evidence at each.
- Assertion point: what is read, and why it is the furthest safe one.
- Probe source: injection, bounded wait, content assertion, cleanup.
- Negative test: the break to introduce and the expected red.
- Not covered: stages the probe does not exercise.

</deliver>

<refuse>
Do not accept process liveness, file existence, or a health endpoint returning success as evidence that data flows. Do not assert on arrival alone when content can be checked. Do not write a probe that cannot clean up after itself against a live path. If no safe canary input exists, state that the pipeline is unverifiable without a staging path and stop.
</refuse>
How you know it worked

Copy the configuration to a scratch path. Edit one pinned value so it deviates from the expectations. Point the check at the copy and run it. A real check exits non-zero and names the deviation. If it still reports green, the check measures nothing, and the result can be reported as negative.

from a real incident, 2026-08-26 · stack config-driven router · scheduled health check · verified by check exit status against an intentionally corrupted config · health-checksmonitoringverificationfalsifiability

← every Field Kit