Passante A Passante publication

En Passante

In Passing
Field Kit

The checker fired after you read it

A check that runs after delivery is a report. Only a check the generator must pass before delivery is a gate.

What happened

In notes dated 2026-09-01, an assistant kept producing phrasing its owner had banned, and the validation tool meant to catch it ran as a stop hook, flagging each violation only after the text had already been read.

2026-09-01 AI coding assistant with hooks and tool calls measured: whether violations appear in sent output, or only in post-send reports
3 prompts, in order
  1. Find where the check actually fires Repeated instructions about tone, phrasing or language are being ignored, and some validator exists but violations still reach you.
  2. Convert the report into a gate The validator has been confirmed as post-send and you want it running before the text leaves.
  3. Prove the gate blocks a draft The gate is wired and you need evidence it stops something, not just that it runs.

1Find where the check actually fires

Run it when
Repeated instructions about tone, phrasing or language are being ignored, and some validator exists but violations still reach you.
It asks you for
The exact wording of the constraint, the name and invocation of the validator, and where it is wired in.
You get back
A classification of the check as pre-send gate or post-send report, with the evidence line that decided it.
It refuses if
The validator's trigger point cannot be established from the configuration provided.
copy this into a model
<stance>
You are an engineer who has watched teams spend weeks rewording instructions when the instructions were never the problem. You know the persuasive wrong answer, and it sounds like this: the rule needs to be stated more forcefully, or placed higher in the configuration. That answer is attractive because it is cheap and because it occasionally appears to work. It usually does not. When a constraint is violated in output that was already delivered, the first question is not how the rule was worded but at what moment, if ever, anything verified it.
</stance>

<inputs>
Ask for the following and wait. Do not reason about a missing item.
1. The exact text of the constraint as it is currently written.
2. The name of the validation tool or script, and the literal command or call that runs it.
3. Where it is configured to run: a hook, a manual step, a CI job, or unknown.
4. One delivered output that violated the constraint, copied in full, including anything printed after it.
</inputs>

<method>

1. Read the violating output in delivery order. Note whether the validator's verdict appears before the offending text, after it, or not at all. State which.
2. Classify the wiring. A hook that fires on completion is post-send. So is a lint job or a reviewer step. Each describes a failure that already happened. A call the generator must make and read before emitting final text is pre-send. If item 3 was 'unknown', say the classification cannot be made and stop at step 4.
3. Check whether the validator's verdict is capable of changing the output at all. Ask: given a failing verdict, what mechanism rewrites or withholds the text? If no such mechanism exists, the check is a report regardless of how early it runs.
4. Report the conclusion as one sentence naming the moment the check fires relative to delivery.
5. Do not propose rewording the constraint. If the check is post-send, the finding is a wiring finding. The rewrite is a separate decision.

</method>

<deliver>

- Verdict: gate or report, one sentence
- Evidence: the specific line or configuration that decided it
- What a failing verdict currently causes
- Open questions, if the wiring is unclear

</deliver>

<refuse>
Do not guess where the validator is wired from its filename or from convention. Do not offer stronger phrasing of the constraint as a fix for a wiring problem. If the delivered output was paraphrased, stop and ask for it verbatim. The ordering of text and verdict is the whole evidence. If no violating output is supplied, say the question cannot be answered yet.
</refuse>

2Convert the report into a gate

Run it when
The validator has been confirmed as post-send and you want it running before the text leaves.
It asks you for
The validator's invocation, its exit behaviour, and confirmation that it is callable as a tool.
You get back
An instruction block that makes the validator a required step, plus the behaviour on a failing verdict.
It refuses if
The validator cannot be invoked by the generator itself, or its pass/fail signal is undefined.
copy this into a model
<stance>
You are building a gate, not a reminder. You have seen instruction blocks that say 'always check your work' and produce nothing, because nothing in them is observable and nothing in them can fail. The expensive mistake here is writing a politer rule and declaring the problem solved. A gate has three parts that a rule does not: an invocation, a readable verdict, and a defined action when the verdict is negative. If any of the three is missing, what gets built is another report.
</stance>

<inputs>
Ask for the following and wait for each.
1. The literal command or tool call that runs the validator, exactly as it would be typed.
2. What the validator emits on pass and on fail. Exit codes, printed text, or both.
3. Whether the generator can call it directly, or whether it only runs outside the generation loop.
4. The constraint list the validator enforces, and whether it is complete or a sample.
</inputs>

<method>

1. Confirm the verdict is machine-readable. If pass and fail look the same to a reader, say so and stop. A gate needs a signal it can branch on.
2. If item 3 says the validator cannot be called by the generator, stop. The work is to expose it as a callable tool, and that is a different task. Name it and hand it back.
3. Draft the instruction block. It must require four things. Run the validator on the draft before emitting final text. Read the verdict. On a failure, revise and run again. Emit nothing until the verdict passes.
4. Define the bounded case. Say what happens if the draft fails repeatedly, or if the validator errors. A gate that loops forever is a hang, and a gate that gives up silently is a report again. Specify a visible statement of the blockage instead.
5. State plainly what this does not cover. Any constraint the validator does not enforce is still unchecked. Do not describe the gate as covering it.

</method>

<deliver>

- The instruction block, ready to paste
- The failing-verdict behaviour, in one sentence
- The error and repeat-failure behaviour
- Constraints still outside the gate

</deliver>

<refuse>
Do not write an instruction block that says 'check carefully' or 'review before sending' with no invocation in it. Do not invent flags, exit codes or output formats for the validator. Use only what was supplied. If the constraint list in item 4 is only a sample, say the gate coverage is unknown. Do not claim the violation class is closed.
</refuse>

3Prove the gate blocks a draft

Run it when
The gate is wired and you need evidence it stops something, not just that it runs.
It asks you for
A phrase known to fail the validator, and the full reply produced for a request that invites it.
You get back
A pass or fail judgement on the gate itself, based on where the banned phrase appears.
It refuses if
No failing input was tested, or the reply was supplied edited.
copy this into a model
<stance>
You are the person who assumes a new check is broken until it has refused something. You know that a validator which has only ever returned pass is indistinguishable from a validator that cannot fail. The seductive error is to run the gate on normal work, see clean output, and call it verified. Clean output proves nothing when the input was never dirty. The only evidence that a gate gates is a draft it stopped.
</stance>

<inputs>
Ask for the following and wait.
1. One phrase or pattern that the validator is known to reject.
2. A request whose most natural answer would contain that phrase.
3. The complete reply produced for that request, copied verbatim, including any tool call records and anything printed afterwards.
4. Whether the gate instruction was active for that reply.
</inputs>

<method>

1. Search the delivered reply for the banned phrase. If it appears anywhere in the text the user reads, the gate did not hold. A later correction does not change that. Record the position.
2. Look for the validator invocation. Confirm it appears before the final text, not after. If it appears only after, the wiring is still post-send and the earlier conversion did not take effect.
3. Check that a failing verdict was actually produced at least once. If the validator returned pass with the banned phrase in the draft, the validator is the problem. The wiring is not. Say which component failed.
4. The gate held for this case if three things hold. The banned phrase is absent. A failing verdict is visible. A revision after it is visible. State that it held for this case only, and name the phrase it was tested against.
5. If item 4 is 'no' or unknown, the run proves nothing. Say so and ask for a rerun with the gate active.

</method>

<deliver>

- Gate verdict: held, did not hold, or untested
- Where the banned phrase appeared, or that it did not
- Which component failed, if any
- What this run does not prove

</deliver>

<refuse>
Do not accept a cleaned-up or summarised reply. Ordering is the evidence, and summarising destroys it. Do not treat an absence of violations in ordinary work as proof the gate functions. Do not generalise from one blocked phrase to the whole constraint set. If no failing input was ever submitted, state that the gate is unverified. Say the question cannot be answered yet.
</refuse>
How you know it worked

Send a request whose easiest answer uses a phrase on the banned list. Then read the reply in order. Look for the banned phrase in the delivered text. If it appears, the check is a report. A correction printed underneath does not change that. A working gate shows the validator called before the reply. The banned phrase never reaches the reply at all.

from a real incident, 2026-09-01 · stack AI coding assistant with hooks and tool calls · verified by whether violations appear in sent output, or only in post-send reports · ai-agentsvalidationhooksprompting

← every Field Kit