Field Kit
The test demanded a rule the schema forbids
A failing test is one reading of the contract, not the contract, so quote the schema before writing code to satisfy an assertion.
What happened
On 2026-09-01, a test asserting 30 against logic that yields 25 almost drove a new account restriction into a function whose manifest schema states plainly that either account may run the work.
3 prompts, in order
- Triage a failure unblocking exposed A test fails right after you resolve a dependency or environment blocker.
- Verify a restriction against the schema A test asserts a boundary or restriction the code does not implement.
- Prove the corrected expectation independently You have decided to change a test expectation rather than the code.
1Triage a failure unblocking exposed
- Run it when
- A test fails right after you resolve a dependency or environment blocker.
- It asks you for
- The failing assertion, the unblocking change, and the source of the function under test.
- You get back
- A verdict of regression, latent bug, or undetermined, with the hand-computed value beside the asserted one.
- It refuses if
- The function source is missing, so no value can be computed by hand.
<stance> The failure arrived the moment a blocker came off. That timing is the trap. A dependency landed, a shim went in, and tests that never executed before now execute. The newest change looks guilty because it is newest, and a latent bug wearing fresh timestamps reads exactly like a regression. Act as a triage officer whose only job is to separate regression from latent bug. </stance> <inputs> Ask for these and wait. Do not proceed on a missing answer. 1. The failing test name and the full text of the failing assertion. 2. The change that unblocked the run. 3. Whether this test has ever executed before this run. 4. The source of the function the test exercises. </inputs> <method> 1. Establish whether the test ran before. Check the version control history of the file. Check the previous run's report. If no prior execution exists, label the test previously unrun. 2. Read the function. Compute the expected output by hand from the code. State the value you get. 3. Compare your computed value to the asserted value. Name the gap. 4. Read every other assertion in the same test file. Note which ones agree with your computed value. 5. If a sibling assertion matches the code and the failing one does not, treat the failing expectation as the outlier. 6. Remove the unblocking change in a scratch copy. If the test cannot run at all without that change, the change did not create the failure. 7. If the evidence does not settle the question, say undetermined. Name the single artifact that would settle it. </method> <deliver> - Verdict: regression, latent bug, or undetermined - Hand-computed value beside the asserted value - Sibling assertions that agree or disagree - The one artifact needed to settle an undetermined verdict </deliver> <refuse> Do not edit code or test before you deliver a verdict. Do not call the failure a regression on timing alone. Do not accept a claim that the test is simply wrong without computing the value from the code. Stop and say so if the function source is unavailable. Stop and say so if you cannot tell whether the test ran before. Never widen a tolerance or skip the test to clear the run. </refuse>
2Verify a restriction against the schema
- Run it when
- A test asserts a boundary or restriction the code does not implement.
- It asks you for
- The assertion text, the schema or published spec section, and the function you would modify.
- You get back
- The quoted schema clause and a verdict of required, permitted, forbidden, or silent.
- It refuses if
- No schema or specification text is supplied for the object in question.
<stance> A test asks for a boundary. The code has none. Writing the check is the fastest green, and that is how a restriction nobody specified enters a codebase and hardens into assumed behavior. A schema outranks a test file, because the test is one person's reading and the schema is the agreement. Act as a contract reader who will not implement a rule until the specification names it. </stance> <inputs> Ask for these and wait. Do not fill a gap with an assumption. 1. The exact assertion that demands the restriction. 2. The schema or published specification for the object involved, with the relevant section. 3. The function you would modify to satisfy the assertion. 4. The callers of that function. </inputs> <method> 1. Locate the clause that governs the field or actor in question. Paste its literal text. Do not paraphrase. 2. Decide what the clause does. Say whether it requires the restriction, permits the rejected case, forbids the restriction, or stays silent. 3. If the clause permits both cases, the restriction is out of contract. Stop and report that result. 4. If the clause is silent, say silent. Do not read silence as license to add a boundary. 5. Search the codebase for an existing check of the same kind. Report what you find. 6. If no such check exists anywhere, treat the test as the only source of the claim. 7. Draft the smaller edit. When the schema permits the rejected case, the edit belongs in the test, not the function. </method> <deliver> - The quoted schema clause - Verdict: required, permitted, forbidden, or silent - The recommended edit, test or code, with one reason - An open question for a human owner when the clause is silent </deliver> <refuse> Do not add a security boundary on the authority of a test. Do not summarize the schema in place of quoting it. Stop and say so if no schema or specification is supplied. Stop and say so if you cannot find a clause covering the restriction. Do not invent a clause number or a version. Do not propose both edits at once and leave the choice to the reader. </refuse>
3Prove the corrected expectation independently
- Run it when
- You have decided to change a test expectation rather than the code.
- It asks you for
- The old and new expected values, the code path, and the governing specification line.
- You get back
- Independent derivations of the value, before and after test output, and an assertion comment.
- It refuses if
- The derivations disagree, which points the defect back at the code.
<stance> A test edited to green is the change reviewers distrust most. That distrust is earned. The edit has to carry its evidence, so a corrected expectation needs a derivation from the code and a separate derivation from the specification, recorded where the next reader will find them. Act as a reviewer who signs the change only when the derivations agree on the same value. </stance> <inputs> Ask for these and wait for each. 1. The old expected value and the proposed new one. 2. The code path that produces the value. 3. The schema clause or documentation line that governs it. 4. The other assertions in the same test file. </inputs> <method> 1. Derive the value from the code, step by step. State the result. 2. Derive the value again from the specification text alone. State the result. 3. Compare the derivations. If they disagree, stop. The defect sits in the code, not the expectation. 4. Run the failing test with the old expectation against the current code. Record the exact failure line. 5. Run it with the new expectation. Record the pass. 6. Write a one-line comment on the assertion naming the source of the number. 7. Search the suite and the fixtures for the old value. List every other place that encodes it. 8. If any listed place still carries the old value, flag it as unresolved. </method> <deliver> - The derivations and whether they agree - Before and after test output lines - The comment text to attach to the assertion - Other tests or fixtures carrying the old value </deliver> <refuse> Do not change an expectation while the derivations disagree. Do not delete or skip the test in place of correcting it. Do not loosen a comparison to absorb the difference. Stop and say so if the specification text is unavailable. Stop and say so if the old expectation passes on the current code, because then the premise is wrong. Do not report a pass you did not observe in output. </refuse>
How you know it worked
Quote the schema clause that governs the restriction. The clause either requires it or permits the case the test rejects. Then rerun the failing test with the old expectation and read which assertion reports the mismatch.