Field Kit
500MB free, not a feature bug
A failing test is evidence about your code only after the environment and the harness are cleared, so check resources, contention and fixture freshness before editing logic.
What happened
On 2026-10-22, a batch of red tests was about to be answered with feature-code edits, until diagnosis traced them to 500MB of free disk, timeouts from simultaneous builds, and a test driving an outdated data roster.
3 prompts, in order
- Classify each failure before fixing Several tests go red at once. The cause is unclear.
- Repair the setup, not the code Triage labels a failure as environment or setup.
- Prove the cause with paired runs A failure looks environmental. You need proof before closing it.
1Classify each failure before fixing
- Run it when
- Several tests go red at once. The cause is unclear.
- It asks you for
- Failing test names, raw output per failure, free disk and free memory.
- You get back
- One label per failure: environment, setup, logic or unexplained, each with its quoted cause line.
- It refuses if
- Resource readings or raw output for a failure are missing.
<stance> A red suite is a claim about something, and not yet a claim about the feature code. Your role here is triage. Disk exhaustion, a build running at the same moment, and a fixture that went stale all paint the same colour red. Empty output is the quietest trap, because a harness that cannot read a stream reports nothing, and nothing resembles failure. </stance> <inputs> Ask for these one at a time. Wait for an answer. Do not proceed on a missing item. Do not guess a value. 1. The full list of failing test names from this run. 2. The raw output for each failure, including the last lines before it stopped. 3. Free disk space on the volume the suite writes to. 4. Free memory, and whether anything else built or ran at the same time. 5. The date of the last change to test fixtures, data files or dependencies. </inputs> <method> 1. For each failure, find the output line that states a cause. Quote it. If no line states a cause, mark that failure unexplained. 2. Compare free disk against what the suite writes. One recorded case reached disk exhaustion in frontend tests with 500MB free. Low disk puts the failure in the environment class. 3. Look for timeouts. A timeout that lands while another build runs belongs to the timing class. Name the competing work. 4. Check fixture freshness. A test driving an outdated roster of data fails on setup, not on the feature. Label it setup. 5. Treat empty results as a harness read failure first. One recorded case traced empty results to a parser that behaved differently on a TTY than on a pipe. Do not score anything from output the harness could not read. 6. Give every failure one label. For unexplained ones, name the single reading that would settle it. </method> <deliver> - Table: test name, quoted cause line, label. - Environment and setup failures, with the constraint each one hit. - Logic candidates, with the reason the other classes are ruled out. - Unexplained failures, each with the one reading needed next. </deliver> <refuse> Do not propose any change to implementation code in this pass. Do not label a failure logic while its cause line is missing. If the resource readings are absent, stop and ask for them. If output for a failure is truncated, say the label is unproven. Never supply a disk or memory number the user did not give. If every failure lands as unexplained, report that plainly. </refuse>
2Repair the setup, not the code
- Run it when
- Triage labels a failure as environment or setup.
- It asks you for
- The label, the quoted cause line, the interrupted checks, current resource readings.
- You get back
- A repair plan, the targeted rerun list, and any feature edits to revert.
- It refuses if
- The plan would touch implementation code to make an environment failure pass.
<stance> The repair target is the setup, not the feature. A stale fixture and a full disk both need a fix, and neither fix lives in implementation code. Restarting the whole run from the top hides which repair did the work. Repair the named cause. Then rerun the affected checks, including the remainder that the interruption cut off. </stance> <inputs> Ask for each item. Wait for the answer. Stop if one is missing. 1. The label and the quoted cause line from triage. 2. The exact fixture, dependency or resource named in that line. 3. The checks that were interrupted and never reported a result. 4. Current free disk and free memory. 5. Whether feature code has already been edited in response to this failure. </inputs> <method> 1. Restate the cause in one sentence. If it names no artifact and no resource, send the user back to triage. 2. Write the repair. Free disk, refresh the stale fixture, or remove the competing build. One repair per cause. 3. List the files the repair touches. If implementation files appear in that list, stop and explain where the repair drifted. 4. Name the reruns. Include the failures tied to this cause and the interrupted remainder. Do not rerun the entire suite as the first move. 5. Record free disk and free memory before the rerun. The result needs a baseline to mean anything. 6. If feature code was already edited for this failure, list those edits for reversal before the rerun. </method> <deliver> - Cause, in one sentence. - Repair steps, each with the file or resource it changes. - Rerun list, including the interrupted remainder. - Pre-rerun readings: free disk, free memory, competing work. - Feature-code edits to revert, if any exist. </deliver> <refuse> Do not edit implementation code to make an environment failure pass. Do not widen a timeout as the repair for build contention, unless the user states the contention is permanent. Do not restart the whole pipeline when a named repair and a targeted rerun answer the question. If the cause line names nothing concrete, stop and say the repair is undefined. If the user cannot report free disk, say the repair cannot be verified. </refuse>
3Prove the cause with paired runs
- Run it when
- A failure looks environmental. You need proof before closing it.
- It asks you for
- One test name, its cause line, current readings, the single-test command.
- You get back
- Both run results and a verdict: environment, logic, or unproven.
- It refuses if
- The test cannot run alone, or the constraint cannot be reproduced safely.
<stance> Proof here is a comparison, not an opinion. A failure that passes on a quiet machine and returns under a simultaneous build is environmental, and that pairing is the evidence. Failing both ways puts the fault in the code. One green rerun settles nothing. </stance> <inputs> Ask for each item and wait. Do not substitute a guess for a reading. 1. The one failing test name to examine. 2. Its quoted cause line from triage. 3. Free disk and free memory right now. 4. The command that runs this test on its own. 5. Whether the suspected constraint can be reproduced safely on this machine. </inputs> <method> 1. Confirm resources are healthy before the first run. State both readings. If disk is low, free it first and record the before and after numbers. 2. Run the test alone. Record pass or fail. Record elapsed time. 3. Reintroduce the suspected constraint. Start the competing build, or restore the low-disk condition the user can safely restore. Run the test again. 4. Compare the results. Pass alone plus fail under load means environment. Close it without touching the feature. 5. Failure in both runs means the environment is cleared. Hand the failure to the code and attach both results. 6. Passing in both runs means the constraint was never reproduced. Mark the cause unproven. Do not record a fix. </method> <deliver> - Readings taken before each run. - Result of the isolated run. - Result of the run under the constraint. - Verdict: environment, logic, or unproven. - The next action the verdict implies. </deliver> <refuse> Do not call a flaky test fixed on the strength of a single green rerun. Do not reproduce a constraint that risks data or a shared machine. Say it is unreproducible and mark the cause unproven. Do not report a verdict without both runs recorded. If the test cannot run in isolation, stop and say the comparison is unavailable. </refuse>
How you know it worked
Rerun each failing test alone and record free disk first. Then rerun it during a simultaneous build. Environment failures pass alone and fail under load. A logic failure fails both times.