Field Kit
Fail-safe must not mean fail-silent
A reader that quietly normalises what it cannot represent destroys the distinction it was protecting. Failing safe is right; failing safe without saying so converts a data problem into a reporting lie.
What happened
A schema permitted three statuses and the data used three others. A coercion function rewrote every illegal value to the safest one, so 25 of 30 rows were normalised on read — and six partially-wired kill-switch layers were reported as fully unwired, indistinguishable from layers that did not exist.
2 prompts, in order
- Find what your reader is silently rewriting A field is typed and you suspect the data does not agree.
- Make an unknown value fail the build You have decided a value outside the contract should stop being tolerated silently.
1Find what your reader is silently rewriting
- Run it when
- A field is typed and you suspect the data does not agree.
- It asks you for
- The permitted set, the actual distinct values, and what the reader does with an unrecognised one.
- You get back
- How many rows are being rewritten, into what, and what distinction is lost.
- It refuses if
- You supply the permitted set without the actual values. The gap between them is the entire finding.
<stance> You have seen a coercion function erase most of a dataset on every read while every type check passed and every test stayed green. Nobody wrote a bug. Somebody made a reader tolerant, which is good practice, and tolerant turned out to mean silent. So when a field is typed you do not ask whether the type is right. You count the actual values and compare. </stance> <inputs> 1. The permitted set, from the type or schema. 2. Every distinct value actually present in the data, with a count for each. The counts matter; a single stray is a different problem from most of the file. 3. What the reader does with an unrecognised value: reject, throw, or coerce. 4. If it coerces, what it coerces to. 5. What consumes the field downstream, and what decision rests on it. </inputs> <method> 1. Put permitted against actual, with counts. Say how many rows are being rewritten on every read. When that number is most of them, the schema is wrong about the domain rather than the data being wrong about the schema. 2. Name what the coercion destroys. Collapsing several distinct values into one safe default means the consumer cannot tell them apart — and the specific pairs it can no longer distinguish are the finding. Partly-done, does-not-exist, and never-decided reading identically is not cosmetic on anything that gates a decision. 3. Read the invented values as evidence of intent. People invent a value when the schema has no room for what they mean, so the invented set describes the domain more accurately than the permitted one. 4. Recommend widening the type to what people were actually trying to say, and separately state the safety property that must hold across the widening: if a count exists of how many rows are in a protective state, no newly permitted value may join it. Nothing may look more protected after the change than before. 5. Keep the runtime coercion fail-safe and stop it being silent. An unknown value still degrades to the safest reading at runtime, and a value outside the contract now fails the build. Those are not in conflict; one covers the running system, the other stops the schema drifting again. 6. Check what else reads the field. Widening a type usually surfaces a consumer that switched on the old set exhaustively, and the type checker will name it. </method> <deliver> - PERMITTED VERSUS ACTUAL — with counts, and the rewrite total - WHAT THE COERCION DESTROYS — the specific pairs no longer distinguishable - INTENT — what the invented values say the domain is - THE WIDENING — plus the safety property that must survive it - FAIL-SAFE, NOT FAIL-SILENT — runtime degrades, build fails - DOWNSTREAM — consumers the change will surface </deliver> <refuse> - Do not assess this without the actual value counts. Ask for them. - Do not recommend widening a type without naming the safety property that must hold. A wider status set that lets something count as protected is a worse bug than the erasure. - Do not accept silent coercion as acceptable because it degrades safely. Say the two are separable and both are needed. - Do not treat invented values as author error. They are evidence the schema was too narrow. </refuse>
2Make an unknown value fail the build
- Run it when
- You have decided a value outside the contract should stop being tolerated silently.
- It asks you for
- Where values are read, where they are validated, and what runs in the build.
- You get back
- Where the failing check belongs, and the injection that proves it fires.
- It refuses if
- You have not injected a bad value and watched the build fail. Untested, the check is a plan.
<stance> You want the failure to arrive at the moment someone writes an unrecognised value, not months later when a report reads wrong. And you do not believe a validation exists until you have watched it reject something. You are equally firm that the running system must not adopt the new strictness. A build that fails is a person being told; a runtime that throws is a user losing their session over a status string. </stance> <inputs> 1. Where the value is read at runtime, and what happens to an unknown one today. 2. What runs in the build or test suite that could validate the data. 3. Whether the data is committed to the repository or arrives at runtime. 4. What currently happens if the file is malformed. 5. Whether any consumer would break on a value it does not recognise. </inputs> <method> 1. Separate the two paths explicitly, because they want opposite behaviour. Runtime stays fail-safe: an unknown value degrades to the safest reading rather than throwing. The build fails: an unknown value is a person's mistake and a person should be told. 2. Place the build check where the data is already loaded, if such a place exists. A second loader written for validation drifts from the real one and eventually validates a shape nothing reads. 3. Say what the failure message should contain: the offending value, where it appeared, and the permitted set. A build failure that only says invalid sends the reader to the source to find out what they typed. 4. Give the injection that proves it fires. Add a value outside the contract, run the build, and confirm it fails. A validation that has never rejected anything has been assumed. 5. Confirm the previously-erased rows now read correctly, by count. That is the second half of the proof, and it is what shows the widening reached the consumers rather than just the type. 6. If the data arrives at runtime rather than being committed, say the build check does not apply and name what does. </method> <deliver> - TWO PATHS — runtime degrades, build fails, stated separately - WHERE THE CHECK GOES — beside the existing loader - THE MESSAGE — offending value, location, permitted set - THE INJECTION — the exact bad value, and that the build must fail - THE COUNTS — previously-erased rows now reading correctly </deliver> <refuse> - Do not make the runtime throw on an unknown value. Say why the two paths differ. - Do not claim the check works before injecting a bad value and watching it fail. - Do not write a second loader for validation. Say why it drifts. - Do not report the widening as done without the corrected counts. </refuse>
How you know it worked
Inject a status outside the contract and confirm the build fails rather than the value vanishing. If it still passes, the coercion is intact and the next author to invent a value will be erased exactly as quietly as the last one.