Field Kit
The manifest listed models the machine lacked
A static manifest records what someone expected to be installed, so enumerate dynamic resources by querying the live system and let the file supply metadata only.
What happened
On 2026-09-08, a resource list built from a YAML manifest showed installed software models the machine did not have and omitted ones it did, so the interface misrepresented what the user could actually use.
3 prompts, in order
- Diff the manifest against a live probe Your resource list shows items the machine does not have.
- Rebuild the list probe-first The diff proves the file and the machine disagree.
- Prove the list follows the machine The patch is in and you need evidence it tracks reality.
1Diff the manifest against a live probe
- Run it when
- Your resource list shows items the machine does not have.
- It asks you for
- The manifest contents and the raw output of a live query.
- You get back
- Three buckets: manifest-only entries, live-only entries, and entries present in both.
- It refuses if
- No live query is available, or its output is described rather than pasted.
<stance> A resource list is a statement about the machine, not about a file. Manifests record intent. The live system records fact. They drift apart the moment a user installs or deletes something. Your role here is auditor, not designer. Measure the gap before anyone starts building around it. A file that parses cleanly still tells you nothing about what is present on disk right now. </stance> <inputs> 1. The full contents of the config or manifest file that feeds the list today. 2. The exact command or API call that asks the live system what is installed. 3. The raw, unedited output of that command on this machine. 4. Where the list appears in the product, and who edits the manifest. Wait for all four answers. Do not proceed on a summary of item 3. Ask again and stop. </inputs> <method> 1. Parse the manifest into a flat list of resource identifiers. Print that list. 2. Parse the pasted live output into a flat list of identifiers. Print that list too. 3. State every normalisation rule you apply to either side. Name the rules explicitly. 4. Build three buckets. Manifest-only, live-only, and present in both. 5. For each manifest-only entry, say what the interface currently offers the user. 6. For each live-only entry, say what the user cannot reach today. 7. Do not guess that two differently named entries are the same resource. List ambiguous pairs separately. 8. Ask the user to resolve each ambiguous pair before you count it. 9. If both lists match exactly, say so. Then name the next place drift could enter. </method> <deliver> - Live inventory, as parsed from the pasted output - Manifest inventory, as parsed from the file - Three buckets, with counts - Ambiguous pairs awaiting a human decision - User-visible consequence, one line per mismatch </deliver> <refuse> Do not treat the manifest as the inventory. It is only one input here. Do not infer what is installed from imports, documentation or defaults. Do not fill a gap in the live output with a plausible entry. If the live command errors, report the error and stop. An error is not an empty inventory. If no live query exists for this resource type, say that plainly. Then state what would have to be built first. </refuse>
2Rebuild the list probe-first
- Run it when
- The diff proves the file and the machine disagree.
- It asks you for
- The code path that builds the list and the probe's captured output format.
- You get back
- A patch that enumerates from the probe and joins manifest metadata onto it.
- It refuses if
- The probe output shape is assumed instead of captured from a real run.
<stance> Authority is what changes here, not parsing. A live probe decides which resources exist. The manifest supplies labels, descriptions and ordering. Keep those jobs apart. An easy error is to keep looping over the file and merely filter it against the probe, which still hides anything the file never mentioned. Enumerate from the machine. Decorate from the file. </stance> <inputs> 1. The function or module that currently produces the list, pasted in full. 2. The captured output of the live query, in its real shape. 3. The metadata fields the manifest supplies that the probe does not. 4. What the interface should show for a live resource with no manifest entry. Wait for item 4. Do not pick a fallback label on the user's behalf. </inputs> <method> 1. Name the exact line where the manifest becomes the iteration source. Quote it. 2. Write the probe call as a separate function. Return identifiers and nothing else. 3. Make that function the only thing the list iterates over. 4. Join manifest records onto probe identifiers by a key the user confirms. 5. Handle the unmatched case the user described in item 4. Do not invent a placeholder. 6. Decide where the probe runs. State whether it runs per render or per refresh. 7. Define behaviour when the probe fails. Choose a visible error over a stale cached list. 8. Confirm the patch no longer reads the manifest to answer the existence question. 9. If the probe output lacks a stable key, stop. Report that the join is unsafe. </method> <deliver> - The patch, as a diff - Probe function, with its exact command and parsing - Join rule, and the key it uses - Unmatched-resource behaviour, as the user specified it - Probe-failure behaviour, stated in one line </deliver> <refuse> Do not filter the manifest by the probe. That still drops live-only resources. Do not cache the probe result past the refresh the user approved. Do not write a parser against an output shape you have not seen. If the pasted code is partial, ask for the rest. Do not reconstruct the missing part. If the probe exposes no field that matches a manifest key, say so and stop. </refuse>
3Prove the list follows the machine
- Run it when
- The patch is in and you need evidence it tracks reality.
- It asks you for
- One resource you can add or remove outside the app, plus the pre-fix code.
- You get back
- A test that fails on the manifest-reading version and passes on the patch.
- It refuses if
- The proposed test passes against the pre-fix code path.
<stance> A test that passes before the fix proves nothing about the fix. The claim under test is narrow. Lists must reflect the machine, not the file. So the evidence must come from changing the machine while the file stays still. A fake probe that returns the same thing the manifest says will pass either way. That is the result to guard against. </stance> <inputs> 1. The pre-fix version of the list-building code. 2. The post-fix version. 3. One resource the user can genuinely install or remove on this machine. 4. The manifest state during the test, confirmed unchanged. Wait for item 3. If nothing can be added or removed, ask what can be faked at the probe boundary instead. </inputs> <method> 1. Write one test with a live-only resource. The probe reports it. The manifest omits it. 2. Run that test against the pre-fix code. Record the failure and quote it. 3. If it passes against the pre-fix code, the test is wrong. Rewrite it. 4. Write a second test with a manifest-only resource absent from the probe. 5. Assert the list excludes it. Run it against the pre-fix code and record the result. 6. Run both tests against the post-fix code. Record pass or fail verbatim. 7. Do an out-of-process check. Remove the real resource, reload, and read the list. 8. Add the resource back. Reload again and read the list a second time. 9. If the out-of-process check disagrees with the tests, trust the reload. Report the gap. </method> <deliver> - Live-only test, with its pre-fix failure output - Manifest-only test, with its pre-fix result - Post-fix run, both tests, output quoted - Out-of-process reload observation, before and after - Any disagreement between the tests and the reload </deliver> <refuse> Do not report a test as proof without showing it fail on the old code. Do not stub the probe to return the manifest's contents. That erases the distinction being tested. Do not edit the manifest during the test. The file must stay fixed. If the pre-fix code is unavailable, say the fix is unproven and stop. If the reload cannot be performed, report the tests as partial evidence only. </refuse>
How you know it worked
Remove one installed resource outside the app. Reload the list. The entry disappears without any edit to the manifest. Add the resource back. The entry returns on the next reload.