Passante A Passante publication

En Passante

In Passing
Field Kit

The wrapper counted as one record

Confirm whether the parsed root is the array or an object holding the array before changing any schema, because a wrapper iterates as one item.

What happened

On 2026-07-07, a script reading JSON reported zero receipts, because it iterated the top-level wrapper object `{ receipts: [...] }` as a single record rather than walking the array inside it.

2026-07-07 Node or Python · a JSON-reading batch script measured: records read from the wrapped file: zero before the fix
3 prompts, in order
  1. Find the real top-level shape A reader returns zero or one record from a file that is not empty.
  2. Normalize array or wrapped object The root shape is confirmed and the loop needs one entry point for records.
  3. Prove both shapes reach the loop The normalizer is in place and the fix needs evidence beyond a passing run.

1Find the real top-level shape

Run it when
A reader returns zero or one record from a file that is not empty.
It asks you for
One real input file, the iteration code, and the count the pipeline reports.
You get back
The observed root type, its keys, the array length, and a one-line cause statement.
It refuses if
No sample file is available, or the file fails to parse at all.
copy this into a model
<stance>

The reader returns zero records and the file on disk is not empty. That gap usually lives in the shape of the parsed root, not in the parser. A validator or schema change looks like progress here and moves nothing, because the loop still receives one object where it expects many. Shape first, schema second. Hold this diagnosis to printed evidence from the actual file.

</stance>

<inputs>

1. The path to one real input file that reproduces the low count.
2. The code at the point where the parsed JSON enters the loop.
3. The record count the pipeline currently reports for that file.
Wait for all three. Do not reason about the shape from memory of similar formats.

</inputs>

<method>

1. Parse the file. Print the type of the root value. Report whether it is an array.
2. If the root is an object, print its keys. Print the type and length of each value.
3. Name the key whose value is an array of records. If two keys hold arrays, stop. Ask the user which key holds records.
4. Count the elements under that key. Compare that number with the count the pipeline reports.
5. If the pipeline reports 1 and the key holds many, the loop iterates the wrapper. State that as the cause.
6. If the pipeline reports zero and the array is genuinely empty, the defect sits upstream. Say so and stop.
7. Repeat on a second file when one exists. Record whether both files share one root shape.

</method>

<deliver>

- Observed root: type, array or object, key list
- The records key and its length
- Reported count against actual length
- Cause statement, one line
- What remains unknown

</deliver>

<refuse>

Do not edit a schema, validator or type definition during this step. Do not guess the wrapper key from a field name used elsewhere in the codebase. Do not accept a one-element-array fallback as an explanation of the shape. If no sample file is available, say the root shape is unknown and stop. If the parse itself throws, report the parse error and stop there. Shape work waits for a file that parses.

</refuse>

2Normalize array or wrapped object

Run it when
The root shape is confirmed and the loop needs one entry point for records.
It asks you for
The confirmed shapes, the exact array key name, and the current iteration site.
You get back
One normalizer function, the call site change, and an error message for unknown shapes.
It refuses if
The array key name is unconfirmed, or the caller swallows the shape error.
copy this into a model
<stance>

Normalizing input is a small change with one trap. A fallback that wraps an unknown value in a single-element array makes the low count vanish and keeps the defect alive. The normalizer accepts the shapes that exist in real data and rejects everything else loudly. Short list, hard edges.

</stance>

<inputs>

1. The confirmed root shapes, for example a plain array and an object holding an array.
2. The exact key name under which the array sits.
3. The function or block that currently iterates the parsed value.
Wait for the key name. Do not infer it from naming conventions.

</inputs>

<method>

1. Find every place where parsed JSON enters iteration. List them. If more than one exists, normalize at the earliest shared point.
2. Write one function that takes the parsed value and returns an array of records.
3. Return the value unchanged when it is already an array.
4. Return the key's value when the root is an object and that value is an array.
5. Throw in every other case. Put the observed type and the root keys in the message.
6. Do not add a fallback that wraps the root in a one-element array.
7. Change the call site so the loop only receives the normalizer's output.
8. Print the record count immediately after normalizing. Keep that line in the code or the log.
9. If real data shows a third shape, add it by name. Do not widen the check to accept any object.

</method>

<deliver>

- The normalizer function, complete
- The call site diff
- The exact error text for unknown shapes
- The count line and where it prints
- Shapes accepted and shapes rejected

</deliver>

<refuse>

Do not change schema or validation logic in this step. Do not catch the unknown-shape error and continue with an empty array. Do not normalize inside the loop body. If the key name is unconfirmed, stop and ask for it. If the surrounding code also swallows parse errors, report that and ask before touching it.

</refuse>

3Prove both shapes reach the loop

Run it when
The normalizer is in place and the fix needs evidence beyond a passing run.
It asks you for
The normalizer, the confirmed shapes, and the command that runs the suite.
You get back
Three fixtures, test code, counts with and without the fix, suite output.
It refuses if
The wrapped case passes with the normalizer bypassed, or the suite cannot run.
copy this into a model
<stance>

The test must fail without the fix. A normalizer test that passes on both shapes proves nothing until the wrapped case fails on the old path. The fixtures carry the whole argument, so each one holds a known record count. Same records, same count. Treat the bypass run as the real result.

</stance>

<inputs>

1. The normalizer and the call site as they now stand.
2. The confirmed shapes, the plain array and the wrapped object with its key name.
3. The command that runs the test suite.
Wait for the command. Do not invent a runner.

</inputs>

<method>

1. Build two fixtures holding the same three records. One is a plain array. One wraps that array under the confirmed key.
2. Assert the reader returns 3 records for each fixture.
3. Build a third fixture whose root is an object with no array value.
4. Assert the reader raises an error. Assert the message names the observed keys.
5. Bypass the normalizer call. Run the wrapped fixture again.
6. Record the count seen without the fix. Confirm the wrapped test fails in that state.
7. Restore the normalizer. Run the full suite with the stated command. Paste the output.
8. If the wrapped test passes while the normalizer is bypassed, it misses the fixed path. Rewrite it and repeat step 5.

</method>

<deliver>

- Fixture files with their record counts
- Test code for all three cases
- Counts observed with the fix and with it bypassed
- Suite output from the stated command
- Any shape still untested

</deliver>

<refuse>

Do not generate fixtures by exporting from the pipeline under test. Do not assert absence of an error alone. Assert the record count. Do not skip the bypass run, it is the only evidence of coverage. If the suite cannot run, report the failure and stop. If the fixture record counts are unknown, stop and ask for them.

</refuse>
How you know it worked

Run the reader on a wrapped fixture holding three records. The printed count reads 3. Bypass the normalizer and run the same fixture. The count drops to 1.

from a real incident, 2026-07-07 · stack Node or Python · a JSON-reading batch script · verified by records read from the wrapped file: zero before the fix · jsondata-shapeiterationfixtures

← every Field Kit