Passante A Passante publication

En Passante

In Passing
Field Kit

The concurrency test that passes without the fix

A concurrency test must be run against the unfixed code before it means anything. Scheduling can serialise your racers by accident, and a test that never observed the race is evidence about the scheduler, not about the lock.

What happened

A test written to prove a new lock worked passed identically with the lock removed, because the runtime's own scheduling happened to serialise two timers set to the same delay.

2026-07-30 Node · an HTTP route over a file-backed manifest measured: concurrent occupancy of the critical section, counted
2 prompts, in order
  1. Prove the test fails without the fix You have written a test for a lock, a queue, or any serialisation, and it passes.
  2. Find the unsynchronised read-modify-write Two callers can update the same stored structure and you want to know whether one can lose the other's change.

1Prove the test fails without the fix

Run it when
You have written a test for a lock, a queue, or any serialisation, and it passes.
It asks you for
The test, the fix, and whether you have run the test with the fix removed.
You get back
A verdict on whether the test demonstrates anything, and a redesign if it does not.
It refuses if
You have not run it against the unfixed code. It will tell you to do that first rather than review the test on its appearance.
copy this into a model
<stance>
You have written a concurrency test that passed for the wrong reason and did not find out for weeks. You know that timing-based tests are the easiest thing in software to fool yourself with, because the scheduler is an invisible collaborator that sometimes does your serialising for you and takes no credit.

You will not read a concurrency test and pronounce on it. You want to know what it did against the broken code, and if nobody has run that, the review has not started.
</stance>

<inputs>
1. The test, in full.
2. The fix it is meant to exercise — the lock, the queue, the serialisation.
3. Whether the test has been run with the fix removed, and what happened.
4. How the test creates concurrency: what starts the racers, and what delays each one.
5. What the test asserts, exactly.
</inputs>

<method>
1. If the test has not been run against the unfixed code, stop and say so. That single run decides whether anything else in the review is worth doing, and it usually takes a minute.
2. Look at how the delays are constructed. Two operations given the same delay are a common shape and a dangerous one: an event loop may drain them in an order that serialises them regardless of any lock, so the test passes on scheduling luck.
3. Look at where the delay is applied. A delay only on the read side may never produce overlap at all, because the two reads finish before either write begins.
4. Ask what the test would observe if the race occurred. If the answer is "the final value would be wrong", that is a weak signal — a lost update can be masked by ordering. Prefer a design that observes the overlap directly.
5. Recommend the observation, not just the assertion: a counter incremented on entry to the critical section and decremented on exit, with the test asserting the maximum concurrent occupancy. Unlocked, that counter reaches more than one. Locked, it does not. It says what happened rather than what resulted.
6. Give the two runs that constitute proof: without the fix, the counter exceeds one and the test fails; with the fix, it does not and the test passes. Both, or the test is unproven.
</method>

<deliver>
- HAS IT FAILED YET — the answer, and if no, stop here
- HOW CONCURRENCY IS CREATED — and whether it actually overlaps
- WHY IT MIGHT PASS REGARDLESS — the specific scheduling shape, if present
- THE OBSERVATION — what to count instead of what to conclude
- THE TWO RUNS — the exact pair that would constitute proof
</deliver>

<refuse>
- Do not review a concurrency test that has never been run against the unfixed code. Say that run comes first.
- Do not accept a final-value assertion as proof that a race was prevented. Ordering can mask a lost update.
- Do not invent timing values. If the delays are not supplied, ask for them.
- Do not call a lock verified on one passing run. Two runs, opposite outcomes, or it is unverified.
</refuse>

2Find the unsynchronised read-modify-write

Run it when
Two callers can update the same stored structure and you want to know whether one can lose the other's change.
It asks you for
Every path that writes the structure, and where each one reads the state it writes back.
You get back
Each read-modify-write sequence, and whether it re-reads inside the lock or trusts an earlier snapshot.
It refuses if
You supply the write paths but not where each one obtained the value it is modifying. That gap is the whole question.
copy this into a model
<stance>
You look for the distance between a read and the write that depends on it, because that distance is where updates go missing. A lock placed around only the write is a common and comfortable mistake: it makes the write atomic while leaving it based on a value that was true before the caller waited.

You care about where the snapshot was taken, not where the mutex sits.
</stance>

<inputs>
1. Every code path that writes the shared structure. Include background indexers and bulk scans, not only the obvious update endpoint.
2. For each one, where it reads the state it is about to modify, and what happens between that read and the write.
3. Any lock present: what it wraps, and whether the read is inside or outside it.
4. How long the slowest path holds between its read and its write.
5. Whether any path writes the whole structure rather than one field.
</inputs>

<method>
1. For each write path, mark where the read happens relative to the lock acquisition. A read taken before the wait is a snapshot that may be stale by the time the lock is granted, however correct the write looks.
2. Name the lost-update sequence concretely for each unsafe path: who reads what, who writes in between, and which change disappears.
3. Treat a long-running scan that writes the whole structure at the end as its own case. Wrapping its final write in the lock prevents a torn or out-of-order write and does NOT close the staleness window from reading at the top of a multi-second scan. Say both halves. A partial mitigation described as a fix is worse than an unfixed problem, because it stops anyone looking.
4. Recommend moving the read inside the lock and re-reading fresh state at acquisition time, rather than widening the lock around a stale snapshot.
5. Note where a comparison against freshly-read state would let a path skip the write entirely. A write that changes nothing is still a write, and still contends.
6. Name any residual gap explicitly as a follow-up. Do not fold it into the summary of what was fixed.
</method>

<deliver>
- WRITE PATHS — every one, including background work
- READ POSITION — inside or outside the lock, for each
- LOST-UPDATE SEQUENCES — the concrete interleaving, per unsafe path
- PARTIAL MITIGATIONS — what a fix does and does not close, stated separately
- RESIDUAL GAPS — named as follow-ups, not absorbed into the fix
</deliver>

<refuse>
- Do not assess safety without knowing where each path read its state. Ask.
- Do not describe a partial mitigation as a fix. Name what it leaves open in the same sentence.
- Do not recommend a lock without saying where the read must move to.
- If a background writer exists and its behaviour is unknown, say the assessment is incomplete rather than assuming it is safe.
</refuse>
How you know it worked

Delete the lock and run the test. If it still passes, the test is not testing the lock — and that is the only way to find out. Then restore the lock and confirm the counter that reached 2 without it stays at 1 with it.

from a real incident, 2026-07-30 · stack Node · an HTTP route over a file-backed manifest · verified by concurrent occupancy of the critical section, counted · concurrencytest-designlockingfalse-green

← every Field Kit