The check passed because it never saw the edit

I put a known bug back into Battuta, a MIDI editor I’m building, to see whether a new check would catch it. The edit was wrong in exactly the way the check was meant to detect. It still passed: the broken edit had kept its own evidence out of the check.

I use Battuta to work on music with an AI agent. The agent reads descriptions of notes and instruments; I listen to the result. A file can satisfy those descriptions while giving the player instructions in the wrong order. This check was meant to catch some of those mistakes before playback.

The timestamp was right; the sequence was wrong

A MIDI note has separate instructions to start and stop it. Suppose one note is shortened so that it ends exactly when the next note of the same pitch begins, on the same channel. The ending has reached the right timestamp. There is still an ordering decision inside it.

The second sequence can cut off the note that has just started. The stop instruction carries a pitch and channel, without a note identity saying “only the earlier one.” Checking the requested end time therefore misses the relationship that matters: the release must precede the next start.

Battuta stored time and position within a timestamp separately. The resize operation updated the time but omitted the call that would place the release among its new neighbours. The file could still be encoded; its validity said nothing about whether the intended note would survive playback.

The resize regression checks the output event sequence. A shared check could also enforce that relationship after any edit that moved a release, without needing a separate listening test for each operation.

The missing call also hid the change

The check ran after the batch of edits. It needed to distinguish events the batch had changed from events already present in the input: Battuta preserves the order of untouched events, even where its own editing rules would have placed them differently.

The first implementation recorded events when they passed through the placement function. After the batch, it checked their placement. But the resize defect was an edit that never called that function.

The implementation record documents that failed attempt: restoring the resize defect left the check silent. The mistake was in how the check selected its evidence, before it made any comparison.

The fix records changes in the underlying mutation methods, including the one that sets the timestamp. Now the release enters the check even if its caller forgets to place it afterward. The check can finally compare its resulting order with the rule.

There was a related trap in deciding which rule to apply. The placement function accepts a category from its caller: a note start, a release, or a state change such as an instrument selection. Trusting that same category in the checker would let both agree on a wrongly labelled event.

Instead, the checker reads the MIDI message itself. A release remains a release, whatever the edit called it. One unit test deliberately places a controller change as though it were a note start. The checker rejects the resulting order.

Correct placement can still follow the wrong choice

The design record grouped five defects under a similar symptom: an edit could succeed without producing the intended playback. It proposed that checking event order would catch all five. The implementation showed why that claim was too broad.

One defect selected the wrong instrument-change event. The input contained two selections at the same address. The later one was in force, but the edit changed the earlier one. Here are the values from its regression fixture.

Changing the first event can leave every placement relationship valid. The edit selected the wrong target and then placed it correctly. A check of placement has no reason to object.

Its regression test supplies what is missing: the input, the request, and the expected output events. It asserts that the earlier selection stays 40 and the later one becomes 42. An arbitrary file containing 42 followed by 60 would not, by itself, establish that this edit had gone wrong.

The other defects needed separating too. A batch could look up a named controller event again after an earlier edit, reach a different event, and change something the request had never named. That needed a target-selection regression. State and notes on different tracks raised another question: whether the file supplied an order between them at all. Battuta handles that with a separate check, described in the audio comparison.

Putting the defect back tests the claim about the check

The implementation record reports deliberately reintroducing the defects against the actual code. The placement check caught the misplaced release and the state change written behind the notes it governed. Reversing the order of statements also triggered it. The two target-selection defects passed that check and failed their dedicated regressions.

Those results support a narrower claim than the design record made. The placement check catches particular violations in events the batch wrote, provided the mutation methods record those writes. It does not establish that the request selected the right events, that every cross-track dependency has an order, or that playback will be identical on every synthesiser.

I would keep the audio comparisons. They test a consequence that structural assertions only model; another audible failure may expose a relationship the model has not represented yet. The cheaper check earns its place by catching the ordering mistakes it can identify before a render is needed.

For a test of the output, the question is whether the result is wrong. For a test of the checker, there is an earlier question: did the evidence of that wrong result ever reach it?