Notes from the Pipeline

On the morning of August 30, five Claude Code entries went out on our LLM page. They were identical to the five from the day before. Same Korean headline, same English headline, same two-line explainer.

A person caught it, not the logs. All our record said for that day was "14 items".

The key is the title

We pull release notes from ten places. The formats have nothing in common, so we cannot select by date. Half of them carry no per-item date at all. So we do not ask "was this published today?" We ask "have we run this before?"

We answer that by title. We lowercase it, strip the punctuation, and compare. Release notes reword the same entry and repost it often enough that a literal comparison misses things.

The trouble was that the two titles being compared come from different code. One is built by the collector. The other is read back out of what we already published.

what the collector produced
  Added PreModelSwitch and PostModelSwitch hook events (block, confirm,
  or annotate a model switch);...

what got published
  Added PreModelSwitch and PostModelSwitch hook events (block, confirm,
  or annotate a model switch); SessionStart resume hooks now receive
  session staleness and the estimated re-cache cost

The collector cuts long entries at 110 characters. A single markdown bullet routinely runs past 100, and you cannot use that as a headline. The next stage does the opposite and writes down the full untruncated text, because that field is what we check a shaky translation against. Truncated, it is useless.

Each choice is right on its own. They only disagree where they meet.

The other end broke too

The parser that reads the Claude app release notes was skewed the opposite way. It dragged the start of the next entry onto the end of a title, and the next stage tidied that up before writing it down. Here the published title is shorter than the one the collector made.

Opposite directions, same result. Compared literally the two strings do not match, and an entry that does not match is new again every day.

Why it stayed hidden

We record how many entries got filtered out. On August 29 it was 115. On August 30 it was 125. The number was going up.

That number counts what was filtered. It does not count what should have been. An entry whose key does not match sails straight past the comparison, so it lands in no bucket at all. A failure that shows up as zero gets noticed. This one showed up as a healthy rising count.

The spacing helped hide it too. Candidates compete for forty slots every day, and a resurrected entry does not win one every time. The Claude app's Fable 5 entry ran on August 25, then the 27th, then the 30th. Two days off, then three. It read like coincidence rather than a repeating fault.

Each run treated it as brand new, so we rewrote it from scratch each time. Here are the three.

In Korean it was worse: the product name itself came out spelled three different ways. Until you line them up, they do not look like the same entry.

The fix

If one normalized title is a prefix of the other, treat them as the same entry. A truncated title is a prefix of the full text. A tidied title is a prefix of the glued one. One rule covers both directions. To keep short titles from colliding on a shared opening, the prefix has to run at least 40 characters.

The other option was to forbid the next stage from touching the title at all. We passed on that. Leaving the fallback evidence truncated is worse, and a rule that a person has to honor every single day eventually gets broken. Better for the side doing the judging to absorb the difference.

One more, found alongside

Digging through the parser turned up the inverse symptom. When a title swallows the next entry, the swallowed one never becomes a candidate at all. One entry from July 1 and three from April 9 were simply absent. We thought we were picking up 92 entries. It was 99.

That one leaves no trace. A duplicate at least shows up twice on the page. A missing entry is just missing. The only tell was a title running past 200 characters, and that value never appears on any screen.

What stays

The two symptoms are opposites with one root. Treat the same thing as different and you get duplicates. Treat different things as the same and entries disappear. Neither raises an exception, and both keep the counters looking fine.

We removed the duplicates that already shipped. August 30 went from 14 entries to 8, and one came off August 27. We left the days that ran them first alone.

Adding another counter would not have caught this. The one we had could not see the thing that was broken. How many you filtered will never tell you how many you missed.