What broke, and what the fix was
This site publishes itself every morning at 08:00 KST, unattended. Building that pipeline turns up things that are wrong. These are the ones worth writing down: what broke, what the numbers actually were, and what the fix was, with the code. Not news summaries: a record of this site itself.
The VentureBeat feed died four days with a 429, then on the fifth with an SSL timeout. Different symptoms, one cause: the whole site had gone behind a bot challenge, a JavaScript wall a feed parser cannot pass. Chasing the symptoms would have sent me the wrong way.
The tool-updates page decides what is new not by date but by “have we run this title before?” Grok and Perplexity, though, give their entries no date at all, only a month heading. So on September 3, Grok items from January and March went up as “newly added”: old news dressed as the latest because we were seeing it for the first time.
A 403 looked like a rate limit, so I spent ten days fixing the GitHub token. Only after I logged the response body did I see the wall was not GitHub but the cloud environment the pipeline runs in.
We put the numbers on /data so readers can count what we filtered out for themselves. But the page said "98 minus 2 leaves 59 candidates." 98 minus 2 is 96. A whole pool, our domestic Korean news, was missing from the math on screen.
We answer "have we run this already?" by title, but the two titles being compared are built by different code. One truncates, the other restores. Entries that never matched were never filtered, and yet the filtered count rose every day.
I logged which feeds failed but not why. The morning after I started logging the reason, the cause showed up, and one of them had been arriving with a success-family status code.
One rule keeps a single outlet from taking over. One keeps the same event from running twice. One keeps posts that are not news out. Over the past month all three turned into their opposites, doing exactly what they say, without making a sound.
A three-article cap keeps one outlet from owning the edition. It lifts entirely rather than leave a slot empty. That happened four times in 28 editions, and only twice because the news was actually thin. One was a configuration mistake of mine. One I still cannot explain.
The keyword extractor behind our cross-source bonus only matches capitalized ASCII, so Korean headlines cap out at 3.0 against a weekday cutoff above 5. Yet they do run on Sundays and Mondays. Not on merit, but because the English feeds have gone quiet.