We Shipped a 10x Error on General Mills. Here's the Bug, and the Check That Would Have Caught It.

On 2026-08-03 our API was serving General Mills' FY2024 revenue as $2.038 billion. The real figure is $19.857 billion. We were wrong by an order of magnitude on an S&P 500 company, for weeks, and every automated check we had said the data was fine.

This is the post-mortem. It is worth reading if you build anything on SEC XBRL, because the root cause is not exotic — it is the most natural way to write the code, and it is wrong.

What the filing actually says

General Mills tags two different numbers in the same 10-K for the same period:

Both come from SEC's own companyconcept API. Neither is a mistake by the filer. Revenues here is a component; the consolidated top line lives under the ASC-606 element.

We picked Revenues.

Why we picked the wrong one, and why it was defensible

Our tag resolution used a static priority list, with Revenues ranked first. That ordering was itself a fix. Earlier, we had ASC-606 contract revenue ranked above Revenues, and it produced this:

• MetLife FY2024 — served at $2.2B against a true $71.0B
• Capital One — 85% low
• AvalonBay — 99.8% low, for eight consecutive years

For insurers, REITs and card issuers, contract revenue is a small slice and Revenues is the real total. So we promoted Revenues, verified MetLife came out right, and shipped.

That fix was correct for MetLife and catastrophically wrong for General Mills — because there is no single XBRL element that is the consolidated top line for every filer. Revenues is the total for an insurer and a minor component for a consumer-goods company. Any fixed ordering serves one and betrays the other.

We had traded one class of error for another, and we could not see it, because we validated the fix against the company that motivated it.

Why nothing caught it

This is the part worth generalising. We had four automated gates:

• a claims check comparing published site copy to the database
• a minimum-row-count guard against truncated builds
• endpoint smoke tests
• an invariant scan for internal consistency

Every one of them asks "is this data consistent with itself?" None asks "is this number correct?"

$2.038 billion is internally consistent. It is correctly dated, plausibly sized for a public company, sourced from a real XBRL element in a real filing, and it passes every range check. It looks exactly like money. There is no internal signal that separates it from the right answer.

We even had a flag firing. The row carried qa_status = FLAG:ambiguous_tag, which our engine sets when another element in the same filing reports materially more. The system detected the ambiguity and published the smaller number anyway. Detecting is not deciding.

The second bug, which was worse

While fixing the first, we found that our bank-revenue path had been silently broken for months.

Banks frequently tag no single total-revenue element, so we synthesise one: net interest income plus noninterest income, the standard definition. That code computed Fifth Third's FY2023 revenue correctly at $8.708B. Then it stamped the value with the wrong filing date — 2026-02-24 instead of 2024-02-27 — because of this:

nii = {p["end"]: p for p in annual_points_for_tag(...)}

A dict comprehension keeps the last entry per key. SEC reports the same period across many filings, so the last one is the most recent re-filing. The synthetic bank total therefore always looked like it was filed years late.

Our selection logic picks the earliest-filed candidate, on purpose — that is the point-in-time discipline. So the correct bank total always arrived "late" and lost to the fee-income tag it existed to replace. The fix computed the right answer and threw it away, every single time. Fifth Third kept publishing $0.577B against a true $8.708B.

One line, and it silently reversed the outcome of a fix everyone believed had shipped.

The check that would have caught both

The missing piece was not more tests. It was one test of a different kind: compare the data to something outside itself.

node scripts/verify-against-sec.mjs

For a sample of served values, it asks SEC's companyconcept API for every candidate revenue element the filer reported for that exact period, and asserts that what we serve is the consolidated total rather than a component of it. It deliberately does not reuse our tag-selection logic — otherwise it would reimplement the bug and agree with itself.

Run against the broken data, it printed:

WRONG GIS FY2024: serving $2.038B vs SEC $19.857B — 10.3% of the total
WRONG GIS FY2023: serving $1.957B vs SEC $20.094B —  9.7% of the total
WRONG PG  FY2014: serving $29.400B vs SEC $83.062B — 35.4% of the total

It now runs on every data refresh and fails the build. A gate that cannot catch its own motivating case is decoration, so we proved it fired before shipping it.

The actual fix

We stopped ranking elements and started comparing them. Every tag in our candidate list is intended to be a consolidated total — the gross-overstating ones are explicitly excluded — so when two candidates in the same filing disagree materially, the larger is the total and the smaller is a component of it. That rule gets MetLife and General Mills right simultaneously, which no fixed ordering can.

The regression test keeps them in the same file, deliberately, because they pull in opposite directions.

What this cost, and what changed

Wrong values reached the paid API and the free sample. P&G FY2014 shipped at $29.4B against a true $83.1B in the public CC0 file — the one our own pricing page tells prospects to verify against.

All of it is corrected now: General Mills at $19.857B, P&G at $83.062B, Fifth Third at $8.708B, M&T at $9.279B, MetLife still right at $70.986B. Roughly 320 values across all seven concepts have since been checked against SEC's API with zero mismatches.

Three things we would tell anyone building on XBRL:

1. There is no universal top-line element. Any static tag priority is wrong for some filer. Compare magnitudes within the filing instead of trusting an order.
2. Detecting an ambiguity is not resolving it. We flagged the row and published it anyway. If your system knows something is uncertain, decide what to do about it.
3. Self-consistency is not correctness. Every gate we had compared the data to itself. Wrong numbers pass those effortlessly, because wrong numbers are usually well-formed. Something in your pipeline has to compare against the outside world.

We would rather publish this than have you find it. The free sample is 40 large caps, 3,280 point-in-time rows, CC0, no signup — github.com/christianpichichero-max/pit-fundamentals — and the methodology is public specifically so it can be attacked.

Not investment advice.