Source: Wang (2025) — “F®iction in Machines: Accounting Hallucinations of Large Language Models”
This is one of those risks that sounds abstract until you imagine explaining it after the fact. Then it suddenly becomes very concrete, very expensive, and very difficult to hide behind a slide deck.
The hallucination problem in large language models is well-known in general terms. This paper maps it specifically onto accounting information — querying LLMs about financial statements for SEC-reporting firms — and the results are more nuanced and more concerning than the general framing suggests. The researcher identifies two distinct types of accounting hallucination. The first is deviation: the model returns a number that exists in the financial statements but gets it wrong — the revenue figure for 2021 is reported incorrectly. The second is fabrication: the model returns a number for a financial item or period that doesn’t exist at all in the reported financials — inventing a figure rather than retrieving or misreporting a real one. The counterintuitive finding: more accounting information in the public domain leads to fewer deviations but more fabrications. When a firm has more historical filings and more media coverage, LLMs are less likely to get the real numbers wrong — they’ve seen them more often in training. But the same familiarity seems to enable more confident fabrication of nonexistent items, because the model has enough context about the firm to construct plausible-sounding but fictional numbers.
In plain English, that is why the result matters beyond the chart. It changes where people should look, what they should question, and which comfortable assumption probably needs to be retired.
My takeaway: the danger is rarely the dramatic thing in the headline. It is the quiet gap between knowing a risk exists and assigning someone to do something about it. Very unglamorous. Very important.