The System Was Never Broken. The Reporting Was.

A company I work with had been running the same weekly pipeline report for two years. Every Monday it produced a win rate, a forecast, and a list of deals at risk. The number moved around in ways nobody could explain, so people stopped arguing with it and started working around it. By the time I saw the report, three different people were keeping private spreadsheets, and each of those spreadsheets disagreed with the other two.

Nothing in that system was broken in the way the word usually means. The database held the records. The automations fired. The report ran on schedule and delivered to the right inboxes. What had failed was narrower and more expensive. The numbers no longer meant what the people reading them believed they meant.

Two causes, both ordinary

The first was a categorization field left blank on a subset of records. A deal without that field was excluded from the report entirely. Not flagged and not zeroed. Excluded. The report was accurate about the deals it could see and silent about the ones it could not, and silence reads exactly like absence. Roughly one deal in six had gone missing from a forecast the company used to plan hiring.

The second was a definition problem. The report summed contract value and labelled the column revenue. A twelve month retainer signed in August contributed its full annual value to an August figure that leadership read as money in the building. The gap between the two numbers was large enough to change decisions, and it had been sitting in plain view on a slide every week for two years without a single person catching it, because the column had a name and everyone assumed the name was accurate.

Neither of those is a software failure. Both are governance failures, and they share a structure worth naming. A reported figure with no written definition behind it will drift toward whatever the reader already assumes, and the drift stays invisible because the number itself never stops looking correct.

Why this failure mode survives for years

This is the failure mode I see most often and the one companies are least equipped to detect. Bad data announces itself. Someone opens a record, sees a mangled name, and files a complaint. A broken automation announces itself when the thing that was supposed to happen does not happen. Reporting that has quietly diverged from reality announces nothing at all. It keeps producing numbers on schedule, the numbers keep being read, the decisions keep being made, and the only symptom is a slow loss of confidence that people express by building private spreadsheets, never by reporting a fault.

Reconcile one figure against the bank

The check is unglamorous. Take one reported figure, the most important one, and reconcile it against the bank. Not against another report. Against money that actually arrived. Where the two disagree, the difference is either a definition nobody wrote down or a filter nobody knew was applied. In every engagement where I have run this reconciliation, the difference has been material, and in most of them nobody inside the company could account for it on the first attempt.

Then write the definition down. What the figure counts, what it excludes, in what currency, over what period, sourced from which system. A metric without a written definition is a convention held in the heads of whoever built it, and conventions held in heads do not survive a departure, a reorganization, or a new hire who reads the column label and believes it.

What the fix cost

The fix at that company took under a day. A backfill on the categorization field, a rename of the column to what it was actually measuring, and a second report built alongside it for collected revenue so the two figures could stop being confused for each other. The private spreadsheets went away within a month, which is the only measure of a reporting fix that means anything. People do not maintain shadow systems when the real one can be trusted.

What took longer was persuading the company that a two year old report had been wrong the entire time. There is real institutional resistance to that finding, because accepting it means accepting that decisions were made on a figure that did not hold. The resistance is understandable and it is also the reason these failures persist for years. A system nobody trusts gets replaced. A system everybody trusts and should not is the more expensive condition, and it is invisible from the inside.

The question worth asking about any number your company runs on goes past whether the report works. Ask whether anyone can say, without checking, what the number counts and what it leaves out. When the answer takes longer than a sentence, the reporting has already drifted, and the only question remaining is how far.