Fact Checking Research: The Pass Before It Goes Out

A fact check is a separate pass over a finished research document, run by somebody reading for correctness rather than for argument. Every figure goes back to the paper it came from, every calculation is redone, every attributed statement is matched to what was actually said, and every period, unit and scale is confirmed. The check cannot establish whether the view is right.

Three things are taken as settled. Research output comes in shapes, and a short document carries a different load from a long one. The research post mortemA later reading of a call that went wrong, run over the reasoning that was written down at the time rather than over how it turned out. examines a call once the outcome is known. A published sentence is therefore something somebody will have to trace back later. And how to read a profit ladder is settled in the accounting material. The pass itself is the subject: what it looks at, in what order, and the three traps it exists to catch, none of which a careful proofreader will find.

Where does the pass sit, and what makes it a different reading?

The feeling of having been careful is unreliable, so the pass is defined by position instead. The pass happens after the writing is finished and before the document goes out. Not during. Not as a habit of care while drafting. The pass is a discrete operation with a start, an end and an output. While it runs, the person doing it asks one question over and over: does this document say what its sources say? The question asks about correspondence between two objects sitting side by side. The question is not whether the argument holds together, whether the prose flows, or whether the conclusion is any good.

The two readings cannot be run at the same time by the same brain. The impossibility is the whole reason checking is a separate operation with its own name. Reading for argument means holding the case in mind and testing whether each sentence advances it. Reading for correspondence means emptying the mind of the case entirely. Each sentence then becomes a small independent claim to be matched against a paper. Attempting both yields neither. The moment the argument is being followed, context is already being supplied, and supplying context is exactly what makes an error invisible.

The same division is familiar from outside research altogether. A wedding invitation goes to the printer. Somebody in the household reads it for whether it sounds right, whether the elders are named in the correct order, whether the tone is warm enough. Somebody else takes the same card and reads only the date, the day of the week, the pin code and the spelling of the venue, against the actual booking receipt. The second person is not being pedantic. The second person is doing a different job, and the household that skips it is the household that prints four hundred cards with the wrong Tuesday.

What actually gets checked?

Five classes, written down so that nothing depends on remembering. Class one, figures back to the paper they came from, meaning the exchange filingThe copy of a result, an announcement or a shareholding statement that a listed company lodges with the exchange. A checker treats that copy as the original. or the annual report or the transcript, and not back to the analyst's own spreadsheet. The spreadsheet may be where the error entered. Class two, arithmetic recomputed rather than re-read: a wrong sum looks exactly like a right one on a second reading, and only doing it again catches it. Class three, attributed statements matched against what was actually said, word by word, including whether the person said it about the thing the sentence says they said it about. Class four, periods, units and scales confirmed on every number. Class five, names, dates and spellings held consistent from the first line of the report to the last.

The fifth class looks trivial and catches the errors that cost the most credibility. A reader who spots the entity misspelled early in the report stops trusting the arithmetic later in it that they cannot check for themselves. The inference is not fair, and it is completely reliable. The reader has no way of independently verifying most of what the analyst has written. One thing the reader can verify, instantly and without effort, is whether Nandivarman Paints Limited is spelled the same way twice. When that fails, the reader draws the obvious inference about everything else, and is not wrong to.

Why do periods cause more trouble than figures do?

Here is the subtlest of the five and the reason class four is written out separately rather than folded into class one. A figure can be entirely correct and still belong to a different length of time from the figure sitting next to it. Gross margin of 46.0 per cent is right. A gain of 3.0 points is right. Realisation per unit up about 7.5 per cent is right. Put in one sentence, the three of them may amount to a claim that nothing in the record supports, and no amount of checking the figures will reveal it.

The period mismatch survives every other kind of checking for one reason: there is nothing wrong with either number, so a checker verifying figures one at a time will tick both, and only a checker who asks what period each figure belongs to as a separate question ever catches it. The error does not live in a number. The error lives in the space between two numbers, and that space cannot be looked up in a filing. The relationship has to be established by asking what each number is a measurement of.

The household version is a rent argument. The landlord says the rent has gone up by two thousand rupees. The tenant says the maintenance has gone up by six hundred. Both true. But the landlord's two thousand is over three years and the tenant's six hundred is since April, and until somebody says so out loud, the two of them will argue about proportions for an hour without either being wrong about a single figure.

What does the pass find when every figure is already correct?

Take a real sentence from a draft on Sarvani Coatings Limited, an invented maker of decorative paints and industrial coatings whose published record runs through this material. The draft reads: Sarvani Coatings' gross margin rose 3.0 points because realisation per unit rose about 7.5 per cent against materials cost per unit up about 3.6 per cent. The sentence is a good one. The wording is compact, it names a cause, and every one of its three figures is going to verify. Run the pass on it exactly as a normal pass runs, one figure at a time, and watch what happens.

First figure. Gross margin was 43.0 per cent in year one and 46.0 per cent in year three. Recompute both rather than reading them off. Year one puts revenue of Rs 1,840 crore against a cost of materials of Rs 1,048 crore, leaving gross profit of Rs 792 crore. The division comes out at 43.0435 per cent. Year three puts Rs 1,111 crore over Rs 2,415 crore, or 46.0041 per cent. The record publishes those as 43.0 and 46.0, and the gain taken from the unrounded results is 2.9607 points, printed as 3.0. Tick. Second figure. Revenue moved Rs 2,120 crore to Rs 2,415 crore, a climb of 13.9151 per cent, set against volume growthHow much more physical output was sold, counted in litres and tins rather than in rupees, so a price rise cannot flatter it. of 6.0 per cent, so realisation per unitWhat one unit actually fetched, found by dividing a year's sales by that year's volume. It shifts when the price shifts and it also shifts when the mix of what got sold shifts. rose 7.4671 per cent, printed as 7.47 and quoted as about 7.5. Tick. Third figure. Materials moved Rs 1,187 crore to Rs 1,304 crore over that same year, up 9.8568 per cent and quoted in the draft as about 9.9, carried across the identical 6.0 per cent of volume, so materials cost per unit rose 3.6385 per cent, printed as 3.64 and quoted as about 3.6. Tick.

Three figures, three passes, three ticks. A checker running class one and class two has now finished and found nothing, and the sentence is wrong.

Now ask the period question. The 3.0 point gain spans two years, reaching back from year three all the way to year one. Both per unit figures belong to a single year, the one running from year two into year three. The 6.0 per cent is the volume growth for that year alone, and year one volume growth was never put into the record at any point. The sentence has explained a two year move with a one year decomposition. Worse than a mismatch of length, the two year move cannot be decomposed per unit in the first place, for exactly that reason: without year one volume there is no per unit arithmetic to do. The gain over the one year the per unit figures actually describe is 1.9947 points, printed as 2.0, from 44.0 per cent to 46.0 per cent.

So the corrected sentence reads: gross margin rose 2.0 points in year three, from 44.0 per cent to 46.0 per cent, because realisation per unit rose about 7.5 per cent against materials cost per unit up about 3.6 per cent. Notice what the correction did and did not touch: the two per unit figures do not move by a hair, and the only thing that changed is which margin move they were being asked to explain. The contrast is worth sitting with, and it is why the error is so hard to see. Nothing about the causal half of the sentence was ever wrong.

The corrected sentence contains a trap of its own. A flourish is tempting: that the cost of materials went from 56.0 per cent of revenue to 54.0 per cent, and that 56.0 times 1.036 over 1.075 gives 54.0 exactly. It does not. Carried on those printed ratios the sum lands on 53.9684, a shortfall of 0.0316 of a point. Carried on the unrounded ratios it lands on 53.9959, the year three share to the last decimal. The volume index appears in both the numerator and the denominator and cancels straight out. The cancellation makes it forced arithmetic rather than a check that could ever have failed, and a display figure is not an input: a rounded 1.036 fed into an identity produces a residual where the exact ratio ties perfectly.

What are the unit and scale traps, worked rather than warned about?

Three more errors survive proofreading, and they survive it for the same reason the period trap does. None of them produces a sentence that reads oddly. Each one produces a sentence that reads beautifully and means something other than what the writer intended.

The first is per cent against percentage point. Suppose the draft says gross margin rose 2.0 per cent from 44.0 per cent. Read literally, that is 2.0 per cent of 44.0, or 0.88 of a point, and it puts margin at 44.88 per cent. The writer almost certainly meant 2.0 percentage points, and the intended reading puts margin at 46.0 per cent. The two claims are 1.12 points apart, and they describe two different states of the company. A movement stated in the wrong unit is not a rounding problem or a stylistic slip, it is a different assertion about a different number.

The second is scale. Indian reporting runs on two units at once, and Rs 1,304 crore and Rs 1,304 lakh sit a hundred times apart while looking almost identical in print. In whole rupees the first is 13,04,00,00,000 and the second is 13,04,00,000. Put the smaller one against the published revenue of Rs 2,415 crore and the cost of materials becomes 0.54 per cent of sales rather than 54.0 per cent, a share that would make Sarvani Coatings the most profitable paint maker in recorded history. The scale error is enormous and completely silent. The digits are right, the grouping is right, and only the unit is wrong.

The third trap catches good analysts. Two figures are involved, both correct and both about cost. A share of revenue is not a per unit figure. Over year three the cost of materials fell from 56.0 per cent of revenue to 54.0 per cent, and over the same single year materials cost per unit of output rose 3.6385 per cent. Both are true. The two figures point in opposite directions because they answer different questions: the share is asking what proportion of each rupee of sales the input consumed, and the per unit figure is asking what one unit of output cost to make. A checker who reads a falling share of revenue as a falling input cost has passed a sentence that states the reverse of what happened, and the two figures that prove it are sitting side by side in the same report.

What can the pass never establish?

Over-reliance on checking is a real occupational hazard, and it produces a very particular kind of confident error. The limit deserves stating plainly. The pass cannot establish that the view is right. The pass cannot repair a broken thesisThe short testable claim a piece of research is built to defend, plus the handful of things that would have to be true for it to hold.. And the pass is completely blind to an assumption that is wrong but applied consistently. Consistency is precisely what a correspondence test rewards. Where an analyst has assumed that realisation will keep rising at 7.5 per cent for four more years, and has used that assumption faithfully everywhere, the pass will tick every line it touches.

A fully checked document can be completely mistaken, and that is not a weakness of checking but a description of what checking is: a test of correspondence, not a test of judgement. The two live on different axes. Movement along one of them says nothing about position on the other. Which is why a document that passes cleanly settles exactly one thing, that the arithmetic and the attributions are sound, and settles nothing whatever about whether the view is right.

Why can the author not run it on their own work?

Stated as a rule, the ban on self-checking sounds like a courtesy, and people skip courtesies when they are late. A mechanism holds it up instead. The mechanism is this: the writer's eye supplies the meaning the writer intended. The sentence as written is not what gets read; the sentence in the head is, and the two are usually the same. The one time they differ, the writer sails straight past it. The intention is still live in memory and it fills every gap the words leave, so the words never get tested at all.

Notice what that means. The filling-in is not a failure of care. Nor does it improve with a second coffee, a slower reading, or twenty years on the desk. A senior analyst reading their own draft has the same intention sitting in the same place doing the same filling-in, and if anything it is worse. Seniority brings speed and speed brings more filling-in. The checking pass is therefore a role rather than a task: it belongs to somebody who was not in the writer's head.

Where no second person exists, and on a small desk that is often the honest position, the pass is run after a delay long enough for the intention to fade, and the delay is stated in the record rather than assumed. Overnight is the usual minimum. A named delay is auditable and a vague one is not. A checker who writes "checked the following morning" has left behind something a reader can weigh. One who writes "checked" has left behind nothing.

What does the pass leave behind?

A record, and the record is what makes the whole operation auditable rather than merely virtuous. Four columns are enough: what was checked, against which paper, on what date, and what changed as a result. The last column matters more than it looks. A pass that changed nothing and a pass that never happened produce identical documents, and only the record separates them.

A check with no record cannot be told apart from a check nobody ran. The record is therefore the deliverable and the ticks are only the process. The check record has a second use as well, and it reaches back to the research post mortem. When a call goes wrong and somebody sits down a year later to work out why, the first question is whether the research was wrong or whether the writing was. A wrong research view and a wrong sentence need completely different remedies. A check record answers that question in about a minute. Without one, the examination has to reconstruct what was known at the time from memory, and memory is exactly the faculty that has spent the year quietly rearranging itself around the outcome.