Outliers: Error or Signal, and How to Decide Which It Is

Somebody hands over a small file, an eye runs down one column, and one number jumps out. The instinct that follows is almost universal and almost always wrong: that figure looks mad, so it must be a mistake, so pull it out before it spoils anything. The instinct is worth sitting with for a moment. One thing is known. The figure is unlike its neighbours. Nothing whatever is known about how it got there. The distance between those two facts is what everything below turns on.

What does calling a figure an outlier actually establish?

An outlier is a figure that sits far from the others in the same column. The definition stops there, and it is worth noticing how little it carries. An outlier is a statement about the column, not a statement about the world. The word says a figure is unusual against its neighbours. A real event and a finger slipping on a keyboard both produce a figure that is unusual against its neighbours, and they produce it in exactly the same way. The word cannot tell the two apart.

Take it out of finance for a second. Six months of household electricity bills sit around Rs 1,200/- each and the fourth one is four times the rest. Two things could have happened. Eleven relatives came to stay for three weeks and two extra coolers ran day and night. The consumption was real and so was the bill. Or the meter reader copied the wrong dial and the household used what it always uses. Now here is the part that matters: the bill is the same piece of paper in both worlds. The bar on the chart is exactly the same height. The six numbers are identical in both worlds, so nothing done with them will show which of the two happened.

Which two cells in this record read the same number?

Now the record itself. In month 3, NB-07 Sundari Chaat's takings cell reads Rs 4,80,000/-. In the same month, NB-08 Peeli Mithai's takings cell also reads Rs 4,80,000/-. Every other cell in that column that carries a usable number sits between Rs 0/- and Rs 55,000/-, so both of these stand roughly nine times clear of everything around them. One of the two is exactly what it says. The other is a slip.

NB-07 Sundari Chaat really took Rs 4,80,000/- that month. The stall catered a wedding orderA whole wedding catered on one booking, billed once and paid once. For a stall that otherwise sells by the plate, one of these can dwarf everything around it., one booking, paid in one go, and the stall's ordinary months are Rs 40,000/-, Rs 41,000/- and Rs 42,000/-. NB-08 Peeli Mithai took Rs 48,000/- that month, and the cell carries a trailing zeroA nought that should not be there, sitting at the end of a figure. A finger holding a key a fraction too long turns a number into ten times itself, and the result still looks like money.: one nought too many at the end, ten times the true figure. Its ordinary months are Rs 42,000/-, Rs 45,000/- and Rs 47,000/-.

Both cells hold the number Rs 4,80,000/-, and that is not a near miss or a coincidence worth admiring, it is the fact everything below turns on. Printed side by side, there is nothing to look at. Same figure, same column, same month, same record. Whatever the data is hoped to reveal, it has already given up everything it holds.

Can a flat rupee cutoff separate the two cells?

The first rule anybody reaches for is a line: pick a rupee figure, call everything above it suspect, and get on with the day. Here is what that does. A line at Rs 1,00,000/- leaves both cells above it. A line at Rs 3,00,000/- leaves both above it. Neither cell exceeds itself, so a line at Rs 4,80,000/- leaves neither above it. Slid anywhere between Rs 10,000/- and Rs 6,00,000/-, in steps of Rs 1,000/-, the line takes 591 different settings. Not one of them flags one cell and clears the other.

The failure is arithmetic, not a finding about this record, and the difference matters more than the result. A flat cutoff reads one thing: the number in the cell. The two cells hold the same number. A rule that reads only the number therefore has to reach the same verdict on both, every time, on every setting, forever. Nobody has merely failed to find the clever cutoff yet. There is no cutoff, and there could not be, and the same would be true of any two equal cells in any record anywhere. Nothing about the Neelbagh market is doing the work here.

Can a ratio rule separate them, and which way round?

So read something other than the cell. A ratio rule says: flag a figure that is more than k times the same stall's smallest other month. Now the rule looks at two numbers rather than one, and the second number differs between the two stalls. NB-07 Sundari Chaat's smallest other month is Rs 40,000/-. NB-08 Peeli Mithai's is Rs 42,000/-. Two different bases, one shared cell figure, and suddenly the rule has something to bite on. So this kind of rule can separate them, and it is worth being precise about the fact rather than waving it away.

Take every setting of k from 2.00 to 20.00 in hundredths. There are 1,801 of them. Sort the answers. From 2.00 to 11.42, both cells are flagged. From 12.00 to 20.00, neither is. In between sits a band running from 11.43 to 11.99, and inside that band exactly one cell is flagged. The flagged cell is the real month. The wedding order at NB-07 Sundari Chaat gets picked up as suspect, and the typing error at NB-08 Peeli Mithai walks away clean. Not one of the 1,801 settings does it the other way round.

Look at why. The reason is almost cruel. Rs 4,80,000/- is twelve times NB-07's Rs 40,000/-, and it is about eleven and four sevenths times NB-08's Rs 42,000/-. The error's stall had slightly better ordinary months, so the error is a slightly smaller multiple, so it crosses out of the flagged region first. The rule is not confused. The rule is working perfectly, on a signal that happens to point the wrong way.

So what actually decides which cell is the error?

Neither cell is judged from the takings column at all. Both are judged on paper that sits outside the record, and here is what that paper says.

For NB-07 Sundari Chaat, the market office day bookA dated notebook in ordinary prose, filled in on the day rather than at month end. What makes it worth reading is the timing: nobody writing in it yet knows which figure will later be questioned. carries one sentence, written on the day, recording that the stall catered a wedding order and naming the day it was paid. The stall's own copy of the booking agrees with it. One sentence, in the right place, written before anybody had a reason to defend a number.

For NB-08 Peeli Mithai, the stall's return slipThe month end declaration a trader writes out and brings in. It is the paper the office types from, so where the two disagree, remember which one came first. reads Rs 48,000/-. One disagreement between two pieces of paper proves nothing on its own, so there is a second check: the office's own arithmetic on that month's stall feesWhat the market office charges each pitch for the month. It is worked out quite separately from anything sold, which is what makes it an independent line to test a takings figure against. also agrees with Rs 48,000/-. Two independent trails, one figure, and the cell in the record is the odd one out.

Provenance decides, and the shape of the column never does. Read that as a working rule rather than a slogan. The column can raise the question, and raising it is genuinely useful. Rules are worth running for that alone. But every answer comes from somewhere the column cannot reach: a notebook, a slip, a booking, a fee calculation, a phone call to somebody who remembers. The two cells in this record differ in where each came from, and in nothing else at all.

What does each decision cost?

Every one of these decisions moves a published number, so the honest way to weigh them is to compute the move rather than describe it. A figure whose denominator is left vague can be argued into any shape at all, so the convention behind the headline average is worth restating before anything is done to it.

Before either cell is touched, those 29 cells sum to Rs 20,03,700/- and the headline is Rs 69,093.10/-. Correct NB-08 Peeli Mithai's cell from Rs 4,80,000/- to the Rs 48,000/- its return slip shows, and Rs 4,32,000/- comes out of the sum, leaving Rs 15,71,700/- and a headline of Rs 54,196.55/-. The headline moves down by Rs 14,896.55/-, and the move had to be made: the record was carrying a figure the stall never took.

Suppose NB-07 Sundari Chaat is then treated the same way, its real month pulled back to the Rs 40,000/- its quiet months suggest. The sum falls to Rs 11,31,700/- and the headline to Rs 39,024.14/-, a further move down of Rs 15,172.41/-. The correction that should never have been made is the larger of the two moves. Not by much, and the closeness is itself the lesson: on this record, the reflex of pulling in whatever looks extreme costs slightly more than the genuine error it was supposed to protect against. Pulling in the extreme figure feels like the careful choice. The careful-looking choice is the more expensive one.

Is every strange looking figure an outlier?

Go back to the same column and look at NB-02 Chandan Tea's month 4 cell. The month 4 cell reads Rs 99,999/-. Against that stall's other three months of Rs 31,000/-, Rs 33,000/- and Rs 32,000/-, it looks like the fourth strange figure in the record, and it is not one of them. Rs 99,999/- is the office code for no return received. The stall reported nothing that month and the office wrote a fixed value into the cell so it would not sit empty. No money changed hands at Rs 99,999/-, and no money changed hands at all.

Now run the ratio rule over it. Five times NB-02 Chandan Tea's smallest other month is five times Rs 31,000/-, or Rs 1,55,000/-. The cell reads Rs 99,999/-. Nothing flags it, nothing comes close to flagging it, and nothing would flag it at any setting of k above about three and a quarter. The one cell in this stall's row that holds no money at all is the one no hunt for extremes will ever find. A great deal of effort goes into chasing the biggest numbers in a column, and the most misleading cell here is a middling one. The point is worth holding on to.

What gets written down when a figure is left in?

Here is the part that decides whether any of the work above survives the person who did it, and it is also where a lender, an analyst, a valuer or anybody who reads records for a living actually earns the fee. Suppose everything has been done right: the Rs 4,80,000/- at NB-07 Sundari Chaat was flagged, the day book was found, the wedding order was checked and found real, and the figure was left where it was. Six months later somebody else opens the file, runs an eye down the column, sees the same figure, and has the same instinct. If nothing was written down, they will pull it out, and that careful decision will have lasted exactly as long as the memory of it.

So three lines go beside the cell. The first names the cell and its figure. The second names the document that supports it, specifically enough that somebody else can go and find the same piece of paper. The third says what the figure would have become had it been corrected, and what that would have done to the headline number. The third line does the work. A judgement becomes a comparison the next reader can check, rather than an opinion they have to trust. On this record it reads: pulled to Rs 40,000/-, the headline falls from Rs 54,196.55/- to Rs 39,024.14/-. Anybody who wants to overturn the decision now has to argue with a number.

The same three lines work anywhere a figure survives a review. A lender leaving one enormous month in a small trader's turnover statement writes down the invoice that backs it and what the average would have been without it. An analyst keeping a one off receipt in an operating line writes down the contract and the margin the other way. In each case the note is not paperwork for its own sake. The note is all that stands between a verified figure and the next person's reflex.