Data Revisions: Why the First Print Is Rarely the Final One

A statistical office publishes early because a late figure helps nobody, and early means incomplete. Later prints add returns that had not arrived and methods that had not yet been applied. So one release carries several correct values at different vintages, and the vintage is part of the figure rather than a footnote sitting underneath it.

Two measures sit underneath everything below, and each is set out under its own subject. One is the output measure, the total of what an economy produced in a period, together with the growth rate that sets one period of it beside another. The other is the price index, the release a reader meets most often after that one. A third property sits on top of both and applies to every release either of them will ever produce: the figure in front of the reader was assembled at a particular moment, and a different moment would have assembled it differently.

Take the units first. This subject carries several kinds of number at once, and mixing them is the easiest mistake available. A growth reading is a per cent. The gap between two growth readings is a point. So when the Sankhya growth seriesOne measured quantity recorded again and again over successive periods, so that its values line up in order and can be compared with each other. reads 6.20 per cent at one moment and 6.80 per cent at another, it has moved 0.60 points. Calling that movement 0.60 per cent would be wrong. Every figure below obeys that split, and every figure below also carries the moment it belongs to.

Why does a published figure change after it is published?

A wedding hall shows this problem happening rather than merely described. Somebody has to tell the caterer how many people actually ate. At ten in the evening a number can be given, and it will be a real number, arrived at by counting plates going out of the kitchen. Next morning, with the trays counted, the unopened packets returned and the two late tables remembered, a better number can be given.

Now ask the only question that matters here. Was the ten o'clock number wrong? The ten o'clock number was the best count available from what had come back to the kitchen by then. The caterer needed a number that night, so it was also the only number that could possibly have been used. And it was going to move, and everybody in the hall knew it was going to move.

A statistical office is standing in exactly that hall. A figure covering a period has to be published before every statistical returnA form or filing that a business, a farm or a government office sends in, reporting what it produced, sold or spent during a period. The building block a compiled figure is added up from. covering that period has arrived. The returns do not all arrive at once, and some of them arrive very late indeed. A large business with an accounts department files early. A small workshop with three people files when somebody has time. Activity in a village that nobody has surveyed yet does not file at all and has to be estimated from something else. So at the moment of publication the office holds part of what it will eventually hold, and it knows which part is missing.

A statistical office is choosing between timely and complete, it cannot have both, and publishing early is a decision about usefulness rather than a lapse. Put the choice the other way round to see how forced it is. An office that refused to publish until every return was in would publish a figure describing a period that had finished long ago, and nobody deciding anything would still be waiting for it. The value of a figure decays. The office trades some accuracy for the fact that anyone still cares.

The missing part is not a secret. An office publishes what its coverageThe share of the activity that a figure is meant to describe which has actually reported by the time that figure is compiled. Low coverage at first publication is normal and is stated by the office. was at first publication, states which sources were still outstanding, and says that the figure will be prepared again. The reader who treats the first print as a final answer has ignored a warning the office printed itself.

What is a vintage, and why must every figure carry one?

A vintage is the moment a figure was assembled, and it belongs to the figure the way a unit belongs to a measurement. The Republic of Sankhya, invented for these notes, publishes growth for one period three times over. At first print it reads 6.20 per cent. At the first revision it reads 6.50 per cent. At a later vintage, once the slow returns and the fuller methods have been applied, it reads 6.80 per cent. Same period, same economy, same office, three published numbers.

All three of those readings are correct as of their vintage, a Sankhya growth figure quoted without its vintage is not a figure at all, and two readers holding different vintages will disagree about something they both think is a single fact. The disagreement between two readers is the practical damage, and it is worth sitting with. Neither reader is careless. One wrote 6.20 per cent into a note when 6.20 per cent was what Sankhya had published. The other wrote 6.80 per cent later, from the same release. Put them in a room and they will argue about the economy when the entire distance between them is a label neither of them recorded.

The shape of the movement in the Sankhya growth series matters more in its components than in its total. From the first print at 6.20 per cent to the first revision at 6.50 per cent is 0.30 points. From the first revision at 6.50 per cent to the later vintage at 6.80 per cent is another 0.30 points. The two legs add to 0.60 points, the whole distance from the first print to the later vintage. Each leg is stated separately so the total can be added up rather than taken on trust.

There is a second reason the vintage has to travel with the figure, and it is duller than the disagreement but costs more over time. A figure filed into a stored series without its vintage cannot be compared with anything that arrives afterwards. The figure sits in the column looking exactly like its neighbours, and the reader who comes back in six months has no way to tell that one cell was compiled from partial returns and the next from full ones. The series looks consistent and is not, and nothing on the surface will ever reveal it.

Does a revision mean the first print was wrong?

No, and the word is doing the damage. Revision suggests an error and describes an arrival. A revision brings information: the returns that had not come in, the sources that report on a slower cycle, the method that could only be applied once a fuller picture existed. Nothing about the first print was mistaken. The first print was the correct summary of what had reached the office, published with a label saying so.

The household version is immediate. Suppose a month's spending is totalled on the evening the month ends, from the cash, the transfers and the bills that are remembered. Two weeks later a card statement arrives with four transactions that had been forgotten and a subscription nobody knew had renewed, so the total is done again and comes out larger. The first total was not a lie. The evening total was the correct total of what could be seen.

Now the exception, stated plainly. Hiding it is what makes ordinary revisions look like failures. A genuine mistake is also possible. An office can transpose a figure, misclassify a group of returns, or discover that the software applied the wrong seasonal adjustmentArithmetic that removes the part of a movement which happens in the same way in the same part of every cycle, so that what is left can be compared across periods. to part of a series. When that happens the office publishes it as what it is: a correction, usually with a note describing what went wrong and which periods it touched.

A revision and a correction both produce a new number where an old number stood, so on the surface they look the same. Telling them apart is entirely a matter of reading what the office said alongside the figure. A revision arrives on a stated cycle of estimation and the office says in advance that it is coming. A correction arrives when something has been found, and the office says that too. Collapsing the two categories into one turns a routine, announced, expected movement into evidence that the numbers cannot be trusted, and the arithmetic never supported that conclusion.

The cost of that confusion is not abstract. A reader who believes every revision is an admission of error will discount every first print, and a first print is the most timely information available. The same reader has no surprise left for a real correction, having spent it on the routine ones.

How big is a revision next to the surprise everyone reacts to?

A size only means something against another size, so set the two movements side by side. When Sankhya published growth at 6.20 per cent, the consensusThe average of the forecasts published by professional forecasters before a release, gathered so that the release can be compared with what was expected of it. assembled beforehand stood at 6.40 per cent. The first print therefore came in 0.20 points below what had been expected. The gap of 0.20 points is what everybody had in front of them on the morning of publication, and every note written that day was written about it.

By the later vintage the same release read 6.80 per cent. The whole distance travelled from first print to later vintage is 0.60 points. Divide one by the other: 0.60 divided by 0.20 is exactly three.

The Sankhya number moved three times as far as the surprise anyone reacted to. Read that again with the direction stripped out. The direction is not the point. Something small was measured against an expectation and discussed at length. Something three times larger happened to the same figure afterwards, quietly, on a stated cycle of estimation, and was discussed by almost nobody. The proportions of the attention were the reverse of the proportions of the movement.

How the surprise itself is measured, why it is measured against a consensus that nobody goes back and revises, and what a reader should do with the gap on the morning it appears, are a separate subject, set out under data surprise. A reader who takes a first print as final and reacts only to its distance from the consensus is calibrating against the smaller of the two movements that release will make. The larger one is still to come and is not in the reaction at all.

What does one Sankhya release look like across three vintages?

Here is the whole release, laid out so that every step can be added up by hand. The left half is the reading and the movement. The right half is what a reader would have concluded about the same release at each vintage, measured against the one thing that never moves, the consensus of 6.40 per cent assembled before the first print.

Two things in that table are worth stopping on. The first is the movement column: 0.30 points and 0.30 points add to the 0.60 points on the total row, so nothing has been asserted that cannot be recomputed. The second is the last column, and it is stranger than it looks. The consensus never moved. Nobody went back and revised it. Yet the same release was a miss of 0.20 points, then a beat of 0.10 points, then a beat of 0.40 points against that unchanged expectation. The verdict flipped from short of expectations to ahead of them, and doubled in size on the way, purely because the figure kept arriving.

A second Sankhya series, revising the other way

One series rising through its vintages proves nothing about direction, and a reader who has only seen the growth series will quietly conclude that revisions go up. So take a second invented Sankhya series, construction volume growth for the same period, and follow it the same way. The construction series prints at 4.80 per cent. The first revision brings it to 4.40 per cent, a fall of 0.40 points. The later vintage brings it to 4.10 per cent, a further fall of 0.30 points. The two legs add to 0.70 points, and the series ends lower than it started.

Put the two Sankhya series together and revisions turn out to have no reliable direction. Only their size can be anticipated. Both series were compiled by the same office, for the same period, on the same cycle of estimation, from returns arriving in the same way. One ended 0.60 points above its first print and the other 0.70 points below. Anyone who had learned from the growth series that a first print understates would have been wrong about the construction series, and would have been wrong in a way that felt like experience.

Which figures revise most, and why?

The size of a revision is not a mystery and it is not random across releases. Revision size follows from how the figure was put together, and how a figure was put together is knowable before it is published.

Three kinds of construction produce wide movement. The first is a figure built from returns that arrive slowly, where a large share of the source material is simply not in hand at first publication. The second is a figure covering activity that is hard to observe: work done by very small units, activity that is not registered anywhere, output that has to be estimated from a related indicator until something better arrives. The third is a figure assembled from many separate sources. A total built from a dozen inputs inherits the movement of all twelve, and the national accountsThe set of accounts that adds a country's production, income and spending into one consistent picture, so that the pieces reconcile with each other rather than merely sitting side by side. are the clearest instance of exactly that.

Two kinds produce narrow movement, and the reason is the mirror image. A figure whose respondents answer at the time about what they are doing at the time has almost nothing left to arrive: a survey based diffusion indexA reading built by counting how many respondents report an improvement rather than by measuring how much output changed, so it records breadth and not size. The diffusion index has its own treatment. is complete when the responses are in. A figure gathered directly on a fixed collection round, with an enumeratorThe person who physically collects prices or answers on the ground for a statistical office, at a shop, a market or a household, rather than waiting for a form to be filed. writing down what a shop is charging, has already got its raw material at the moment of compilation.

Revision size is a property of how a figure is built, so it can be anticipated in advance even though the direction cannot. Anticipating size is what a reader can actually use. Before a release lands, it is already known whether the number about to be read is the kind that stays roughly still or the kind that travels, and it is known from the construction of the series rather than from any view about the economy.

Put it in a street. Three shopkeepers are asked what they sold last month. The one with a billing machine answers the same day. The one with a paper register answers by the weekend. The one who has to ask his brother about the deliveries answers next week, and his answer changes the total. Knowing which of the three the total is waiting on gives how much it can still move, and gives it without any opinion at all about whether the market is busy.

What does knowing this change about reading a first print?

Knowing this does not make a first print useless, and any reading that ends there has taken the wrong lesson. A first print is the only figure that exists at the moment it exists, it carries real information, and a reader who waits for a final figure before forming any view has chosen to know nothing for a long time.

The first print stops being a point and becomes a point with a width. For the Sankhya growth series the width is known from the last time the series was compiled again: 0.60 points of travel. So a first print at 6.20 per cent sits in the company of a later vintage that could plausibly read 6.80 per cent, or 5.60 per cent, and the reader who holds both edges in mind is reading the same figure more accurately than the reader holding only the middle.

A reader who treats a first print as final has thrown away information they already had, namely how far that particular series usually travels. Note what that width is not. The width is not a measured range of outcomes and not a calculated interval of any kind. The width is one distance only: how far this invented series moved on the last occasion it was compiled again, drawn on both sides because the direction is not knowable in advance.

The tailor's estimate is the honest version of this. Asked what a stitched jacket will cost, a good tailor gives a figure and adds that it moves with the lining and the buttons. The tailor's answer is not vague. A figure and its width is strictly more than a figure alone, and a customer handed only the middle would plan differently.

How does an analyst actually handle a vintage?

By writing it down beside the figure, every time, as a matter of routine rather than judgement. An analyst who stores a growth reading of 6.20 per cent stores three things in the same row: the reading, the period it describes, and the vintage it was compiled at. The third one takes two seconds and is the one that gets skipped.

A figure filed without its vintage cannot be compared with anything that arrives later, and it will silently corrupt every series it is filed into. Silently is the word that matters. Nothing breaks. No cell turns red. The column of readings looks orderly, one value per period, and somewhere inside it two cells were compiled at different stages and are not the same kind of number. A year later somebody charts that column and describes a movement that is partly the economy and partly the office.

The habit has three parts and none of them is difficult. Record the vintage with the value. When comparing two periods, check that both values came from the same stage of estimation before drawing any difference between them. And when only one vintage is available for one of the two periods, say so in the note rather than presenting the comparison as clean. A stated limitation is information. An unstated one is a trap for whoever reads the note next.

A lender assessing a borrower does the same thing with a different word for it. The accounts filed before the audit and the accounts filed after are both real, both signed, and not interchangeable, and a credit file that mixes them has a comparison in it that nobody can unpick later. The discipline is identical and the reason is identical.