Economic Indicators: What Each Release Actually Measures

An economic indicator is a number built to stand for something larger than itself. The useful fact about any one of them is not its level but its construction: what it counts, whom it asks, what share of the thing it actually covers, and how it weights the parts. A reader who can answer those four can use it.

Every release is the far end of a procedure. Somebody decided what would be counted. Somebody went and collected it from a stated set of respondents. Somebody decided how much of the thing they were going to try to reach, and how the parts would be added together. The figure that arrives on the screen is the last step of that procedure and shows none of the earlier ones. Reading an indicator well means reading the procedure. The procedure is published, and almost nobody opens it.

Why is an economic indicator a construction rather than an observation?

Start with a household. The arithmetic there is small enough to hold in the head. Two people in one house each keep a record of what the month cost. The first totals the bills that were actually paid inside the month. The second totals what was actually used inside the month, counting the electricity consumed in the last week even though the bill arrives later. Both records are honest. Both are correct arithmetic. The two records will not match, and they will not match for a reason that has nothing to do with either person making a mistake. The two records counted different things.

Now enlarge that to a street. Ten shops sit in a row. Walking down it and asking each shopkeeper whether trade was better than last month gives a count of how many say yes. Walking down it again and adding up what the ten actually sold gives a second figure. The two walks give two numbers about the same street in the same month. If seven small shops did slightly better and the one large shop did badly, the first number says the street improved and the second says the street shrank. Neither walk was careless. The first counted shops and the second counted sales, and the street is entitled to have both things be true at once.

The two walks carry the whole idea. Everything after them is detail. An indicator is not a thing somebody found lying in the economy. An indicator is a thing somebody built, out of four decisions, and the number is the output of those decisions rather than a sighting of the world. Any aggregateA single total for the whole economy, or for a large slice of it, arrived at by adding up parts. Output and the price level are the usual examples. ever published is like this, and so is every narrower measure.

Two indicators of the same thing were built differently, so both can be correct and still disagree, and that is the single most useful thing to know about any of them. A reader who holds that idea is protected from a whole category of confusion. When two releases point opposite ways, the question stops being which one is lying and becomes which choice the two made differently.

Which four questions turn an indicator into something usable?

There are exactly four. The set is short enough to run through in under a minute and complete enough that nothing important falls outside it, so it is worth learning whole.

What does it count? Not the vague thing it is about, but what physically gets tallied. Tonnes leaving a plant is one answer. The price on a shelf is another. A person saying they worked last week is a third. A manager saying conditions improved is a fourth. Two indicators that are both described as measures of activity can be counting completely different objects, and the description will not say so.

Whom does it ask, or what does it measure? Some indicators go to a list of units and require a return from each. Some go to a sample and scale the answers up. The list they draw from has a name, the sampling frameThe list of units a survey draws its respondents from. If a unit is not on the list it can never be selected, however large it is., and anything absent from that list is invisible to the indicator no matter how big it is. Some indicators do not ask anybody: they read prices off shelves or read tonnages off meters.

What share of the thing does it cover? Coverage is the question that gets skipped, and coverage decides whether a number can be quoted at all. An indicator built on a fifth of something is a perfectly good indicator of that fifth. The indicator becomes wrong only at the moment somebody quotes it as standing for the whole. Coverage is usually stated as a share, and it is usually stated plainly in the method note.

How is it weighted? Many parts have to be added together, and adding them means deciding whose answer counts for how much. One respondent one vote is a decision. Weighting by size is a different decision. Both are defensible, and both produce different numbers from identical raw returns. Most disagreements between indicators run on exactly that mechanism. One more thing usually sits beside the weights in the method note: the reference windowThe stretch of time a question is about. Asked whether they worked, a person answers differently for last week and for the last year, so the window is part of the answer. each answer refers to.

A reader who can answer those four questions about an indicator can use it, and a reader who knows only its level cannot. The level on its own supports no comparison at all. A bare level cannot be set beside another indicator, beside another country, or beside its own past, until what it counts and how much of the thing it reaches are both known.

What is the National Statistical Office, and what does compiling official statistics involve?

The National Statistical Office is the government body that compiles India's official statistics. The office sits inside the Ministry of Statistics and Programme Implementation, and its work is the unglamorous, enormous business of turning returns from thousands of separate respondents into a small number of published series that everybody can use. The office is a compiling body rather than a forecasting body and rather than a policy body: it does not decide what the numbers should be and it does not say what they mean.

Compiling official statistics involves three things done over and over. The first is collection: returns come in from producing units, from sellers, from offices and from sampled households, in whatever untidy state respondents send them. The second is method: one consistent set of definitions, one consistent decision about coverage and one consistent set of weights are applied to every return that arrives. This period can then be set beside the last period without the definitions having quietly moved underneath. The third is publication on a stated basis: the figure goes out together with a statement of what basis it went out on, including which base yearThe period an index is expressed against, usually set at one hundred. Everything the index says is a comparison with that period rather than an absolute quantity. an index is expressed against.

Publication on a stated basis separates an official statistic from a number somebody worked out, and it is the part worth sitting with. When a compiling body publishes, it publishes twice: once as a figure and once as a description of how the figure was made. The description is the boring document nobody downloads, and it is the entire reason the figure can be used by a person who was not in the room.

A statistical office publishes a method as well as a number, and the method is the part that makes the number usable by somebody who did not collect it. Strip the method away and what is left is a digit with no coverage, no weights and no definitions attached. Such a digit cannot be compared with anything, including itself a year earlier.

What are the four kinds of evidence an indicator can be built from?

Indicators are usually presented as a long list. A list invites reading straight down it, comparing numbers that were never comparable. Sorting works better. There are four kinds, and once an indicator is placed in a kind, most of what it can and cannot do follows.

Volumes. These count things: tonnes, units, kilowatt hours, passenger journeys. Volumes produce an index of quantity, and their respondents are producing units. A volume measure has the great virtue of being about real stuff rather than about opinion, and the great limitation of only reaching the stuff somebody thought to count.

Prices. These count what a fixed list of items costs. The fixed list is the basketThe fixed list of items a price index prices, held still deliberately so that a change in the index reflects prices moving rather than the list changing., and holding it still is the point: if the list changed every period a price movement could not be told from a change of contents. A price indicator produces an index of price, and its respondents are sellers.

Intentions and assessments. These count respondents rather than quantities. A survey asks a panel whether conditions are better or worse than last time and reports how many said better, so what comes out is a share of respondents. The people asked are usually the ones who take the decisions. A purchasing managerThe person in a company who decides what it buys and in what quantity, which makes them an early witness to whether the company is expanding or contracting. is therefore a favourite respondent for this kind: they know about next month before the tonnage does.

Counts of people. These count persons, jobs or households. The unit is a number of persons, or a share of persons, and the respondents are households answering directly about themselves. Both the reference window and the definition of work decide who is inside the count, so both matter more here than anything else.

Volumes, prices, intentions and counts are four different kinds of evidence, and comparing across the kinds without saying so produces most bad macro reading. A share of respondents and an index of quantity do not even have the same units, so the two cannot be set beside each other and treated as two readings of one thing. Each of those four measures is taught separately; what matters here is placing an unfamiliar release in the right column before reading it.

What do four Sankhya indicators look like once the four questions are answered?

The Republic of Sankhya, an invented country, has four indicators set out below with the four questions answered for each. A row read across gives the whole construction of one indicator. The coverage column read downward shows that no two of them are measured against the same base, and differing bases are what make most careless comparisons careless.

The parts are printed beside each total, so each total can be added up rather than taken on trust. A coverage share can be quoted on its own. A level cannot.

Four things in that table are worth pointing at. First, no level appears for any of the four: a level invites quoting before anybody has looked up the coverage share. Second, the four coverage shares are measured against four different bases, so 22.00 per cent of output and 92.00 per cent of household spending and 68.00 per cent of the population are not three readings on one scale and cannot be ranked against each other. Third, every share is printed with its parts, so 45.00 and 7.00 and 10.00 and 9.00 and 6.00 and 5.00 and 10.00 can be added to 92.00 by hand and the unpriced 8.00 recovered by subtraction. Fourth, one of the two built on the industrial register takes every unit and the other takes a panel of a hundred holding 55.00 per cent of industrial output, so the same register yields different coverage, 22.00 per cent against 12.10 per cent.

Why do two indicators of one thing disagree?

Take that last pair and make them disagree on purpose. Four Sankhya industrial units are all that is needed. Each has a share of Sankhya output and each reports a change in its own volume. Nothing else is happening: no revision, no seasonal question, no difference in the period covered. The only difference between the two answers below is how the four are added together.

Work it through and the two answers are both plainly right. Give every unit one vote and the four changes are up 4.00, up 3.00, up 2.00 and down 3.00, summing to up 6.00 and averaging up 1.50 per cent. Weight each change by the share of output the unit holds and the contributions are up 0.40, up 0.36, up 0.40 and down 1.74, summing to down 0.58 per cent. Three of the four units improved, and the units that improved hold 42.00 per cent of output between them while the single unit that fell holds 58.00 per cent.

So one honest procedure says the sector rose and another honest procedure says it fell, from the same four returns in the same period. There is no error to find. There is a choice to find, and the choice is in the weights. Coverage does the same job in a different way: two indicators disagree when one reaches parts of the thing the other never sees. Collection does it a third way. A question asked of a manager and a tonnage read off a meter can come apart even when both are recorded faithfully.

The disagreement is information about how the two were constructed, and resolving it means reading both methods rather than picking the more congenial one. The reflex to fight is the one that asks which number to trust. The useful move is to ask what the two did differently. Once that difference is known, both numbers become usable and something has been learned that neither of them said alone.

What does an indicator not measure?

Whatever falls outside its scope, and for every indicator ever built that is most of the economy. A narrow scope sounds like a criticism and is not one. An indicator has to choose a slice to be collectable at all, and the slice is the source of its precision. The failure is never that the slice is narrow. The failure is quoting the slice as though it were the whole.

Sankhya gives the cleanest possible case. Industry is 22.00 per cent of Sankhya output. An industrial volume index therefore reaches 22.00 per cent of output and says nothing whatever about the remaining 78.00 per cent, where services of every kind, farming and allied work, construction, and government and the rest of it all sit. The two shares add to 100.00 per cent, so nothing has gone missing from the picture: the other 78.00 per cent is simply not in this particular index, and never was, and was never claimed to be.

Industry is 22.00 per cent of Sankhya output, so an industrial volume index is silent on 78.00 per cent of it, and that is a scope rather than a flaw. Read the same sentence about the price index and it holds in a different unit: a basket priced across 92.00 per cent of household spending is silent on the 8.00 per cent it does not price, and that too is a scope. Read it about the firm survey and the silence is larger still, at 12.10 per cent of output covered.

One extra thing an indicator does not measure is worth naming because it catches people out separately: it does not measure its own reliability. A published figure comes with a vintageWhich edition of a figure is in hand. The same period is often published more than once as later returns arrive, and the editions differ. rather than a guarantee, and the number in hand may be the earliest edition of it. Vintage is a separate subject from scope and is treated on its own elsewhere.

What does somebody experienced do first with an indicator they have never seen?

Watch an analyst meet an unfamiliar release and the striking thing is what they do not do. The analyst does not look at the number. Finding the method note comes first, and four lines out of it get read: what is counted, whom it is collected from, what share of the thing it reaches, and how the parts are weighted. Only after those four does the level get looked at, and even then it gets looked at beside something comparable rather than on its own.

The reason is not scrupulousness, it is efficiency. A level with no coverage share attached to it cannot be set beside anything, so time spent looking at it is wasted time. A coverage share, on the other hand, immediately shows which comparisons are available and which are closed off. Once industry is known to be 22.00 per cent of output, it follows at once that this index can be compared with itself over time, can be compared with industrial measures elsewhere, and cannot be spoken of as the economy.

Everyone downstream of the analyst uses the same order without always knowing it. A lender deciding whether a borrowing sector is tightening wants to know whether the release covers the borrower's sector at all before it cares what the release says. An investor comparing two countries wants to know whether the two indices count the same objects. If they do not, the comparison is a comparison of methods and not of countries. A household reading a headline about prices is doing the same thing when it asks whether the basket includes what the household actually buys.

The coverage share is the first thing worth finding and the last thing anybody looks for, and reversing that order is most of what separates a usable reading from a confident wrong one.

The Republic of Sankhya, its statistical office and the four Sankhya indicators described here are invented.
Educational material. Not advice on any investment, tax, budget or market position.