Confidence Interval vs Prediction Interval: Two Questions

A confidence interval answers where the true average of every month sits. A prediction interval answers where one month that has not happened yet will land. Worked on one invented record of fifty months, the first spans minus 0.88 to 1.88 per cent and the second spans minus 9.35 to 10.35, so their widths are 2.76 and 19.69. Lengthening the record shrinks one and barely touches the other.

Two questions turn up at the same desk in the same week, and they are close enough in sound that most people answer them with one number. One asks what this thing typically does in a month. The other asks what it might do next month. Both get answered with a low figure, a high figure and a per cent sign, printed in the same typeface, sitting in the same column of the same note. The two are not the same question, they do not have the same answer, and on the record used below one of the two answers is a little over seven times wider than the other. Telling them apart takes one question, and confusing them costs more than it looks.

Everything below rests on three objects that were built elsewhere and are only used here. One of them is the Nakshatra unit, an invented and deliberately unnamed traded unitAn object that carries a price, with that price left free to wander. Here the phrase sits on a made up thing, so no instrument anybody could actually go and buy is being pointed at. whose monthly changeThe distance a price covered between the opening of a month and its close, scaled by where it opened. A reading of 6.00 says the close finished six per cent above the open. is the only quantity in play. Another is the generatorA rule on paper naming the handful of readings permitted and the weight riding on each. Composed rather than observed. Everything true about it can be worked out with a pen and no data whatsoever. sitting behind it, which permits five readings and no others: minus 9.00, minus 4.00, 1.00, 6.00 and 11.00 per cent, weighted 0.08, 0.18, 0.48, 0.18 and 0.08. The last is a single record of fifty months produced by that rule.

One fact about this unit matters more than any other. The rule went onto paper first and the months came out of it later. The truth about this unit is therefore on the table. The average monthly change it truly has is 1.00 per cent, and the scatter it truly has is 5.00 per cent. Neither figure estimates anything. Both follow from how the rule was written. Almost no record comes with its own truth attached. This one does, so every answer below can be measured against the truth rather than merely suspected of missing it. Every claim below is checked against that truth at least once.

The record itself is a tallyFive numbers saying how often each permitted reading showed up across a record. Nothing else is kept, and nothing else is needed to redo every figure below.: 5 months landing on the first reading, 9 on the second, 25 on the third, 8 on the fourth and 3 on the last. Averaged out, those fifty months give 0.50 per cent, precisely one half of the truth. Its own spreadA single figure for how widely the readings are scattered about their middle. Building one, and the two competing denominators it can be built with, is covered separately. comes to 4.9744 per cent. The record's standard errorHow much an average of this many months would bounce if somebody went out and collected an entirely fresh record of the same length. Its origin and its behaviour are covered separately., the figure describing how much that average would bounce, comes to 0.7035 per cent. The average, the spread and the standard error, plus a multiplier of 1.959964, produce every band below.

Computing a band is covered separately, and where the standard error comes from is covered separately too. No line is fitted to these fifty months, and no coefficient is chosen to make one thing agree with another: both bands are arithmetic worked on five counts. Naming the value a single future month will take is a different problem, covered separately.

What question does the band for an average answer?

Take the question first and leave the formula alone. The formula makes these two look like siblings; the question tells them apart. The band for an average answers one question: where does the true average monthly change of this unit sit, given what fifty months were able to show? Notice what the question is about. The question is about a single number that describes the whole unit, across every month it has produced and every month it ever will.

A canteen behind a bus depot makes this concrete without any statistics at all. The manager has fifty days of plate counts in a notebook. She can ask how many plates the canteen sells on an average day. The question is about the canteen, and its answer is one number that already exists whether or not she ever writes it down. Fifty days of counting does not create that number and does not change it. Fifty days of counting narrows down where she thinks it is.

The band for an average has exactly that character. The thing it describes is fixed. The true average is not wobbling, not waiting to happen, and not affected in any way by how much counting anybody does. The uncertainty in a band for an average is uncertainty about the observer, not about the thing observed. The unit has an average. Nobody observing it knows that average. The band says how far off the best guess might be.

On the fifty month record the answer runs from minus 0.8788 to 1.8788 per cent. The band was built by taking the record's average of 0.50 per cent and reaching out 1.3788 per cent on each side, that reach being the multiplier of 1.959964 laid on the standard error of 0.7035. A note would ordinarily print it as minus 0.88 to 1.88, and that is a fair presentation. Rounding an endpoint inward moves it. Carry the four decimal version whenever the endpoints go on to do further work. A value sitting exactly on the rounded edge answers one way at minus 0.88 and the other way at minus 0.8788. Which questions those are is covered separately.

Then the check that this record can make and an ordinary one cannot. The true average is 1.00 per cent, and minus 0.8788 to 1.8788 contains it. The single best guess was 0.50 per cent, half the truth and badly wrong. The band around that guess was still right. Hence the standing preference for a band over a point estimateA single number handed over as the answer, carrying nothing that says how far from the truth it might sit. The 0.50 per cent here is one, and it reads as far more settled than it is..

What question does the band for one month answer?

Now the other question, and it is about a completely different kind of thing. The band for one month answers this: where will the next single month land? Not the average of months. One month, taken by itself, the one that has not happened yet.

Back to the canteen. The second question the manager has is how many plates she will sell tomorrow, and tomorrow is not a fixed quantity sitting there waiting to be discovered. There is no true number for tomorrow that fifty days of counting slowly reveals. Tomorrow has not been drawn yet. Tomorrow could be a wet Tuesday with three buses cancelled, or the first day of a festival week. The band for one case is not narrowing in on something that already exists; it is describing how far apart the possibilities are.

For the Nakshatra unit the possibilities are set out in the rule itself. Next month can come out at minus 9.00, minus 4.00, 1.00, 6.00 or 11.00 per cent, and it will do so with the weights the rule gives. No amount of past counting removes any of those five from the list. On the fifty month record the band for one month runs from minus 9.3467 to 10.3467 per cent.

Here is the check that makes both bands feel real rather than notional, and it uses only the fifty months already on file. Take the wide band and ask how many of those fifty months fall inside it. Forty seven of them do, and only the three months that read 11.00 per cent fall outside. Ninety four per cent of the record, then, against a stated level of ninety five. Now take the narrow band and ask the same thing. Twenty five of the fifty months fall inside minus 0.8788 to 1.8788 per cent and twenty five fall outside, so exactly half the record the band was built from lies beyond it. A band that half the observations miss was never describing an observation.

Where exactly do the two part company?

The two bands part company at one place only, and once that place is seen the rest is bookkeeping. The band for one month carries everything the band for the average carries, and then one more thing on top: the ordinary month to month variation that a single month is subject to and an average is not. The extra variation is an addition, not a different method and not a separate kind of calculation. The addition is why the wide band can never, on any record, come out narrower than the narrow one.

Think about why an average is steadier than a month. Fifty months get added up and divided by fifty. A month that came in at minus 9.00 gets partly cancelled by a month that came in at 11.00, and the more months there are the more of that cancelling happens. A single month gets no such help. A single month arrives on its own, at whatever value the rule hands it, with nothing to average against. So there are two separate sources of wobble in play, and only one of them touches the average.

The first source is how far a fifty month average would jump if the whole record were collected once more, and that quantity comes to 0.7035 per cent. The second source is how far a single month sits from the centre in the ordinary run of things, and that is the spread of 4.9744 per cent. The band for the average uses the first alone. A future month is subject to both, and the band for one month therefore carries both. The centre is not known exactly, and even if it were, the month would still scatter around it.

Two independent wobbles combine by adding their squares rather than by adding themselves. The total therefore comes out smaller than 0.7035 plus 4.9744 and larger than 4.9744 alone. Square the two, add, take the root: 0.4949 plus 24.7449 is 25.2398, whose square root is 5.0239 per cent. The combined 5.0239, multiplied by 1.959964, gives a reach of 9.8467 per cent on each side of 0.50. The band lands at minus 9.3467 to 10.3467.

How much wider is it, on this record?

Put the two widths beside each other. The band for the average is 2.7576 percentage points wide. The band for one month is 19.6934 percentage points wide. Divide one by the other and the answer is 7.1414. On the same fifty months, at the same level, from the same three figures, one honest answer is a little over seven times the size of the other honest answer.

The factor of 7.1414 is not a coincidence and not a property of this particular record. Work it out in symbols rather than numbers and almost everything cancels: the multiplier cancels, the spread cancels, and what survives is the square root of one more than the number of months. Fifty months give the square root of 51, and that root is 7.1414. A record of ninety nine months would give the square root of 100, exactly ten. The factor between the two widths depends on nothing except how many months are in the record.

A second seven turns up in these figures, and the two sevens are near neighbours rather than the same number. The record's individual months scatter by 4.9744 per cent and a fifty month average scatters by 0.7035 per cent. Dividing those gives 7.0711, the square root of 50. The square root of 50 is the factor between a month and an average. The factor between the two band widths is 7.1414, the square root of 51. The difference between them, small as it looks, is precisely the single month's own variation being added back into the wide band rather than left out of it.

What happens to each as the record grows?

Everything so far has concerned one record of fifty months and could have been read off a pair of definitions. How the two widths behave as the record lengthens could not. Pin the centre at 0.50 per cent, pin the spread at 4.9744 per cent, and then ask what the two widths would have looked like had the record run to some other length.

At four hundred months the band for the average runs 0.9750 percentage points wide, down from 2.7576. The band for the average has lost nearly two thirds of its width. At the same four hundred months the band for one month runs 19.5237 points wide, down from 19.6934. The band for one month has lost seventeen hundredths of a point, a line thickness on a picture at this scale.

Month to month variation is a property of the unit rather than a shortage of information, so more months settle the average and do almost nothing for a single month. The first column falls because it is measuring the observer's ignorance, and ignorance is a thing that counting fixes. The second column stands still because it is measuring how much the unit itself moves about, and no amount of counting has ever made anything move about less.

There is a floor underneath the second column and it is worth knowing where it is. Push the record out towards infinity and the extra stretch on the wide band shrinks away entirely, leaving the multiplier laid on the spread alone. The arithmetic is 1.959964 times 4.9744 twice over, or 19.4994 percentage points. The band for one month can never be narrower than 19.4994 points, at any record length whatsoever, and the fifty month record is already within two tenths of a point of that floor. The band for the average has no such floor and heads towards zero.

The everyday version costs nothing to picture. Counting the traffic on one road every day for a year pins down that road's average day to within a very tight range indeed. The same year of counting says almost nothing more than March already said about whether tomorrow morning will be a jam. The first is a question about the road. The second is a question about tomorrow, and tomorrow was never in the notebook.

One honest wrinkle sits inside that panel. At four hundred months the band for the average runs from 0.0125 to 0.9875 per cent, and no longer contains the true average of 1.00. The miss is not a discovery about long records but an artefact of the panel pinning the centre at 0.50 per cent, where fifty months happened to land, and then pretending that eight times as much data would have landed in the same place. A genuinely longer record would report a centre of its own, and it would very probably sit closer to 1.00. The panel is a statement about widths at different lengths and about nothing else at all.

Which of the two does a reader actually want?

The answer here is uncomfortable, so it is worth putting plainly. Almost every practical question anybody brings to a set of numbers is a question about one case rather than about a long run average. The questions run: what will this month look like, how bad could this quarter get, and what range should be expected for the next single thing that happens. The band a reader nearly always wants is the wide one, and the band a reader is nearly always shown is the narrow one.

Three reasons for that, none of them dishonest and all of them costly. The first is that the narrow band is what falls out of the standard calculation. In most of the places the phrase is used, an interval means the one about the average by default. Ask any ordinary routine for an interval and it hands that one back. Nobody chose it over the alternative; it simply arrived.

The second is that the narrow band looks like a result. Minus 0.88 to 1.88 per cent is a tight, confident-looking pair of numbers that a reader can carry away and act on. Minus 9.35 to 10.35 looks like an admission that nobody knows anything. An admission is an unpopular thing to put in a note even when it is the accurate thing to put in a note.

The third is that the narrow band makes the work appear to have settled something. Fifty months of collecting and computing produced a range two and three quarter points wide, and that feels proportionate to the effort. Reporting that fifty months of work leaves next month anywhere in a twenty point range feels like reporting that the work achieved nothing. What the work achieved was an accurate description of a unit that moves a great deal.

What does quoting the narrow one cost?

Here is the cost, in one concrete month. A note goes round quoting minus 0.88 to 1.88 per cent for the Nakshatra unit. The note carries no label saying which question those endpoints answer. A reader takes the endpoints as the range next month will land in, and that was the question the reader actually had. Next month arrives at minus 5.00 per cent.

Against the band the reader was holding, minus 5.00 is not merely outside; it is more than three and a half points beyond the lower end of a band two and three quarter points wide. The month reads as a break. On the only description of the unit the reader had, a month like that should essentially never occur, so it reads as something having gone wrong with the unit.

Against the band that answers the question the reader was actually asking, minus 5.00 sits comfortably inside minus 9.3467 to 10.3467 and is not remarkable in any way. Minus 5.00 sits 1.1057 ordinary spreads below the record's own centre. Five of the fifty months already sitting in the record were worse than the month that triggered the alarm, and fourteen of the fifty were at minus 4.00 or below. Nothing changed about the unit. Nothing was miscalculated. The wrong band was put in front of the right question.

How is it settled in one question?

The same one question settles it in every case, and the people who get this wrong are not people who do not know the difference; they are people who never paused to ask which question they were answering. A lender sizing a facility against a borrower's monthly collections wants to know how bad one month could be, not what the average month has been. An analyst writing a note about a unit wants the range a reader will hold a single period against. A household deciding how much to keep aside wants to know how large one bad month could be, not what the average month costs. All three want the band for one case, and all three will be handed the band for the average unless somebody asks the question.

So ask it first: is this about an average, or about one case? Almost every time the honest answer is one case, and almost every time the number sitting on the desk is the other one.

Then two checks, and they are both quick. The first check rests on the addition. The band for one case carries the same wobble plus another one, so on the same record at the same level it must be the wider of the two. In a pair where the one labelled for a single case is narrower, the two labels have been swapped and nothing else needs investigating.

The second check is the one that catches a mislabelled band when only one band is available, and it needs a second record length rather than a second band. The question is what the same calculation gives on a much longer record. The band for the average shrinks a great deal and the band for one case barely moves, so a band said to describe a single case that shrinks sharply when the record grows does not describe a single case whatever it has been called. On these figures, going from fifty months to four hundred takes the first from 2.7576 down to 0.9750 and the second from 19.6934 to 19.5237.

Neither check requires recomputing anything or being handed the underlying record. Both are therefore usable on somebody else's work. Both also fail safe. Where neither can be run, the correct step is to ask which question the endpoints answer, and that is a reasonable question to ask about any interval anybody has ever published.