How to Summarise a Financial Dataset With Descriptive Statistics

A complete summary has two halves. The descriptive half says what the record holds: the count, the centres, the spread, the extremes and the shape. The uncertainty half says how much of it is settled: the standard error and two intervals. For the fifty month record of the invented Nakshatra unit that comes to twelve lines, and the three in the uncertainty half are the ones usually missing.

Start with the word in the title. A dataset is a record of cases. Somebody decided what one case would be, went and collected a pile of them, and wrote them down in one place. The cases here are months. The invented Nakshatra unit is a traded unitA nameless something that carries a price, and whose price is allowed to wander. None of it has to be decided before the counting can begin, so what it is, who issues it and what it does are all left unsettled on purpose., and its monthly changeHow far the price travelled across a single month, measured against the level it opened at and written as a percentage. A reading of plus 11.00 marks a month that closed a good way up, and minus 9.00 marks one that closed well down. was written down for fifty months running. Fifty cases, one figure each: that is the whole dataset. Twenty five of the fifty months landed on 1.00 per cent, eight on 6.00 and three on 11.00 above that, nine on minus 4.00 and five on minus 9.00 below. Five counts, adding to fifty.

Every figure in the summary below was built somewhere in the notes that came before. The two means, the median and the mode were built in the notes on centres. The standard deviation and the range were built in the notes on spread. The skewness and the kurtosis were built in the notes on shape. The standard error was built in the notes on the distribution of an estimate, and both intervals were built in the notes on intervals. The list itself has never been assembled until now: which of those figures belong in a summary, what order they go in, and which of them a reader is entitled to see whether or not anybody asked for it.

One thing about this record deserves a sentence rather than a footnote, and it allows a verdict at the end that almost no explanation of the subject can offer. The generatorThe recipe the months came out of, naming each reading it is able to produce and how heavily it favours that one. Because the recipe was on paper ahead of the record, its own centre is something these notes can look up instead of estimate. behind the fifty months was written down first. Its true average is 1.00 per cent and its true spread is 5.00 per cent, both facts of construction rather than measurements. So at the end of this guide the finished summary can be laid beside the answer it was reaching for, line by line, and marked.

What is a summary for, and who is it for?

A summary is not a shorter version of the record. A summary is a substitute for the record, written for somebody who will never see it. The reader of a summary reads the twelve lines, forms a view, and acts on that view without ever counting a month. Everything they will ever know about those fifty months is what the author chose to write down.

The substitution gives a test, and it is a hard one. If a reader would reach a different conclusion from the summary than from the record itself, the summary is incomplete, no matter how many figures it carries. Notice what that test does not say. Correctness is assumed rather than tested. The test says correctness is not enough. A summary can be right on every line it prints and still send its reader somewhere the record would not have sent them, purely through what it left off.

Here is the everyday version. A shopkeeper is asked how business went last year and answers that takings averaged a certain amount a day. Every word true. Suppose a road outside was dug up for six months. Half the year then ran at double that figure and the other half at nothing, and the person who heard only the average now believes something flatly wrong about that shop. Nobody gave them a false number. A true one arriving alone did the misleading.

So the reader is the specification. The reader's own next move settles the list of lines. How much record was there? A reader who does not know that cannot know how seriously to take anything below it. What was a typical month, and how far from typical did the months actually run? How bad was the worst one? And how much of all this would a second record of the same thing have reproduced? Six questions. Twelve lines answer them.

What order are the twelve lines built in?

Count first, then the centres, then the spread, then the extremes, then the shape, then the uncertainty. Six stages, twelve lines. The order is not a filing convention and it is not alphabetical. Each stage exists to make the stage before it readable, so building them out of order produces a summary whose lines cannot be interpreted in the order a reader meets them.

Take the transitions one at a time. The count comes first because it decides what everything below it is worth. A mean drawn from four months and a mean drawn from four hundred are printed identically and are not remotely the same object, and a reader who meets the mean before the count has already formed an impression by the time the count arrives to correct it.

A centre on its own is half an answer, so the centres come next and the spread must follow immediately. Two records can share an average exactly and be nothing alike, one of them barely moving and the other swinging violently around the same middle. The spread is a single summary figure and the extremes are the check on it, so the extremes come after. They say whether that one figure is describing the record fairly or whether one freak month is doing most of the work.

The shape comes next because it answers the question the extremes raise. The lowest and the highest month give how far the record reached in each direction; the shape gives whether the misses pile up on one side and how much weight sits far from the middle. And the uncertainty half comes last because it is a comment on everything above it. The uncertainty half does not describe the record at all. Its three lines say how much of the nine above would have come back the same if somebody had collected a second fifty months instead.

Notice the small cruelty at the bottom of that ladder. The stage a reader most needs in order to weigh everything else is the stage that arrives last, when the work already looks finished and the person building it is already tired of building it. Order explains the omission. Nothing excuses it.

What goes in the descriptive half?

Nine lines, and they are simply the output of the first five stages. A summary has rows, so the nine are printed as a list rather than described in prose.

Two of those rows need a word about why they are rows at all rather than a choice between two rows. The first is the spread, printed with its denominator named. There are two ways of dividing when a spread is worked out, they give 4.97 and 4.92 on this record, and which one was used is not something a reader can reverse engineer from the answer. Naming the corrected denominatorOne of the two divisors available when a spread is worked out from a sample rather than from a whole population. Which one to use, and why they differ, is covered separately; a summary only has to say which was used. costs three words and removes an ambiguity that cannot be removed any other way.

The second is the pair of means, and this is the row people cut. A reader who is going to add the figure needs a different line from a reader who is going to compound it, so both means belong in a summary rather than whichever one the author prefers. The arithmetic mean of 0.50 per cent is the one that answers questions about adding and averaging. The geometric mean of 0.38 per cent is the one that answers questions about compoundingGrowth applied on top of growth, so each period's change is worked out on the total the period before left behind rather than on the original amount. Why that needs a different average is covered separately., where each month's change lands on whatever the month before left behind. Adding and compounding are different questions with different answers, and the person writing the summary does not know which one the reader will bring. With both printed, the reader picks. With one printed, the author has picked for them, silently.

What goes in the uncertainty half, and why is it the half that gets dropped?

Three lines. The standard error is 0.70 per cent. The interval for the mean runs from minus 0.88 to 1.88 per cent. The interval for a single month runs from minus 9.35 to 10.35 per cent. All three were built earlier and none of them is rebuilt here.

Together the three lines do what the nine above cannot do for themselves. Without these three lines a reader cannot tell whether the mean of 0.50 per cent is a finding or a shrug, and the entire descriptive half reads as though it had been measured rather than estimated. That is the whole difference. A measured figure is what it is. An estimated figure is one answer out of many that the same procedure could have produced, and the three lines below the rule are the only place in the summary where that is admitted.

Read them in order. Line ten, at 0.70 per cent, gauges the typical distance between this record's average and the average a second batch of fifty months would have thrown up. Line eleven, running from minus 0.88 to 1.88, turns that into a stated range for the true average. The interval for a single month, from minus 9.35 to 10.35, answers a different question again: not where the average sits, but how wild one month can be. Note how far apart those last two ranges are in width. One is under three percentage points wide and the other is nearly twenty. Both are correct and they answer different questions, and which of the two a reader needs is covered separately. Both are computed on the assumption of the normal shapeThe familiar single humped, symmetric curve used here only to supply the multiplier that turns a spread into an interval. The shape itself is covered separately. for the multiplier, which is itself a convention the summary has to declare.

Why do they go missing? Not because anybody decided to hide them. Three reasons: nobody asked for them, they arrive last, and above all a summary with its uncertainty half removed looks exactly as finished as one that still has it. There is no ragged edge, no obviously empty box, nothing a reviewer's eye catches. Nine tidy lines of correct arithmetic present themselves as complete work, and they are not.

Which lines move when the record gets longer, and which do not?

One reading makes the two halves worth separating at all, and it is best met as a question before an answer. Suppose the same unit had been recorded for four hundred months instead of fifty, and suppose the longer record came out with the same mean of 0.50 per cent and the same spread of 4.97. Which of the twelve lines would read differently?

The nine descriptive lines would not move at all. The arithmetic mean is still 0.50 per cent. The geometric mean is still 0.38. The median and the mode are still 1.00. The standard deviation is still 4.97 and the range still 20.00. The skewness is still minus 0.05 and the kurtosis still 3.00. The descriptive lines report what is in the record, and by the terms of the question nothing in it has changed.

Every line in the uncertainty half moves. The standard error falls from 0.70 per cent to about 0.25. The interval for the mean tightens from a span of nearly three percentage pointsThe unit used when subtracting one percentage from another. Going from 4 per cent to 6 per cent is a rise of two percentage points, not a rise of two per cent, and keeping the two apart avoids an ambiguity that has cost people money. to a span of about one. The interval for a single month barely moves at all, from a width of 19.69 to 19.52, and that near stillness is itself informative: how wild one month can be is a fact about the unit, not about how long anybody watched it.

The two halves answer different questions, and only one of them is about how much record there is. That single reading is what a summary's split is for. The descriptive half is a report on a pile of months and it says nothing whatever about how confident anybody should be. The uncertainty half is a report on the reporting, and it is the only part of the summary that responds to effort.

Two settings are worth visiting deliberately. Push the slider to 1 and the panel refuses: a single month has no spread of its own to divide, so there is no uncertainty half to print, and refusing is the honest output. Push it to 400 and watch the interval for the mean shrink to a span of about one percentage point while the nine lines above it sit exactly where they were at the opening setting. Nothing about the record changed. Only the confidence in the reporting of it did.

What must a summary say about itself?

Three more lines, and none of them is a figure. The three describe the summary rather than the record, and together they make the twelve lines above them auditable by somebody who was not there.

The first says what the cases are and over what stretch. Here: monthly changes of an invented traded unit, fifty months. Without it, the reader cannot tell whether a mean of 0.50 per cent is a month, a quarter or a year, and that ambiguity alone can be worth an order of magnitude.

The second says what was excluded and how much of it there was. Here: nothing was excluded, all fifty months are in. A summary with no exclusion count cannot even be checked by its own author six months later, so the exclusion line is what turns a set of numbers into something anybody can check. Nothing was excluded is a real answer and a strong one. Silence is the fatal answer. On paper it looks identical to nothing was excluded, and the two mean completely different things.

The third says which conventions were used. Here: the spread uses the corrected denominator, and both intervals assume the normal shape for the multiplier. A reader cannot recover either choice from the answers, and stating both costs a line.

How did this summary score against the truth?

Here is where these notes can do something they have been able to do all the way through and almost nothing else can. The generator behind the fifty month record was typed out before a single month was counted, so its true average of 1.00 per cent and true spread of 5.00 per cent are facts rather than estimates. The completed summary can therefore be marked.

Line by line. The summary reports an arithmetic mean of 0.50 per cent against a true 1.00, so the centre is out by half. The spread of 4.97 against a true 5.00 is very nearly right. The skewness of minus 0.05 against a true 0.00 detects a lean in a population that does not lean at all. And the kurtosis is reported as 3.00, the value the normal shape carries, for a population that takes exactly five values with a hard wall at minus 9.00 and another at 11.00. A population with hard walls has no tails to be heavy or light about.

Take that in properly. Nobody made a mistake. Every one of the twelve lines was computed correctly from a record that was collected honestly, and this was a good summary by any standard anybody could apply to it from the outside. A complete, honest, correctly computed summary of fifty months got the centre half wrong and said nothing true about the shape, and its uncertainty half said so.

The last clause is the whole point. Look back at line eleven. The interval for the mean runs from minus 0.88 to 1.88 per cent, and the true average of 1.00 sits comfortably inside it. The point estimatorThe recipe a figure was produced by, as opposed to the figure itself. Taking the average of whatever months are to hand is an estimator; the 0.50 per cent it returned on this record is one output of it. missed by half and the interval was still right. A reader given only the nine descriptive lines would have walked away believing the unit runs at 0.50 per cent a month. A reader given all twelve would have walked away knowing that anything from roughly minus 0.9 to roughly 1.9 was on the table. Less satisfying to be told, and very much more accurate. Neither reader was given a wrong number. Only one of them was given enough.

How is the template used on a record of another kind?

An analyst who is handed somebody else's summary does one thing before reading a single figure: they count the lines. Nine means an incomplete document, whatever else is in it, and the correct next move is to go back and ask for the standard error rather than to start reasoning from the mean. A mean of monthly takings with no spread and no count beside it is not evidence of anything, and reading it carefully cannot turn it into evidence. A lender reading a borrower's trading history does the same count in reverse. And a household deciding whether one year of a side business is enough to plan around is asking exactly the question the uncertainty half answers.

So here is the template as something to copy, twelve lines and three about themselves.

And one instruction is worth carrying away above all the others. A blank line and a line that was never there look identical to everybody except the person who wrote them, so fill every line or write not available beside it. Not available is real information. Those two words tell a reader that somebody considered the line, could not fill it, and said so. A reader can respond to that: they can ask why, they can go and get it, they can discount the summary accordingly. A reader who does not know a line is missing has none of those moves.