Thematic Research: Structure, and the Story-Led Trap

Thematic research states a claim, writes the membership rule, predicts where the claim should already be visible and where it should not, and then looks. The order matters because a theme arrives as a story, and a story supplies the conclusion first. Writing down what would disconfirm it before looking is the whole method.

One idea established earlier sits underneath the whole procedure. A research view has to be checkable by somebody other than the person who holds it. A theme comes with its own explanation of the world already attached, and an explanation feels like a finding, so a theme makes that check harder than usual. The steps below are not a procedure for finding things out but a procedure for making a claim capable of being wrong, and only then going to look.

What are the steps, and what order do they run in?

There are four, and three of them happen before a single filing is opened. The claim is stated in one sentence. Then comes the membership rule, the test that decides whether a company is inside the theme or outside it. Then a written note of where the claim should already be visible if it is true, and where it should not be. Only then does the looking begin, and what turns up is recorded against each prediction, including the predictions that found nothing.

The three writing steps come before the looking step, and everything the method is worth comes from that ordering. Reversed, the work still produces a document, with charts, a rule and a sentence. But a prediction written after the evidence has already been seen cannot fail, and a prediction that cannot fail carries no information when it is met. The document no longer offers any way of telling whether the claim was true.

Consider a small thing at home. A household electricity bill jumps, and the new fridge is named as the reason. Naming the fridge is a claim. The useful next move is not to go and find three things that fit. Say instead, out loud and before any checking, what should be expected: the jump should have started in the month the fridge arrived and not before, and the neighbour with the same old fridge should show no jump at all. Only then does the checking begin. Skip that middle step and a reason will turn up, always. Reasons are cheap, and there are dozens of them lying around.

Why does the order matter more here than in ordinary research?

Because of where the analyst is standing when the work begins. Most research starts with a company about which no view is yet held. There is reading, counting, arguing back and forth, and a view forms somewhere near the end. The conclusion is the last thing to arrive. The evidence therefore had a genuine chance to push the conclusion around.

A theme does not arrive that way. A theme arrives as an account of how the world is changing, already shaped, already persuasive, usually handed over by somebody who found it persuasive first. Thematic research is the one situation in which the conclusion turns up before the evidence, and the written prediction step exists purely to put the missing step back. What is called for here is not more rigour than usual as a matter of character. The written prediction step rebuilds by hand one ordinary step of research procedure that the shape of a theme has quietly removed.

What is story-led analysis, and how does it start?

Story-led analysis starts the way good work starts. Somebody describes a change in the world clearly enough to be seen, and that description is genuinely interesting. So the search for material begins. A filing turns up that mentions the change. A figure turns up that sits the right way up. Both are kept. Six weeks later there is a folder of thirty exhibitsA single chart, table or extract presented inside a research document as evidence for the claim being made. and a strong feeling that the case is now made.

Every individual step in that sequence looks exactly like research, and the sequence as a whole is not research at all. Nobody notices themselves doing it for that reason alone. There is no moment at which anyone decides to be fooled. There is no exhibit in the folder that is false. The problem lives in what is absent, and an absence is hard to see. Nothing in the folder could ever have been filed against the claim, and no prediction was ever written that the folder could contradict. The tendency to keep what fits and drift past what does not has a name and a large literature of its own, usually gathered under confirmation biasThe general tendency to notice, seek and remember material that supports what is already believed. Covered under the behavioural material., and it is treated properly elsewhere. The mechanical fact is that a folder assembled without a disconfirming prediction is a record of what was kept, not a record of what is true.

Why is a strong story the most dangerous input?

The relation between how good a story is and how hard it gets tested runs backwards, and it reads that way until it has been sat with. A weak story gets taken apart. Somebody hears it, frowns, asks the obvious question, and the whole thing either improves or dies within a week. A weak story invites the obvious question, so the testing happens on its own.

A strong story does the opposite. It recruits. Once an account of the world is compelling enough, the facts that fit it feel like the pattern and the facts that do not feel like details, exceptions, timing noise, somebody else's problem. The story already answered the obvious question, so nobody asks it. The quality of a story says nothing about the world and is a very good predictor of how little that story will be examined.

The same thing is easy enough to feel outside markets. A vague rumour about a neighbourhood road being widened gets argued over at every tea stall on the street. A vivid, detailed, perfectly coherent version of the same rumour, with names and dates in it, gets repeated instead of argued over. The detail did not make the rumour more likely. The detail made the rumour harder to interrupt.

What evidence separates a theme from a narrative?

One test, and it is quick: what does the claim predict that could turn out otherwise? A theme names a figure, a line it should appear in, a place it should not appear, and roughly when. A narrative predicts that things will go well. Almost any imaginable future satisfies that prediction, so it rules nothing out.

A theme predicts something specific enough to fail. A narrative predicts only that matters will improve, and nearly everything is compatible with that, so a narrative tests nothing. That is the entire distinction, and it can be applied to somebody else's work in about thirty seconds. The claim is read, and the number that would embarrass its author is written down. If no such number can be found, the document is not research, whatever it is called.

How is a theme tested by somebody who already believes it?

Testing a theme already believed is the hardest part in practice. By the time the work is under way the analyst is usually convinced. Three moves help, and they get progressively more useful.

The first is to write the disconfirming prediction before anything else and put a date on it. Not a vague one. Something like: if this is wrong, the number in this line will look like that by the end of the second reported year. A dated prediction is the only kind that ever comes back.

The second is to ask what the strongest person holding the opposite view would point at. Not the weakest version, the one everybody argues against, and not a caricature. The best available opposite case, stated as well as they would state it.

The third is the one that actually works. Find the company the theme should be hurting, and go and check whether it is being hurt. If households really are repainting more often for reasons unconnected to new building, somebody loses: a maker geared entirely to construction volume, a product that gets skipped, a channel that shrinks. A change that hurts nobody is usually not a change at all, only a description. The search for the injured party is therefore the sharpest test available, and the one almost nobody performs.

What happens when the structure is run on a real claim?

Take the repainting theme and run all four steps on it, in order, against the record of Sarvani Coatings Limited, an invented paint maker selling into a field invented alongside it.

Step one, the claim in one sentence. Households now repaint more often than they used to, and for reasons that have nothing to do with how much new building is going on. Coatings demand therefore carries a growing secularThe part of demand that grows for reasons unrelated to the ups and downs of the business cycle. Covered separately. component.

Step two, the membership rule. A company is inside this theme if its revenue rises when households repaint more often, whatever else is true about it. Both Sarvani Coatings Limited and Nandivarman Paints Limited sell to households, so the rule puts them inside, and a purely industrial maker falls outside.

Step three, the four predictions, written before anything is opened. First, field volume should hold up in a year when building activity is weak. Second, the effect should show in decorativePaint sold to households through dealers, as against industrial coatings sold to manufacturers on contract. volume rather than in industrial volume. Third, Thottam Chemicals Limited sells resins and additives from one step further up the chain, and should see the effect too. Fourth, and this is the one written to hurt: if the theme is wrong, the whole volume record should track building activity closely.

Step four, the looking. Here is the volume record the four predictions have to be settled against, and it is worth reading as rows rather than as a sentence.

Alongside that sit the size figures. The whole field earned Rs 48,300 crore, up from Rs 43,500 crore, a rise of 11.0 per cent. Sarvani Coatings grew revenue 13.9 per cent, or 2.9 points faster than the field, and grew volume 6.0 per cent against the field figure of 4.5 per cent.

Set honestly against each prediction, that record gives the following. The first prediction survives, and it survives weakly: a single contraction of only 1.4 per cent is what would appear if a cyclicalMoving with the ups and downs of building and industrial spending rather than with a longer running change in behaviour. dip were being cushioned by a secular floor, and it is equally what would appear if the dip were simply mild. The fourth prediction, the disconfirming one, does not fire. The record swings, as it should, and never collapses, so the wrong-theme case is not made either. The second and third predictions cannot be tested at all. The record does not split volume into decorative and industrial, and it carries no volumes whatever for the supplier one step up.

One weak survival, one prediction that did not fire and two that could not be tested is the honest result of a real thematic exercise, and reporting exactly that is the work. It is tempting to reach further, to treat the mild contraction as confirmation and to fill the two gaps with reasoning. Neither move is legitimate. The gaps are the most informative thing the exercise produced. They name precisely which two findings would move the claim, and where each one would have to be found.

What does thematic research actually produce?

Four things, and the fourth is the one people leave out. A claim in one sentence. The membership rule that decides who is inside it. Every prediction that was written, each with what was found against it, including the ones where the answer was that nothing could be found. And then a plain statement of what remains untested.

The untested section is the most valuable part of the document, and its absence is the clearest available signal that the work was story-led. That sounds like a paradox and it is not. The untested list is the only part of the document that tells a reader where the claim is still soft, which two facts would settle it, and what to watch for over the next two reporting cycles. A document with no such section either claims that every prediction was settled, and that almost never happens, or it never wrote any predictions at all.