Confidence vs Certainty: Stating a Belief Without Claiming One

Confidence is a degree of belief held by a particular person and written down as a number. Certainty is a claim that the outcome is already settled. No research produces that. Call each of the three conditions under the worked claim below 70 per cent likely, treat them as unrelated, and all three holding comes to 34.3 per cent.

The 34.3 per cent is not an answer. The figure is arithmetic done on numbers a reader supplied, on three conditions that are not unrelated at all, and the honest version of it is a range rather than a point. Getting to that honest version takes a little work, and the work is worth doing. The direction the arithmetic points in is the opposite of the direction a list of likely-sounding conditions feels like it is pointing.

Three things settled earlier hold the working up. The worked claim rests on exactly three variables, established when the drivers were named. The observation that would break the claim was written down before the claim was held, established when disconfirming evidence was dealt with. And the claim runs over a stated period, established when the horizon was set. Without a stated period, a belief written today can never be checked against anything.

Where does every number below actually come from?

Sarvani Coatings Limited, an invented maker of industrial coatings, supplies every number below, and the three conditions the worked claim leans on were written for it, along with the record they were lifted out of. Three numbers here are carried across unchanged from that record: a gross marginRevenue less what the materials cost, set against revenue and read as a percentage. Building the figure up from the accounts is set out under gross margin, and only the finished figure is used in this walkthrough. of 46.0 per cent in the most recent published year, on gross profit of Rs 1,111 crore against revenue of Rs 2,415 crore, volume growth of 6.0 per cent against a field growing 4.5 per cent, and a share gain of 0.13 percentage pointsThe unit used when one percentage is set beside another. Going 4.87 to 5.00 covers 0.13 of them, and that is not the same thing as 0.13 per cent., taking that share from 4.87 to 5.00 per cent in a single year.

Two small notes about those. The record prints its margin as 46.0 per cent; dividing the published gross profit of Rs 1,111 crore by the published revenue of Rs 2,415 crore recomputes to 46.0041 per cent, and the published figure is the one printed here. The record also prints a two year margin gain of 3.0 points, from 43.0 to 46.0 per cent; the unrounded absolutes behind those two figures give 2.96 points. Again the published figure is what appears below. Every probability below is one a reader supplied as an illustration, and not one of them was measured, sampled or estimated from anything. A supplied probability shows how the arithmetic behaves, and it never shows how likely anything is at Sarvani Coatings Limited.

What is confidence, and whose property is it?

Confidence is a degree of belief. The belief is held by a particular person, it is about a particular claim, and it exists at a particular moment. Change the person, change the claim or change the moment, and the number is different. Ownership is not a weakness in the idea. Ownership is the idea.

Here is the everyday version. Two people stand outside the same wedding hall on the same evening, both looking at the same sky. One says it will rain before the guests leave. The other says it will not. Both have seen exactly the same clouds. One of them grew up in that neighbourhood and has watched that particular grey sit there for three hours without doing anything; the other has been caught out twice this month and has stopped trusting it. Neither is being irrational. A degree of belief is assembled from more than the evidence in front of it, so the two of them hold different degrees of belief about one claim, on one evening, on shared evidence.

Confidence is a property of the believer rather than a property of the world. Two people with the same filings in front of them can hold different confidences without either of them making a mistake. Meghna Iyer, reading the record on Sarvani Coatings Limited, can put her belief that the margin holds at 55 per cent. A colleague reading the identical record can put it at 75. Nothing in the record decides between them. The record contains no probability at all, only three published years and a question the three years cannot settle.

What is certainty, and why does the word not belong in a research note?

Certainty is a claim that the outcome is settled. Not likely, not strongly indicated, settled. Certainty carries no number because it admits of none, and it leaves no room for the second reader in the drawing above. If the outcome is settled, the colleague at 75 is simply mistaken.

Research produces a claim with a stated belief attached to it and nothing stronger than that, so the word certainly appearing in a note is almost always doing rhetorical work rather than carrying information. Deleting the word from a sentence changes nothing. Sarvani Coatings will certainly hold its margin. Sarvani Coatings will hold its margin. The second sentence says exactly as much as the first. The word added no content. The word added volume, and volume is the written equivalent of saying something louder.

The habit matters more than a style complaint. The word does damage on the way in as well as on the way out. A settled matter does not need an observation that would break it, so a reader who writes certainly has usually stopped looking for one. A reader who writes 70 per cent has, by the act of writing a number below a hundred, admitted there is a version of the world in which this goes the other way, and has therefore left the door open to going and finding it.

Why are three likely things not a likely set of three?

Because the claim is right only if all of them hold. The requirement looks obvious written down, and it is one of the least obvious things in research when a note is being read rather than written.

Take the household version first. Someone is planning to reach a wedding on the far side of the city by seven. The auto has to turn up, the traffic has to behave, and the ceremony has to start late as it usually does. Each of those three, on its own, is a fair bet. Each one would be called likely. Yet the arrival is late often enough that the rest of the household has stopped believing the estimated times, and the reason is not that any single one of the three was misjudged. The reason is that the evening needed all three, and needing all three is a much harder thing to ask than needing any one of them.

The worked claim on Sarvani Coatings has exactly this shape. The claim rests on three conditions and is right only if every one of them holds. The gross margin holds at 46.0 per cent. Volume growth stays above the field's 4.5 per cent, having run at 6.0 per cent, a lead of 1.5 points. The 0.13 percentage point share gain repeats. A reader looks down a list of three conditions each of which sounds likely and comes away feeling the list is likely, and the arithmetic runs firmly and immediately in the other direction.

What does the arithmetic show, level by level?

If the three conditions are treated as independentIn this arithmetic, two conditions are independent when learning that one of them held says nothing at all about whether the other one did., the chance of all three holding is the three chances multiplied together. Each at 70 per cent gives 0.7 times 0.7 times 0.7. The product is 34.3 per cent, and 34.3 per cent deserves a moment. Three conditions each described out loud as likely have combined into a claim that is more likely to fail than to hold.

The obvious next question is whether 70 was an unlucky choice, and the answer is that the shape does not depend on it. Here is the same working at five other levels, and the pattern in the right hand column is the one worth memorising.

The penalty is severe at exactly the levels of belief people describe as confident, and it becomes mild only at levels of belief that research almost never has any business claiming. Raising every one of the three conditions from 70 per cent to 80 is a big move in how sure the belief feels, and it buys 16.9 points of conjunction, from 34.3 to 51.2, which is about 1.49 times the starting figure. All three would have to reach 90 before the set of three arrived at the sort of number that a single condition at 70 already sounded like.

Where does the arithmetic stop being informative?

Right here. Multiplying the three together required an unexamined assumption. The assumption is that the three conditions are unrelated. On this claim they are nothing of the sort.

Look at what the three conditions have underneath them. The margin held at 46.0 per cent, the volume lead over the field, and the share gain repeating all stand on the same unresolved question: whether the improvement came from conditions right across the field, letting makers generally raise prices faster than their inputs rose, or from Sarvani Coatings pricing well on its own account, or from a shift in mix towards industrial contracts, a shift that lifts realisation and input intensity at the same time. The published statements separate none of the three explanations. The peer evidence narrows the question and refuses to settle it. Both peers improved as well over the same two years: Kesaria Surface Solutions Limited by 3.6 points and Nandivarman Paints Limited by 2.4, against the company's 3.0. Notice in passing that 3.0 sits precisely halfway between 2.4 and 3.6. The midpoint is a coincidence of numbers written for teaching, and it is worth naming as one so that nobody reads a midpoint as though it meant something.

Conditions that share a foundation are dependentTwo conditions are dependent when they lean on something in common, so learning about one shifts what should be expected of the other.. If the question resolves the generous way, the margin holding and the share gain repeating both become likelier at once. If it resolves the other way, they weaken together. Dependence means the three tend to hold together or fail together, so the true figure sits above the product of 34.3 per cent and below the smallest of the three at 70 per cent, and nothing available here can say where inside that band it belongs.

A band is an uncomfortable place to leave a reader, and it is the correct place. Printing 34.3 per cent and stopping would replace one false precisionA figure written more exactly than the evidence underneath it can support, so the decimal places carry confidence the working never earned. with another, and that would be worse than setting out no arithmetic at all. A reader would walk away holding a wrong number instead of a vague feeling. The arithmetic still earns its keep, though, and what it earned is exactly this: it fixed the direction. Intuition said a list of likely conditions is a likely list. The multiplication says the set of three is worse than any single member of it, and dependence moves the answer back up without ever moving it above the weakest condition. Both of those are true and neither is obvious.

The band is not a fixed width either, and how it changes is worth a moment. At 70 per cent on every condition it runs 35.7 points, from 34.3 up to 70.0. With every condition pushed to 80 it narrows to 28.8 points, from 51.2 up to 80.0. At 90 per cent it is down to 17.1 points. The band is at its widest through the middle of the scale, and that is the awkward part of the shape. The amount unknown about the answer is largest exactly where the belief is most ordinary, and it shrinks only as the belief approaches a level nobody was entitled to in the first place.

So what does the conjunction actually argue for?

Not for better probability estimates. Better estimates are the natural conclusion and the wrong one. Each individual number multiplies the whole downwards, so a view needing a great many things to happen is not rescued by any amount of care over any one of them.

Holding the belief steady at 70 per cent for each condition and simply adding conditions shows the shape. Two conditions give 49.0 per cent. Three give 34.3. Six give 11.8. Twelve give 1.4 per cent. Every extra condition a claim needs multiplies its chances downwards, and the same arithmetic sits underneath the rule that a view resting on twelve conditions is usually resting on none of them. Nobody who writes a twelve variable model believes each variable individually is a long shot. Each variable looks fine on its own, and twelve fine-looking variables are exactly the situation the arithmetic is built for.

How is a belief written down so it can be checked later?

Four fields, and it stops being useful the moment any one of them is missing. A number. The specific claim the number attaches to. The date the number was written. The period the claim runs over.

The reason is mechanical rather than moral. Without the number there is nothing to compare an outcome against. Without the specific claim the writer can quietly remember having meant something adjacent that happened to come true. Without the date there is no telling whether the belief was written before or after the news that made it look good. Without the period nothing ever falls due, so the belief never becomes checkable and can be held forever at no cost. A belief missing any one of those four fields cannot be scored afterwards, and a research process that never scores its beliefs has no way of finding out that it is systematically too sure.

There is a second reason to write it down that has nothing to do with the individual call. Beliefs collected over years become a record that can be looked back through, and that record answers a question no single call can: whether the things put at seven in ten actually happen about seven times in ten. The question has a name, calibrationThe separate question of whether a person's stated numbers match reality over many calls, so the things they call seven in ten happen about seven times in ten., and it is a different question from whether any one view was good. The habit of writing a belief as a number so it can be checked later is set out at length by Annie Duke in Thinking in Bets, and the same book is careful to separate a decision from how it turned out.

How does anybody actually use this?

Three quite different people use the same arithmetic, and it is worth seeing all three because the shape of the use changes.

An analyst uses it as a drafting check before the note goes out. Count the conditions the view needs. If there are more than about three, either the extra ones are not really load bearing and should come out of the argument, or they are load bearing and the view is weaker than the prose sounds. Ravindra Setlur runs the identical count from the finance chair, before he gives guidance rather than after he reads it: guidance that needs a good monsoon, a soft input market and a competitor staying disciplined is a conjunction whether or not anybody says the word.

A research head uses the stated numbers rather than the conditions. Collected across an analyst and across years, they show whether the desk is systematically too sure, and that is a fact about the process rather than about any single view. A systematic tilt cannot be discovered any other way, and the tilt is the reason the four fields on the card matter more than they look.

And a household uses it without ever writing anything down. A monthly budget that works only if the bonus arrives, the school fee stays flat and nobody falls ill is three conditions in a conjunction. Anybody deciding anything is doing this arithmetic implicitly, and the only question is whether they have looked at the number it produces.

What does high confidence in a badly built view actually measure?

Something, but not what the holder thinks. Here is the uncomfortable answer, and it is worth saying flatly. Effort and belief rise together while evidence does not, so high confidence in a poorly built view measures how much work went into the argument rather than how much evidence supports it.

The mechanism is easy to feel. An analyst spends three weeks on a model, reads every filing, builds the segment split and argues it out with a colleague twice. At the end of those three weeks the analyst is noticeably surer than at the start. The honest question is what arrived during those three weeks that should have moved the belief. Usually the answer is very little: the filings said what they said on day one. The change is three weeks invested and a great deal of familiarity with the argument, and familiarity feels exactly like evidence from the inside.

Familiarity feeling like evidence is the reason the discipline in this sequence runs the way it does. Disconfirming evidenceThe observation written down in advance that would show the claim was wrong, covered separately in this sequence. is written down before the argument exists, precisely because it cannot be written honestly afterwards. By the time the argument is built, there are three weeks of reasons why the breaking observation would not really break anything. And the related trap sits on the other side of the outcome: resultingJudging whether a decision was good by looking at how it turned out, a trap the decision writer Annie Duke named. is judging the quality of a decision by how it turned out. Judging that way makes a well built view that failed look like bad work and a badly built view that happened to come off look like good work. Neither of those is true, and the stated number written in advance is what makes the difference visible later.