Scenario Analysis vs Stress Testing: Two Different Questions

Scenario analysis and stress testing are different questions, not different severities. Scenario analysis asks what a coherent set of conditions would do, and those conditions need not be bad. A stress test is a scenario picked because it is severe, so its answer is a distance from a threshold. A reverse stress test fixes the outcome first and solves for the conditions. Three questions, one machinery, and Vindhya Commercial Bank Limited, invented, runs all three.

The two names are used as if one were a polite version of the other. On the surface the two exercises look identical: somebody writes down a set of conditions, somebody runs them through a model, and a number comes out the other end. So a committee paper says scenario analysis when the numbers are mild and stress testing when they are frightening, and nobody objects. The machinery really is the same. The question the machinery has been pointed at is not the same. Once that question can be heard, the two stop being interchangeable and a third exercise appears that is not on the same axis at all.

Are scenario analysis and stress testing the same activity under two names?

One sorting rule is the reason the distinction keeps collapsing, and almost everybody uses it without noticing. Start there. The rule is: sort by how bad the numbers are. Mild conditions are scenario analysis, harsh conditions are stress testing, and very harsh conditions are severe stress testing. The rule is easy and the rule is natural, and it produces a set of exercises that differ only in one setting. Sorted by severityHow hard a scenario is, which is a setting on a dial rather than a category of exercise., a moderate downturn and a severe one are the same exercise run twice, and calling one of them by a different name adds nothing that the numbers did not already say.

Here is the everyday version, and it is worth holding on to because the bank version is exactly this shape. A household running on one salary sits down in January and imagines next year. First it imagines the salary rising by the usual amount, the rent rising a little more, and the school fee going up once. The salary version is one coherent picture of a year, and nothing bad happens in it. Then the same household imagines the earner out of work for four months with the same rent and the same fee still due. The out-of-work version is a second coherent picture of a year, and something quite bad happens in it. Both are pictures of a year. Neither is more of an exercise than the other. And then somebody in the room asks a third question that is not a picture at all: how long would the earner have to be out of work before the rent could not be paid? The third question does not describe a year. The third question describes a line, and asks what would have to happen to reach it.

Something has changed between the second question and the third. The first two questions differ in how bad they are. The third differs in which end is held. A severity is not an input to the third question at all, so turning the dial further never arrives at it. The input is an outcome. The shape of the whole comparison is this: two of the three exercises sit on a severity dial, and the third one is the dial read backwards.

What goes in, what comes out, and what is left in hand?

The severity rule gives way to a rule that reads the exercise rather than the numbers. Three questions can be put to any result. What went in? What came out? And what kind of statement is the result? The three answers separate the three exercises cleanly, every time, and they do it without anybody having to agree on what counts as severe.

Conditions in and an outcome out is scenario analysis. Severe conditions in and a distance from a threshold out is a stress test. An outcome in and conditions out is a reverse stress test. Only the last of the three has the arrow pointing the other way, and that is a difference in kind rather than in degree. Read the three rows of the drawing below from left to right for the first two and from right to left for the third, and the reversal is the only thing that has changed about the picture.

The middle box is identical in all three rows, and that is deliberate. The model does not know which exercise it is being used for. Nothing about the balance sheet changes, nothing about the Rs 60,000 crore of risk weighted assets changes, and nothing about the Rs 9,000 crore of total capital changes. The exercise changes which quantity is treated as known and which is treated as the answer. The choice belongs to the question rather than to the machine.

What is scenario analysis actually asking?

Scenario analysisRunning a coherent set of conditions through an institution to see what they do, with no requirement that the conditions be bad. is the act of writing down a set of conditions that hang together, running them through the institution, and looking at what comes out. The whole weight sits on the word that means hanging together. A set of conditions is coherentA property of a usable scenario: the assumptions hang together and do not contradict one another. when its assumptions do not contradict one another. A picture in which industrial demand collapses, borrowers stop paying, and deposit rates fall to nothing at the same time is not a scenario, it is a list. A scenario is a story with numbers attached, and a set of conditions that cannot be told as a story in a paragraph is a spreadsheet of unrelated shocks wearing a scenario name.

The coherence requirement gives a practical test for whether a given document is a scenario. A usable scenario document has five fixed parts, and the fifth is the one that goes missing. A scenario document needs the story of what happens, the variables it moves, the size of each move, the period over which it happens, and a statement of what is being held constant. Everybody writes the first four. Almost nobody writes the fifth, and the fifth is where the argument later lives. Two people reading the same result will disagree precisely at the assumption neither of them was told was being made.

Does a scenario have to be bad?

No, and the clearest proof of that is sitting in every institution already, wearing a different name. The bank's plan for the year is a set of assumptions about a world: bad loans at 3.0 per cent of gross advances, a credit cost of Rs 648 crore, a net interest margin of 3.00 per cent, and a total capital ratio of 15.0 per cent. The plan is coherent, the plan moves the same variables as every other scenario, and the plan is run through exactly the same machinery. By the definition above it is a scenario, and inside the bank's scenario set it is numbered ST1.

Nobody calls the plan a scenario. The plan is called the plan, or the budget, or the baselineThe institution's own plan, run through the same machinery, so that everything else has something to be compared with., and each of those names quietly does the same damage: it suggests the plan is what will happen and the scenarios are what might. The plan is an assumption about a world exactly as much as the severe scenario is, and calling it the baseline hides that from the people reading it. A favourable scenario is a real member of the set for the same reason: it shows what the institution looks like when things go mildly right. Nobody has that information if the only pictures on the table are bad ones.

What makes something a stress test rather than a scenario?

A stress testA scenario chosen because it is severe, so that the answer is a distance from a threshold rather than a description of a world. is a scenario chosen because it is severe, and the choosing is the whole difference. Nobody runs a severe scenario to find out what a severe world looks like. Everybody already knows a severe world looks bad. A severe scenario is run to find out whether the institution is still standing at the end of it, and how much room was left. Because the conditions were picked for severity, the interesting output stops being a description of a world and becomes a distance from a line.

Look at what the invented bank actually gets back from scenario ST3. Gross bad loans reach 6.4 per cent of gross advances. The credit cost of ST3 is Rs 1,944 crore against the Rs 648 crore in the plan, exactly 3.0 times. The net interest margin falls to 2.64 per cent, and the total capital ratio comes out at 11.8 per cent. Now, none of those four numbers is the answer. The answer is the gap between 11.8 per cent and the bank's own internal floor of 11.0 per cent. The gap is 0.8 percentage points, and 0.8 points on Rs 60,000 crore of risk weighted assets is Rs 480 crore of capital. Somebody chose a threshold before running anything, and only that choice makes the Rs 480 crore exist. The Rs 480 crore is what gets reported and what the committee argues about.

Two things follow from that and both are easy to miss. The first is that a stress result is meaningless without the threshold, so an institution with no stated floor cannot run a stress test at all, only a severe scenario. The second is a collision the bank arithmetic sets up, and it is worth naming carefully. The credit cost of ST3 is Rs 1,944 crore. The capital consumed by ST3 is Rs 1,920 crore. Two different objects, Rs 24 crore apart, both belonging to the same scenario. Written as a bare number, either will be taken for the other.

Where does the reverse stress test sit against both of them?

A reverse stress testAn exercise whose input is an outcome and whose output is a scenario. is not a harder stress test. Its input is an outcome and its output is a scenario. The arrow runs the exact reverse of both exercises above. At the invented bank the outcome is fixed first: the total capital ratio reaching the bank's own internal floor of 11.0 per cent. The exercise then solves for what would have to happen to get there.

The arithmetic runs in one line. Total capital is Rs 9,000 crore and risk weighted assets are Rs 60,000 crore, so an 11.0 per cent ratio needs Rs 6,600 crore of capital, and the loss that takes the bank there is Rs 9,000 crore less Rs 6,600 crore, being Rs 2,400 crore. At the bank's own assumed loss given default of 40.0 per cent, producing a Rs 2,400 crore loss needs Rs 6,000 crore of exposure to default. Gross advances are Rs 58,800 crore, so Rs 6,000 crore of them is 10.2 per cent. And now the sentence the exercise exists to produce: the infrastructure and power sector at this bank stands at Rs 7,644 crore, so Rs 6,000 crore is 78.5 per cent of one sector. One sector, already reported as over its limit, is large enough on its own to take this bank to its capital floor without anything else going wrong.

Set that against what the forward programme said. The forward programme ran ST3 and reported Rs 480 crore of headroom, and headroom reads as comfort. The failure is the input to the reverse test, so the reverse test cannot report comfort and never can. A forward exercise is allowed to come back and say the institution survived. A reverse exercise has already assumed it did not, and the only thing it can hand back is the route.

How is one told from the others, from the output alone?

A result often arrives with no covering note. The shape of the output is then the fastest test available. A capital ratio or a loss figure is what a forward exercise produces. A description of what would have to happen, naming an exposure or a sector, is what only the backward one produces. Sorting by the shape of the output removes any need to argue about whether the numbers were severe enough to earn a name.

Run that test across the four scenarios the invented bank actually has in the year and the sorting falls out cleanly. ST1 and ST2 are scenario analysis: neither was chosen for severity, both were run to see what happens, and one of them is a picture in which nothing bad occurs at all. ST3 is a stress test, chosen because it is severe, and it answers with a distance. ST4 is a reverse stress test and is not on that axis at all.

Notice the third column of the last row. The ST4 row is the only one where what went in is a number the bank chose as a limit rather than a condition it imagined, and the only one where what came out is a story rather than a ratio. Everything else about the four rows is the same exercise.

Can one dial be read in two directions?

Here is the demonstration that settles the difference in kind, and it uses nothing but the bank's own locked figures. Put all four scenarios on a single quantity: the capital consumedThe fall in a capital ratio expressed in rupees, computed as the fall in ratio points times risk weighted assets.. Capital consumed is the fall in the total capital ratio times the Rs 60,000 crore of risk weighted assets. ST1 consumes nothing. ST2 falls 1.6 points and consumes Rs 960 crore. ST3 falls 3.2 points and consumes Rs 1,920 crore. ST4 falls 4.0 points and consumes Rs 2,400 crore. As shares of the bank's Rs 2,400 crore of loss capacity, the four are nought, 40.0, 80.0 and 100.0 per cent.

The spacing is the finding: the steps are 1.6 points, 1.6 points and 0.8 points, so the last step to the floor is half the size of the two before it, and the bank's own severe scenario already stands 80.0 per cent of the way there. A committee shown the ST3 result alone hears Rs 480 crore of headroom and reads it as room. The same committee shown the spacing hears that four fifths of the distance has already been travelled by a scenario the bank considers plausible.

Now the part that makes the dial legitimate rather than a teaching convenience. The dial is one straight line: total capital of Rs 9,000 crore less the loss, divided by Rs 60,000 crore of risk weighted assets. The three losses fed into it return the three locked capital ratios of ST2, ST3 and ST4 exactly, to the decimal, without anybody adjusting anything. A line that reproduces three independently stated results can be read in either direction.

What does an institution lose by running only one of the three?

The commonest arrangement is not that an institution refuses to run the other two. The arrangement is that an institution treats all three as one exercise at three settings, runs the middle setting, and calls the whole activity stress testing. Nobody decides to skip anything. The programme simply answers the question it was asked and stops. A well-run programme does exactly that. The three costs are different in kind. Watch what each omission costs.

The dates are what make the second omission undeniable, so they deserve setting out. Nothing on the timeline below was hidden, nobody withheld anything, and no exposure was unknown. The sector had crossed its cap in month 5. A committee had looked at it in month 6, accepted it as a temporary excess, and set a remediation plan running to month 18. The reverse stress test arrived at month 12 and pointed straight at it. The reverse test added a connection rather than an exposure. The forward programme had already answered its own question and stopped, and nobody had gone looking for the connection.

Which of the three does an institution need?

The three do not substitute for each other, so the question is badly posed. Each one fails in a way the other two cannot detect. Calling them complementary is a weaker statement than that. Scenario analysis cannot say whether the institution survives, a stress test cannot say what would actually cause the failure, and a reverse stress test cannot say what the plan assumes, so an institution running one of the three has answered a third of the question.

Work through why each blindness is structural rather than accidental. Scenario analysis cannot report survival because nothing in it was chosen to be hard enough to threaten anything; running a mild scenario and finding the institution intact is not evidence, it is arithmetic. A stress test cannot report the cause of failure because somebody chose the scenario before it ran, so the answer can only ever be about the world that person imagined. And a reverse stress test never runs the plan, so it cannot report what the plan assumes. The reverse test starts at the outcome and works outward, and the plan is not on its path at all.

Who actually reads these three results, and what do they do with them?

Three different people pick up these outputs, and each is looking for something different. Keeping the three exercises distinguishable in the document that carries them is what lets each reader find what they came for.

An independent director on the board risk committee reads the stress result for the distance and nothing else. Sunanda Ravikumar, the chief risk officer at the bank, can put four numbers in front of that director. Only the 0.8 percentage points is a quantity a decision can be made about, so only the 0.8 percentage points will be held on to. Her useful move is to put the spacing beside it: the first two steps down the dial are Rs 960 crore each and the last is Rs 480 crore, so the severe scenario is not comfortably short of the floor, it is four fifths of the way to it. A meeting shown the spacing is a different meeting from one shown only the headroom.

A credit analyst at another institution, looking at this bank as a counterparty rather than as an employer, reads the same three outputs for something else entirely: whether a reverse test was run at all. A published forward result tells the analyst what the bank chose to imagine. A reverse result, if it exists, tells the analyst what the bank found when it stopped choosing. An institution that publishes only forward results has disclosed its imagination, and an institution that publishes a reverse result has disclosed its balance sheet.

And a household or a small business owner does exactly this without any of the vocabulary. The plan is the household budget for the year. The stress test asks whether the household can still pay the rent with the earner out of work for four months. A threshold sits inside that question, so the answer is a distance. And the reverse test is the question a food stall owner outside a single office building asks without ever writing it down: how many weeks of that building being closed would it take before I cannot restock? The stall owner's question names the office building. The other two never do, and naming the thing is the whole return on running it.