Numerical Error and Stability: When Small Mistakes Grow

Stability is whether errors already inside a calculation grow or shrink as it proceeds. Convergence is whether the answer approaches the truth as the spacing is refined. Stability and convergence are different properties, and neither one implies the other. A scheme can be perfectly stable, converge cleanly, and settle on a figure that is simply not the answer.

A stable scheme that converges cleanly on the wrong figure is not a rare accident. Refining one source of error while a second one is left untouched produces a sequence of readings that settles beautifully on the wrong number, and this is the commonest way a numerical result comes to be trusted wrongly. The arithmetic below proves it rather than asserting it, using the grid method on the standard process, checked at every step against a figure that is known exactly.

Four methods have been opened one at a time: the closed form, the sampling method, the lattice and the grid. Each was measured against the closed form of Rs 10.450584/- for the at-the-money contract on the standard process, and each was shown to go wrong in its own particular way. What all four have in common is not a method at all. The common thread is the pair of questions to ask of any method before believing what it prints.

Convergence vs Stability: which of the two is a scheme failing?

Put a one kilogram weight on a kitchen scale. Two things can be wrong with the scale, and they are unrelated.

The first fault: the reading creeps. The weight goes down on the pan and the instrument reads 1.00, then 1.03, then 1.09, then 1.21, climbing away from itself while nothing on the pan has changed. Some tiny disturbance inside the mechanism is being amplified rather than absorbed. The instrument is unstable.

The second fault: the reading is rock steady, repeatable to three decimal places, and it says 1.05 kilograms. Nothing creeps. The weight goes on ten times and the reading is 1.05 ten times over. The instrument is entirely well behaved and entirely wrong, and no amount of staring at its own readings will ever reveal as much. Only a weight whose mass is already known catches it.

The creeping reading and the steady wrong reading are exactly what stability and convergence are about, and a numerical scheme can have either fault without the other. StabilityWhether errors already sitting inside a calculation grow or shrink as the calculation proceeds from one step to the next. asks whether the disturbances already present at one step get bigger or smaller by the next step. ConvergenceWhether the answer approaches the truth as the spacing of the calculation is made finer and finer. asks where the whole calculation is heading as its spacings are made finer.

Look at that top right cell for a moment. The other three failures announce themselves. A price of minus Rs 495.898876/- for a contract that cannot pay less than nothing is not a subtle problem, and a reading of Rs 5.541888/- against a cluster of neighbouring readings near Rs 10.40/- is not subtle either. But Rs 10.394588/- looks exactly like an answer. The figure is the right size, it has the right number of digits, and it agrees with the reading before it and the reading before that. Nothing about it looks wrong. Nothing about it is wrong, except the one thing that matters.

What makes a scheme unstable?

The grid method steps a whole row of values backward through time. Each new value at a point is built out of the three values around that point on the previous row. Writing down what happens to a disturbance sitting on that previous row gives the answer directly: the disturbance is multiplied by something, and the whole question is whether that something is bigger or smaller than one in size.

Take the roughest possible disturbance, one that flips sign at every neighbouring point. Feed it through one step of the explicit update and it comes out multiplied by a factor that depends only on the volatility, the two spacings and the rate.

Put the locked numbers in. The volatility is 0.20, so the variance rate is 0.04 exactly. At a space step of 0.05 the condition reads \(n \ge 0.04 / 0.0025\), giving sixteen. Sixteen time steps is not a rough guideline here; it is exactly where the factor reaches one, and the grid is usable at sixteen steps and useless at fifteen.

Notice how violent the failure is. Fourteen steps does not give a slightly worse answer than sixteen. Fourteen steps multiplies the roughest disturbance by 1.289286 fourteen times over, a growth of about thirty-five, and thirteen steps multiplies by 1.465385 thirteen times, a growth of about a hundred and forty. Ten steps grows it by roughly two thousand seven hundred, and the price comes out at minus Rs 495.898876/-. Instability is multiplicative, so it does not degrade an answer gently; it destroys it, and the destruction accelerates the further below the limit the calculation goes.

There is one genuinely awkward reading in that set. At fourteen steps the grid prints Rs 5.541888/-. Rs 5.541888/- is a positive number of a plausible size for a price. An analyst who had never computed the closed form might have accepted it. Only the neighbours give it away.

What makes a scheme converge?

Convergence is a completely separate question, and answering it starts somewhere else entirely: with what the scheme threw away when it was written down.

Every discrete scheme replaces something continuous with something finite. A derivative becomes a difference between neighbouring values. An integral becomes a sum. An expectation becomes an average over a finite set of outcomes. Each of those replacements leaves a remainder, and the size of that remainder is a power of the spacing that was used. The remainders are the whole content of convergence. The scheme converges when every remainder it created goes to nought as the spacings do.

For the grid on the standard process there are two spacings and therefore two remainders. Stepping forward in time with a single forward difference leaves a remainder proportional to the time step itself. Because the odd-powered terms cancel between the point on the left and the point on the right, approximating the two space derivatives with centred differences leaves a remainder proportional to the square of the space step.

There is a third condition beyond consistency and stability, and it ties the other two together. A scheme that is consistent, meaning every remainder goes to nought as the spacings do, and also stable, meaning disturbances do not grow, is convergent. Without stability, consistency alone buys nothing. The calculation never survives long enough to reach its own limit. Without consistency, stability alone buys nothing either. The calculation proceeds serenely toward a limit that was never the right one.

Stability without consistency is not a curiosity. Stability makes a promise about the steps and no promise at all about the destination, so a scheme can be flawlessly stable and still be heading somewhere other than the truth.

Why is converging not the same as being right?

Here is the everyday version. A room is being measured with a folding ruler held against the wall, and the length is wanted to the millimetre. The ruler looks too coarsely marked, so it is replaced by one marked in half millimetres, then quarter millimetres, then eighths. The reading stops changing. The steady reading establishes one thing: the markings on the ruler are no longer what limits the measurement.

The same steady reading says nothing whatever about the ruler having been held at a slight slant the entire time.

RefinementMaking one of the spacings in a calculation smaller. Shrinking it removes only the error belonging to that spacing and leaves every other error exactly where it was. along one axis buys information about that axis and nothing else. The statement sounds obvious written down. In practice it is not obvious, and the reason is how the evidence presents itself. What appears is a column of numbers that stops moving, and a column of numbers that stops moving feels like an answer arriving. The stillness feels like the calculation announcing that it is done.

The stillness says something much narrower. The stillness says that the parameter that was changed has stopped mattering. A settled sequenceA run of readings that agree with one another closely. Such a run is evidence about the spacing that was refined and about nothing else. is evidence about the parameter that was moved, and about no other parameter anywhere in the calculation.

The wall in that picture has a number attached to it, and the number can be computed rather than guessed. Hold the space step at 0.05 and push the time step toward nought, and the grid heads for Rs 10.393513/-. Rs 10.393513/- is the value the grid has when its time error is gone entirely and only its space error remains. The reading sits Rs 0.057070/- below the closed form, and the time steps were never what was holding it there, so no amount of extra time steps will ever move it.

What happens when the time step alone is refined?

Now the worked instance, and it is the most ordinary thing anybody does with a numerical method. There is a grid. The time step is suspected of being too coarse. The step is refined, several times over, and the answer is watched.

The space step is held at 0.05 throughout, putting the stability limit at sixteen time steps, so the sequence starts exactly there and works upward.

The second column read on its own is all there would be without a closed form to check against. Rs 10.404265/-, Rs 10.397779/-, Rs 10.394588/-. The three readings sit inside a single paisa of one another. The last two differ by Rs 0.003191/-, a third of a paisa. Each move is smaller than the move before it. There is no oscillation, no wobble, no sign of anything wrong. By every internal test available, this sequence has converged.

The third column tells a different story. The error at sixteen steps is minus Rs 0.119311/-. At fifty steps it is minus Rs 0.035558/-, the smallest error in the whole table. And then, as refinement continues, it gets worse: minus Rs 0.046319/-, minus Rs 0.052805/-, minus Rs 0.055996/-. Refining the time step past fifty makes the answer less accurate, monotonically, and it will keep doing so however far it is pushed.

The reversal is not a paradox and it is not a bug. At fifty steps the grid has a positive time error and a negative space error, and they happen to be partly cancelling. Refining the time step removes the positive one and leaves the negative one standing alone. The reading was closer to the truth when it was wrong in two directions at once.

What happens when both spacings are refined together?

Same scheme, same process, same contract. The only change is that the space step now moves as well, and the time steps are raised alongside it to keep the calculation on the stable side of the limit. Each rung halves the space step, and halving quadruples the number of time steps the stability condition demands. Here the time steps are set at sixteen times that minimum on every rung, so the comparison is like for like.

The distances fall 0.220636, 0.052872, 0.013100, 0.003268. Dividing each by the next gives 4.1730, 4.0360 and 4.0086. Halving the space step quarters the error, and the ratios are tightening toward exactly four as the higher-order terms fade. Ratios tightening on four are the signature of a second-order method behaving as it should.

A genuine order of convergenceHow fast the error falls as a spacing is refined. Second order means halving the spacing quarters the error, and the grid behaves exactly so. looks like, and the table above is worth setting beside it. There the readings settled and the error grew. Here the readings move a long way on every rung and the error collapses. A sequence that is still moving substantially can be far healthier than one that has gone quiet.

The accuracy is not free. Halving the space step doubles the number of grid points and quadruples the number of time steps required for stability, so the arithmetic multiplies by roughly eight. Counting grid points multiplied by time steps: 1,344 units on the first rung, 10,496 on the second, 82,944 on the third and 6,59,456 on the fourth. The ratios of those counts are 7.81, 7.90 and 7.95, tightening toward eight.

How many sources of error does one method have?

More than one, always, and that is the sentence to carry away. A source of errorOne thing the method has approximated. Every method compared here has approximated more than one thing, and each approximation leaves its own remainder. is simply one thing the method replaced with something finite, and counting them is the discipline that would have caught the failure above before it happened.

The grid has three. The scheme approximated time with a step, so there is a time error. The scheme approximated the space derivatives with differences, so there is a space error. And it cut off the range of values it covers at a highest and a lowest point instead of running out to nought and infinity, so there is an error from the cut.

The error from the cut is worth measuring rather than assuming away, and here it is genuinely negligible. Running the grid over a range from Rs 36.79/- to Rs 271.83/- gives Rs 10.394588053/-. Widening it enormously, to a range from Rs 13.53/- to Rs 738.91/-, gives Rs 10.394588055/-. The difference is two thousand millionths of a paisa. The cut is not the problem here. Knowing which source is innocent is worth as much as knowing which is guilty, and either one identifies which of the three deserves the effort.

Now split the total error at the finest reading of the first table into its parts. At a space step of 0.05 and a thousand time steps the grid printed Rs 10.394588/-, a distance of Rs 0.055996/- from the truth. Where did that come from?

Ninety eight per cent of the remaining error belongs to the one spacing that was never touched, and the effort went entirely into the spacing that was already contributing almost nothing. The mismatch is not bad luck. Refining along one axis has that consequence with arithmetic certainty, and anyone who had written down the list of three would have seen it in advance.

How to Check Monte Carlo Convergence, and every other method's convergence too?

A sampling method looks like a special case and is not one. Its accuracy scales with the square root of the number of outcomes, a relationship set out under the Monte Carlo method. The answer there never quite stops moving at all, so a check based on watching the answer stop moving is even weaker on a sampling method than on a grid.

For the at-the-money contract on the standard process the spread of the discounted payoff is 14.719404 in rupees, a computed property of that contract. Ten thousand outcomes therefore give a band of about Rs 0.147194/-. To pull that band inside a paisa takes about 21,66,609 outcomes. Every one of those outcomes narrows the band and does nothing whatever to any error introduced by how the path was stepped. The stepping error is a separate source sitting quietly outside the band.

So the check cannot be internal, on any method. Here is what internal evidence can and cannot establish, and it applies identically to the grid, the lattice and the sampling method.

So the benchmark checkRunning the method on a case whose answer is already known exactly, and comparing. The only check that can detect an error present at every setting. is not one option among several. The benchmark check is the only check that can see this failure at all. All four methods were measured against Rs 10.450584/- for exactly that reason: the lattice at twelve steps prints Rs 10.285850/-, and slicing the probability scale into two hundred midpoints prints Rs 10.432104/-, and neither of those distances could have been known from the method alone.

The practical procedure that follows is short. The analyst finds a case with a known answer that exercises the same machinery, runs the method on it and records the distance. Every spacing and every approximation the method contains is listed, and each is refined separately to show how much it contributes. Refining them together then confirms whether the distance falls at the order expected. If it falls more slowly than the order predicts, something not on the list is contributing.

Who actually does this check, and what do they do with it?

Anybody who has written a numerical calculation and now has to decide whether to believe what it printed. The list runs from a student rebuilding a published figure, to somebody reproducing a result from a working paper, to somebody rewriting an old calculation in a new language and needing to know the two versions agree for the right reason rather than by luck.

All three do the same thing, and it is what a workshop does with a set of scales. Nobody trusts a scale because its readings are repeatable. The workshop trusts a scale because somebody put a known weight on it. The known weight does not establish that the scale is good at everything; it establishes that the scale is right at that weight, on that day. Everything else is inference.

The habit that follows is small and worth naming precisely. Before running anything, write down the list of things the method has approximated. Not one thing. The whole list. For the grid that is three items, and the exercise takes a minute. Then, whenever a sequence of readings stops moving, the list answers which single item has just been proved to have stopped mattering, and how many items remain on it.

The pacing example makes the shape of the mistake vivid. Measuring a coastline by walking it with a metre stick gives one length. Because the shorter stick follows detail the longer one cut across, walking it again with a stick half as long gives a longer one. Halving continues and the number keeps growing rather than settling. The coastline is a case where refining one thing does not settle at all. The grid did the opposite, settling promptly and convincingly, and the settling was the misleading part.

A careful reader will spot one last thing about the arithmetic. The closed form to nine decimals is Rs 10.450583572/-, and it is quoted throughout as Rs 10.450584/-. Every distance in the tables is taken from the nine decimal figure rather than the rounded one. A subtraction done on the rounded figures can therefore disagree with the tables in the sixth decimal place. The distance at a space step of 0.025 is Rs 0.013100/- and not Rs 0.013101/- for exactly that reason.

The mathematics carries no jurisdiction. A stability condition is a statement about arithmetic, and it holds identically wherever the calculation is run, so no rule anywhere changes a single figure of it. The conduct expected of anyone presenting a computed number as a basis for a decision does vary by place, and that duty is covered under professional conduct.