Numerical Methods: Solving What Cannot Be Solved Exactly

No formula can be rearranged to produce the answer, so a numerical method finds it by approximation. The Nakshatra ladder starts at Rs 100/-, grows by a monthly rate and has Rs 2/- taken off it every month. Ask which rate carries it to Rs 120/- in twelve months and that rate sits inside a twelfth power and under a division at the same time. Nothing rearranges, so the rate has to be found by search instead.

Most of school arithmetic trains one reflex: move the unknown to one side and read it off. The reflex works so consistently that when it stops working, the natural conclusion is that a step has been forgotten. It has not. There is a large and ordinary class of questions where the unknown genuinely cannot be moved to one side, and the working answer to all of them looks the same: guess, check, and improve the guess in a disciplined way until the remaining error is smaller than anyone need care about.

Guessing and improving in a disciplined way is a numerical method, and a numerical method is not a lesser kind of answer. The accuracy of an approximation is chosen and paid for rather than handed over by the question, so an answer produced by approximation can be more accurate than one produced by algebra. Three such methods sit side by side below, on one arithmetic rule small enough to hold in the head, and they differ in what each needs, what each gives back, and what each costs.

What arrives already settled?

Three things arrive finished. First, one complete numerical method, built end to end earlier: write down a shape for what a month can do, generate a very large number of invented twelve month histories from it, and count the share that finished below where they began. Second, what a standard errorHow far a figure built from a limited number of draws usually lands from the quantity it was built to estimate. More draws pull it in, and they pull it in slowly. is, and why a lone figure with nothing attached to it is an incomplete answer rather than a confident one. Third, that an invented assumption can be enumerated exactly, so an estimate here can always be set beside the truth it is estimating.

The comparison is what is new. How a rule is repeated until a condition is met is covered separately. Choosing between the three methods, and paying for a wrong choice, is the rest of the subject.

What is a numerical method, and when is one needed?

A numerical method is a rule for getting closer to an answer, applied over and over, until the remaining distance is small enough for the purpose. The definition is the whole of it. One method separates itself from another by where it starts, what its rule does, and how quickly the distance shrinks.

One is needed exactly when no closed formA finished expression that can be written down and worked out in one pass, with nothing to repeat and nothing to guess at. exists for the quantity wanted. A closed form is the comfortable case: an expression written down once, evaluated once, and finished with. Interest on a fixed sum for a fixed number of years has one. The area of a circle has one. A great many perfectly ordinary questions do not have one, and no amount of skill at rearrangingShifting an equation about until the unknown quantity stands alone on one side of the equals sign. will produce it. The absence is a property of the question rather than a gap in the person asking it.

Here is the everyday version. A household knows what it can put aside each month and knows the shopkeeper's total charge for a fridge over ten instalments. Ask what rate of interest is buried in those instalments and there is no line of algebra that hands it over. The household, and the lender in a back office with better tools, try a rate, see whether the instalments come out too high or too low, and adjust. Trying and adjusting is a numerical method run by hand, by somebody who has never heard the phrase.

Where exactly does the algebra stop, in a case that can be watched?

Take one rule, small enough to state in a sentence. Start at Rs 100/-. Each month, multiply by one plus a monthly rate, then subtract Rs 2/-. Do that twelve times. Call it the Nakshatra ladder, an invented arithmetic rule written down for teaching. The ladder is not a product, a contract or an instrument of any kind: nothing is bought, nothing is sold, nobody puts anything in and nobody takes anything out. The ladder is twelve lines of arithmetic and a name.

The Rs 100/- it starts from is the starting price of the Nakshatra unit, an invented traded unit these notes have used from the beginning. The Rs 2/- is a standing chargeA fixed amount taken off every month whatever else happened that month. The charge does not scale with anything.: fixed, monthly, indifferent to what the rate did.

Now the case that behaves. Ask at which monthly rate the ladder never moves at all, and one line of algebra answers it. If the reading is to come back to Rs 100/- after the multiplication and the subtraction, then the growth in the month has to be exactly Rs 2/-, so the rate has to be Rs 2/- over Rs 100/-, or 2.00 per cent a month. At 2.00 per cent the ladder reads exactly Rs 100.0000/- in every one of its twelve months, and that is a closed form: it was written down once and evaluated once.

Now change one number in the question and watch the algebra fail. Instead of asking which rate holds the ladder still, ask which rate carries it to Rs 120/- after twelve months. Write the twelfth month reading out in full and two things happen at once. The growth compounds on itself, so the rate appears inside a twelfth power. The twelve standing charges accumulate at a size that depends on the rate they were growing against, so the rate also appears underneath a division. No rearrangement takes the rate out of both places at the same time, and that is a fact about the question rather than a shortcoming of whoever is holding the pen.

What does the same rule look like worked forwards and worked backwards?

Forwards, the ladder needs no method at all. Pick a rate, do twelve lines of arithmetic, read the answer. Five rates give five readings, and the whole table below took less effort than reading this sentence.

Backwards is a different animal. The question is now which rate lands the twelfth month on Rs 120/-, and there is no line to write. The forward table has already given something worth noticing before any method is chosen. Rs 114.1920/- at 3.00 per cent is below Rs 120/-. Rs 130.0516/- at 4.00 per cent is above it. The reading rises without a single reversal as the rate rises. The three readings together are a bracketTwo values the answer is confidently taken to lie between, one on either side of it.: the answer is somewhere between 3.00 and 4.00 per cent, and it was obtained for free from arithmetic that was being done anyway.

From there, halving the bracket repeatedly closes on 3.379155 per cent a month, at which the ladder reaches Rs 120/- to nine decimal places. The step by step working of that halving is covered separately. The bracket existed, and a bracket is the entry ticket for one whole class of method.

Where these readings come from. Not from a market, and not from a book. The Nakshatra ladder is a rule of arithmetic printed in full a few paragraphs above, and each reading came out of running that rule twelve times at one stated monthly rate. Pick any row of the table and redo it yourself: Rs 100/-, multiply, take off Rs 2/-, twelve times over.

What do all numerical methods have in common?

Strip away the names and every one of them has the same four parts, in the same order.

1. A starting position. A bracket the answer is confidently taken to sit inside, a shape already written down, or a record already held. Something to begin from.
2. A rule that produces a new position from the current one. Halve the bracket. Draw one more future. Replay one more stretch of the record.
3. A stopping condition. A statement, decided in advance, of what would end the work.
4. An error agreed in advance. The distance from the true answer that has been decided to be small enough to live with.

The first three are arithmetic. Somebody could write them into a machine and never think about them again. The fourth is a judgement, and almost every argument anyone has ever had about a numerical method is an argument about the fourth part wearing the clothes of the first three. The arithmetic does not know the purpose the answer will serve, so the arithmetic cannot say how close is close enough. A tolerance that counts as close enough for a rough sizing is embarrassing for a settlement figure, and no amount of extra computing resolves the disagreement.

What are the three kinds of numerical method?

Three kinds turn up over and over, and it is worth having plain names for them. Call them searching, drawing and replaying.

Searching starts with a bracket around an answer it knows is inside, and narrows it. On the ladder, a bracket of 2.00 to 6.00 per cent halves at every step and closes on 3.379155 per cent a month. The answer it produces is a single fixed number, and there is no randomness anywhere in the process: the same bracket and the same rule give the same digits every time, on any machine, in any year.

Drawing starts with a shape already written down, generates a great many made up outcomes from that shape, and counts them. On the Nakshatra unit's twelve month question, 10,000 drawn paths from seedThe number a stated arithmetic rule is started from, which fixes every value it goes on to produce. Naming it is what lets somebody else land on the same digits. 12345 said that 24.5300 per cent of them ended below the Rs 100/- they started at. Because the shape was invented rather than measured, the exact answer can be worked out by weighing all 1,820 possible combinations and it is 24.6562 per cent, so for once the estimate can be printed next to the truth. Drawing returns not a number but a number with a width: a share, plus a statement of how far out it might be.

Replaying assumes no shape at all. Replaying takes a record already held and reuses it. The fifty month record of the Nakshatra unit, carried in from earlier, holds fourteen falling months out of fifty. Fourteen in fifty is 28.00 per cent, and the truth underneath it is 26.00 per cent. Nothing was assumed and nothing was invented, and the price of that is a ceiling: replaying can never produce a month the record does not contain, and it inherits every accident of the record it was given.

Searching, drawing and replaying are three answers to three different kinds of question, not three tools competing for one job. Searching answers what value makes this come out right. Drawing answers what fraction of possible outcomes do this. Replaying answers what would have happened had this repeated. Asking one of them the other one's question is the whole of the failure set out at the end of this guide.

How do the three kinds differ, criterion by criterion?

Five questions sort them completely: what each must be given, what each returns, how fast its error falls, what makes it fail, and whether its answer repeats exactly.

The third row is the one that decides which method is affordable. Two of the three have an error that can be bought down by working harder, and they charge wildly different prices for it. The third does not offer the trade at all: a record of fifty months stays a record of fifty months no matter how long anyone sits with it.

How fast does each kind close in on an answer?

The two speeds are usually taught as though they were variations on one theme. Halving a bracket and halving a drawn error are nothing of the kind.

A bracket that halves at every step is a factor of two per step. Sixteen steps take the bracket on the ladder from 4.00 percentage points wide down to about 0.00006 of a percentage point, and twenty two steps take it below one hundred millionth. Each step is one pass of twelve lines of arithmetic. The whole search could be done on paper in an afternoon.

A drawn error also halves. The error falls with the square root of the count, so the price of each halving is four times the draws. The square root here is the same law that governs a standard error, established earlier. So to buy the same sixteen halvings, the number of draws has to be multiplied by four sixteen times over. Four multiplied by itself sixteen times is 4,29,49,67,296. Drawing is not the more powerful method; it is the method for when there is nothing to bracket, and that number is precisely what having nothing to bracket costs.

How is a numerical answer checked when there is nothing to check it against?

Here is the awkward part of the whole business. The reason for running a numerical method is that the answer was unavailable by any other route. So when the method hands back 3.379155 per cent, there is by construction nothing to compare it with, and a wrong implementation produces a wrong number that looks exactly as confident as a right one.

The way out is to aim the method first at a question already answerable by another route, and only afterwards at the one that is not. The ladder has one case with a closed form, so it supplies that check for nothing. The same search rule, given a bracket of 1.00 to 4.00 per cent and a target of Rs 100/-, returns 2.000000 per cent. The closed form, worked in a single line, says the flat rate is Rs 2/- over Rs 100/-, or exactly 2.00 per cent. Search and closed form agree to fifteen decimal places, and only after that does the same rule get pointed at the Rs 120/- target that nobody can check.

Skipping this step is how a broken implementation produces confident wrong numbers for years. Nothing about a wrong answer announces itself: it has the right number of digits, it arrives in the right units, and it sits in the right cell of the report. A check against a known case is the only thing that would have caught it, and it costs one extra run.

Which one should be reached for, and on what grounds?

The choice is settled by one question asked before any method is picked: is what I want a single number, or a share over many outcomes?

If the answer is a single number that can be bracketed, the method is searching. A credit officer working out the rate buried in a set of instalments is in exactly this position, and so is an analyst backing out the growth figure that would justify a stated total. Searching is exact to whatever accuracy is worth paying for, it repeats to the last digit, and the accuracy costs one extra step per halving. If the answer is a share or a spread over many outcomes and the shape those outcomes come from can honestly be stated, the method is drawing. A risk team asking how often a stated set of assumptions produces a shortfall is in that position. And where no shape can honestly be stated, what remains is replaying the record held, with the answer stated plainly as bounded by the record rather than by the world.

Reaching for drawing when searching would have worked is the commonest waste of the three, and it happens because drawing feels more thorough. Ten thousand of anything looks like diligence. The pull is worth noticing for its strength: a sheet of drawn outcomes reads as careful work in a way that twenty two lines of halving does not, even when the twenty two lines gave the exact answer and the ten thousand gave an approximation with a width on it. The sensitivityHow far the answer moves when one input is nudged a little. Sensitivity says how much precision in the input is worth paying for. of the ladder near its answer is worth knowing here too: a change of one hundredth of a percentage point in the monthly rate moves the twelfth month reading by about Rs 0.1564/-, so there is a point past which more digits in the rate are decoration. Where that point falls, and how it is decided, is covered separately.