Algorithm: A Procedure, Not an Intelligence

An algorithm is a procedure with fixed steps, defined inputs and a defined stopping point. Given the same inputs it returns the same answer every time, and a person can read it from the first line to the last. Neither property is available on the same terms once a component derives its behaviour from past data, and that loss is what changes who can answer for the output.

Every judgement inside a written procedure was made once, by somebody, and then written down where it stays. The comparison, the allowed gap, the order in which the tests run and what happens when a test cannot run at all are choices sitting in the text as ordinary sentences. A written procedure can be argued with, and the argument has a document to point at. A component that derived its behaviour from past examples offers an output to disagree with and nothing at all to disagree over.

What is an algorithm, and what is it not?

One example is already familiar. Long division, the way it is taught at nine years old, is an algorithmA procedure with fixed steps, defined inputs and a defined stopping point. in the full sense. Long division has defined inputs, a dividend and a divisor. Long division has fixed steps, and the steps run in an order that does not change because the queue is long. Long division has a defined stopping point. The procedure stops when the remainder is smaller than the divisor, or when the decimals have been taken as far as was set out at the start. Handing the same two numbers to two different children who both learned it properly gives the same answer twice.

The three features are the whole definition, and none of them is about difficulty. Defined inputs, fixed steps, a defined stopping point. A procedure of four lines qualifies. A procedure of four thousand lines qualifies. Length, cleverness and the amount of arithmetic involved have nothing to do with it. The word covers the way a housing society allots its parking bays as comfortably as it covers anything running inside a bank.

The word is doing a lot of work it was never built for, so the negatives matter. An algorithm is not a synonym for something modern. An algorithm is not a measure of how difficult a thing was to build. Nothing in the definition allows a step to rewrite another step, so an algorithm does not improve itself. And it is emphatically not a description of every automated output. The loose use of the word for anything automated causes the most trouble of the three. The word describes how a result was arrived at, not how impressive the result is.

People forget the stopping point, so it deserves its own moment. A procedure that might run forever cannot be put in a queue that has to clear by six in the evening. Every one of the thirty-four lines set out below either moves the file forward or stops it, and that is a design property somebody had to actively arrange.

Which two properties matter once the procedure is running?

To the builder, what matters is whether the procedure is short, quick and neat. Running it in a bank, with customers on the other end of it, two entirely different properties take over, and both of them are about somebody who did not build the thing.

The first is that the procedure is repeatableReturning the same answer to the same inputs every time it runs.. Nothing inside the text depends on the day, the queue length or who is on shift. Feed the same file in on Tuesday and on Friday, and the same answer comes out. Think of two clerks at two counters of the same office working from the same printed circular. If they reach different answers on the same application, either they are not both following the circular or something one of them is reading has changed underneath. There is no third possibility, and that is a useful thing to be certain about.

The second is that the procedure is readable. A person who did not build it, was not in the meeting and does not write software can be handed the document and can follow it to the end. Responsibility means being able to say why, and saying why means having read the reason. Readability is what turns an output into something a person can take responsibility for.

Now the honest part, and it is more precise than the usual telling. A component that derived its behaviour from past examples is often repeatable in fact: while its fitted numbers stay put, the same file does give the same answer. Repeatability cannot be established by reading anything, only by running the file again and watching. And it is lost completely at a refit, where the behaviour changes without one line being edited and with no revised text for anybody to compare against the old one. So the first property survives only as something to be tested, and the second does not survive at all.

What does one rule set look like when somebody prints it out?

Abstractions about procedures stop being useful very quickly. Component five of the intake chain at Sumeru Bank Limited, an invented lender, is the income corroboration step, a rule setA written procedure a person can read from the first line to the last. of thirty-four lines. Its job is small and completely stated: take the monthly income the applicant declared on the form, work out by corroborationChecking a figure a customer stated against a record from somewhere else. what the applicant's bank statement actually shows arriving as salary, and decide whether those two figures are close enough to carry on with.

Thirty-four lines is a document, not a mystery, and the practical consequence of that is a number: Neelima Rao in the risk function read every line of it end to end in twenty-five minutes. At the end of those twenty-five minutes she could take any one of the six hundred and two files the step routed that month and say exactly which line sent it and why. There is no length of time after which somebody has read a fitted component in the same sense.

Reading down the numbers shows the four things a procedure is made of without anybody labelling them. Lines one to seven fetch the inputs and stop the file if an input is not there. Lines eight to twenty-three work out the corroborated figure and then test it. Lines twenty-four to twenty-nine take the action and write down what was found. Lines thirty to thirty-four say what happens when a test cannot run at all. Nobody reads that part, and every real production failure lands in it.

The middle of it is one branch, and the branch is the entire component. Take the declared figure. Take the corroborated figure. If the first exceeds the second by more than the allowed gap, route the file to a person. Otherwise pass it. Anybody holding the document can predict the answer before the procedure runs. The claim is stronger than it sounds, and nothing else in the chain offers it.

The worked file, and the month it sat in

One application declared a monthly income of Rs 45,000/-. Three months of statement showed monthly salary credits whose middle value was Rs 38,000/-. The gap is Rs 7,000/-, and line twenty of the procedure divides that by the declared figure, giving 15.6 per cent. The bank's own chosen tolerance is 10 per cent. 15.6 is more than 10, so line twenty-two sent the file down the routing branch and a person at the exception desk opened it.

Scale that one file up to a month. In one steady month the step saw all 8,600 files that reached the decision engine. The step passed 7,998 of them and routed 602. The two numbers add back to 8,600. Anybody holding the document and the month's data can check both, and most of what runs beside the step offers nothing of the kind.

Where does the judgement actually sit inside those thirty-four lines?

One trap is worth sitting with. People argue about the software when what they disagree with is a decision, and the decision is almost never spread across the document. In this procedure it sits in three places, and once those three places are visible here they are visible in any rule set.

The first is which record to read. Line three opens the account marked as salary, and somebody decided that a salary credit into a different account does not count. The second is how many months to take. Line seven takes the three most recent full months, and somebody decided that three is enough and that a fourth would not change the answer often enough to be worth the wait. The third is how big a gap to allow. Line seventeen reads a toleranceA gap somebody chose to allow before a test counts as failed. setting, and the figure in that setting at Sumeru Bank Limited is 10 per cent.

The 10 per cent tolerance is the invented bank's own choice, made by people in a room, and it is not a standard, a norm, a requirement or a number anybody outside the bank handed down. The whole reason a written procedure is worth defending is that its judgements are visible as judgements. The moment a chosen number starts being described as what the rule requires, the argument moves out of reach of the people who should be having it, and nobody has to defend a choice that nobody can see was made.

A chosen number has one more property that a slope does not, and it catches people out constantly. A chosen number produces a step. Two files that a person would look at and call the same file land on opposite sides of it, and nothing in the procedure holds any notion that they are nearly the same. At an allowed gap of 10 per cent, a file at 10.1 per cent is routed and a file at 9.9 per cent is passed, and the two are treated as differently as if one had declared nothing at all.

The step is not a defect to be fixed. A written test produces a step by its nature, and a slope would simply move the argument to where the slope starts and stops. A step is where two nearly identical customers get two visibly different experiences, and that is the place a complaint comes from. Knowing where the step sits is what matters.

The shape matters more than the number. The curve falls away steeply on the left and then flattens almost to nothing. The shape reveals something about applicants that no amount of arguing in a meeting would have produced. Most people declare an income close to what their statement shows. The files pile up at small gaps, so tightening the allowed gap from 10 per cent to 5 does not shave a little off the exception queue, it multiplies it by more than two and a half, from 602 files to 1,548. A number chosen in a meeting decides how many people wait two working days for an answer, and the relationship between the number and the queue is nothing like proportional.

Push it the other way and the second lesson arrives. At 20 per cent the rule routes 172 files, at 30 per cent it routes 69, and by then the step is barely doing anything at all. A tolerance loose enough to be uncontroversial is also a tolerance that has stopped being a control. Somebody has to sit between those two edges and decide, and that is the honest description of what setting a tolerance is.

Which parts of one lending chain are written rules, and what does each decide?

Four of the nine components of the intake chain are procedures somebody wrote: the identity match, the income corroboration step set out above, the fraud rules that run on the servicing book, and the workflow routingSending a file to a person instead of deciding it, or on to the next step. rule that decides where every file goes next. The four are not versions of the same thing. The position in the chain decides what a rule can possibly know, so where each one sits is worth seeing.

The identity match runs first, before any document has been read, so all it can compare is what two records say about a name and a number. Income corroboration needs a statement to exist before it can look at one, so it runs after the documents have been read. The fraud rules run after disbursal, on the servicing book, and act on an account rather than an application. And the router does not sit at a point at all: it sits between every pair of steps. The router touched all 8,600 files of the month while deciding the final outcome of only 112.

Together those four written procedures determined 1,355 outcomes in the month, being 15.8 per cent of the 8,600, and every one of those 1,355 has a document behind it that a person can be shown. The point is not that rules are old or simple, but that a sixth of the month's outcomes arrive with their reasons attached.

Where is a written rule the only defensible choice?

Not everywhere. Nobody can write down what a salary slip looks like in a way that survives four hundred employers formatting theirs differently. A written procedure would therefore be a poor way to decide which kind of document an image is. Where the thing being judged is a shape, a pattern or a likeness, a fitted component is simply better at it, and pretending otherwise costs accuracy for nothing.

But there are three places where the written rule is not merely preferable, it is the only answer that can be defended afterwards, and all three are about somebody else being owed an explanation.

The first is where the test is a policy rather than a pattern. Whether a declared income is close enough to a corroborated one is not a fact waiting to be discovered in data. Closeness is a position the lender is taking about how much unverified income it is prepared to lend against. The answer is not in the past but in the decision, so there is nothing for a component to learn. The second is where the person affected has to be told why, in words they can check. An explanation the recipient can verify against a document is a different object from a reason produced after the fact, and only one of the two can be argued with by the person it was about. The third is where somebody must be able to change the behaviour by Friday, deliberately and in a stated direction. Editing line seventeen is an afternoon. Moving a fitted component's behaviour in a chosen direction and no other is not.

The framing that a fitted component produces a prediction which a person still has to act on belongs to Agrawal, Gans and Goldfarb in Prediction Machines, and it is useful here in the negative. Where what is needed is not a prediction at all but a position, prediction is the wrong tool no matter how good it is.

Who wrote it, who changed it, and when?

Ask that question of a written rule and it always has an answer. Not because rules are virtuous, but because a document is a thing that can carry a history beside it. Ask it of a fitted component and the honest answer is a shrug and a date.

A rule set has five named parts, and the first four are inside the file set out above. Its inputs, being lines one to seven. Its conditions, being lines eight to twenty-three, work the figure out and then test it. Its actions, being lines twenty-four to twenty-nine. Its behaviour when a test cannot run at all, being lines thirty to thirty-four. The fifth part is not inside the file, and a review that stops at line thirty-four has missed the place where accountability actually lives: the change recordThe log of who altered a procedure, when, and what the previous version said., carrying a name, a date and what the previous version said.

And the fifth part can be missing. In month seven of the intake chain's life, the waiting time the router holds before it escalates a file was changed. No approval was recorded and no previous value was written down. The change was found three months later, in the month ten sweep, by Ashok Pillai in technology risk. The readability of a rule is a property of the document; the change record is a property of the discipline around it, and only one of those two comes for free.

Where does the word get used for something it does not describe?

Three places, and all three cause a specific kind of trouble rather than merely being untidy.

The first is the commonest: the word is used for a component that derived its behaviour from past examples. There are no steps to read, nothing branches on a line somebody wrote, and the stopping point is not a design choice but a property of the arithmetic. The arithmetic that produces its output is itself a procedure, so calling that an algorithm is not wrong in a textbook sense. The word still invites the listener to believe there is a document, and there is not.

The second is the word used for a whole system. The intake chain is not an algorithm. The chain is nine separately built and separately changeable components, four of which are written procedures and five of which are not. A single word for the whole thing hides exactly the distinction anybody would need in order to act.

The third is the word used as a shield. The algorithm does not allow it, said across a counter, converts a choice somebody made into a fact of nature. In the case here the sentence would be doubly wrong: there is an allowed gap, it is 10 per cent, it is the invented bank's own choice, and a named person can change it. Nothing about that is a fact of nature. Whenever a chosen number is described as something the system requires, ask who chose it. On a written rule that question has a name attached to it.

Why does one complaint split into two, and why can one word not answer both?

Here is where all of this stops being vocabulary. A customer writes in and says: the algorithm rejected my application. Perfectly ordinary sentence, and everyone at the desk knows what it means. Except that in one steady month, 1,290 files reached an adverse outcomeAny result the applicant experiences as a refusal or a delay. the applicant would describe that way, and those 1,290 are two completely different groups wearing one word.

602 of them were routed by the written income rule. Behind each of those sits a printed procedure, a tolerance somebody chose, a change record with a name on it, and two figures the applicant can be shown: what was declared, what was corroborated, and by how much the first exceeded the second. 688 of them were declined by the scoring model. Behind those sits no procedure to print, only an attributed reasonA reason produced after the fact to describe an output, rather than the step that caused it. produced after the outcome to describe it. 602 plus 688 is 1,290.

How does a lender, a reviewer or a customer desk actually use this?