Machine Learning: Learning From Data Rather Than Rules

Machine learning means a component that got its behaviour from past examples rather than from a person writing it down. At the end there is a set of numbers, not a set of instructions, so there is no procedure to read and no line where somebody chose the cut. Reviewing one means reviewing the examples it came from and the behaviour it now shows.

Two components can sit next to each other in the same system, take the same file, and hand back an answer of the same shape. From the outside they look like variations on one idea. The two are nothing of the kind. One of them carries its reasoning in text a person wrote and can edit this afternoon. The other carries its reasoning in numbers nobody chose one at a time, and no text exists anywhere. Every practical difference that follows falls out of that single difference in what exists at the end.

What does it mean to say a component learned its behaviour?

Start with something familiar from a kitchen. A cook who has never opened a recipe still knows when the dal needs another pinch of salt. Asked for the rule, she cannot give one. There is no rule; there are fifteen years of tasting. Her apprentice has a written card taped above the stove: quarter teaspoon per cup, add at the end. Both produce dal. Only one of them can hand the card over.

Machine learningDeriving a component's behaviour from past examples instead of writing it down. is the cook, not the apprentice. Somebody assembled a pile of past examples, ran a one-off process over them, and that process produced a component whose behaviour now reflects what was in the pile. Nobody sat down and decided what the component should do with an applicant aged thirty-one with eleven months in a job. The pile decided, and nobody read the pile line by line either.

The word that misleads everybody here is the present tense. Learning sounds like something happening now, continuously, in the background, the way a person learns a new route to work. It is not. FittingThe one-off process that turned the past examples into the stored numbers. is a one-off event that happens before deployment, on a stack of examples that was frozen on a particular day. After that the component runs. It applies. The component does not carry on learning while it applies, and it will not start unless somebody runs the process again on a fresh stack. Two thirds of the trouble below comes from people who assumed the opposite, so the difference between fitting and applying is worth holding on to.

What is actually stored at the end, if not instructions?

Everything else follows from what is stored at the end, and most people never ask. When a person writes a rule, the process ends with an artefact that can be held: a document, thirty-four lines of it, with a heading and a version number and a name against the last change. When a component is fitted, the process also ends with an artefact that can be held, but a very different object. The artefact is a list of fitted numbersThe values a component holds at the end of fitting, in place of written instructions., thousands of them, each meaningless on its own.

No document was produced at any point, and this is not an oversight somebody can go back and correct. There is no missing file, no report that was skipped, no procedure sitting on a shared drive that nobody has printed. The absence is structural. A person who wanted the procedure would have to write one after the fact, by watching the component and describing what came out. A description of behaviour is not a statement of intent, and the two are not interchangeable.

The everyday version is this. A written rule is a recipe card. A fitted component is the seasoning in the cook's hand. The card can be photocopied and handed to a customer. A hand cannot be photocopied.

What tells the three learning setups apart?

Three setups are listed wherever this subject is taught, usually under names that say nothing about where each one belongs. The names matter less than the question underneath them. A single question about the examples separates the three: what does each example carry with it? The question is enough to place any of them correctly, and it needs no mathematics at all.

The first setup uses examples with an outcome attachedAn example that carries not only the inputs but what actually happened afterwards.. Each row holds the inputs and, beside them, what actually happened next. Eleven thousand past applications, and against each one, whether the borrower fell behind. Because the outcome is sitting right there, the fitted component can be built to produce that outcome for a file where it is not yet known. The attached outcome is the whole trick, and it is the reason this setup dominates in finance: banks keep records of what happened, so the outcomes exist.

The second setup uses examples where nothing happened next, or nothing was recorded. Only the inputs are available: three years of transactions, no marker anywhere saying which ones were dishonest. With structure onlyExamples with no outcome attached, where all a component can find is grouping and unusualness. to work with, a fitted component can group things that resemble each other and flag things that resemble nothing. Such a component has never been shown fraud, so it cannot report that a transaction was fraud. The most it can report is that a transaction is unlike the rest. Unlike the rest and fraudulent are genuinely different statements, and the two get confused constantly.

The third setup learns from the consequences of its own answers. The component acts, something comes back, and the return signal shapes the next act. The third setup has a hard requirement the other two do not: the consequence has to actually come back, and come back fast enough to be useful. Fast return is easy for a system playing a game and awkward for a system deciding loans. The consequence of accepting an applicant is not visible for many months, and the consequence of declining one is never visible at all. A declined applicant simply leaves.

Which of the three actually sits in one deployed lending chain?

Sumeru Bank Limited, an invented mid-sized Indian bank, runs one retail personal loan intake chain, from an application started on a handset through to a decision. Nine components sit in that chain and five of them were fitted rather than written: the liveness check on the selfie, the document classifier, the field reading step, the scoring modelIn the case here, the component that turns an application into a value used to accept, decline or refer it., and the drafting assistant that writes the first version of an exception note.

All five came from examples with an outcome attached, and neither of the other two setups appears anywhere in the chain. The three setups are usually presented as a menu of equal options. In one real deployment the menu had one item on it. The classifier was fitted on past documents where somebody had recorded what kind of document each one was. The scoring model was fitted on a past window of 300,000 applications. Of those, 240,000 had been accepted and had an observable outcome afterwards, being 80.0 per cent. The remaining 60,000 had been declined and therefore had no outcome to attach at all.

The 60,000 declined applications are the third setup's problem in one number. A component cannot be shaped by the consequences of answers whose consequences never arrive, and a declined application produces no consequence the bank will ever see. The applicant does not come back in twelve months to report how the loan they never received would have gone. So the setup that sounds most like learning, the one that improves from its own experience, is precisely the one that lending cannot feed.

What is there to review when no procedure exists to read?

Neelima Rao sits in the risk function at Sumeru and built no part of the chain. Building no part of it is exactly why the review was given to her. When the independent review of the scoring model came round, she opened the file expecting to find something to read. There was nothing to read. None of those things had ever been created. There was no thirty-four line document, no version four and no name against a last change.

She reviewed four things instead, and not one of the four is the component itself. Which applications were in the pile it was fitted on. Which outcome was attached to each of them. Which inputs the running component actually reads when a file arrives. And how its behaviour has looked across recent months, a behaviour reviewExamining what a component has actually been doing, in place of reading what it was told to do. rather than a reading exercise. The pile, the outcomes, the inputs and the behaviour are what exists. A reviewer who insists on a procedure is asking for an object that was never made.

Why does the mix of examples show up in the behaviour?

Take the first of those four seriously for a moment. If the pile is the closest thing to a statement of intent, then who is in the pile is not a technical housekeeping detail. The pile is the nearest thing the component has to a policy, and nobody wrote it down as one.

Sumeru measured this on its own document classifier. Applications reach the bank through several channels, and 62 per cent of the examples in the fitting pile came from a single one of them. The bank then measured how well the component read documents from that channel and how well it read documents from all the others, and the two readings were not the same. A component reads best what it was shown most of, and the difference is a choice about examples rather than a choice about the component. The classic worry about a model's errors falling unevenly across a population, named by O'Neil in Weapons of Math Destruction, stops being an abstraction here and becomes a table in a review file.

Push the control to the far left and something instructive happens. Below a mix of roughly 38 per cent the two lines cross over, and the channel that supplied fewer of the examples is now the one being read better. There is no setting at which the mix stops mattering, only a setting at which the gap happens to close. The bank did not choose 62 per cent as a policy. The 62 per cent was simply the share of applications that channel happened to send during the window the examples were pulled from. Most such choices actually get made that way.

What did the review cost, against twenty five minutes of reading?

Set the two components side by side at Sumeru. Component 5 is the income corroboration step: 34 lines of written procedure that compare a declared monthly income against the median salary credit over three months of statement, and route the file to a person when the declared figure runs ahead of the corroborated one by more than the tolerance the bank chose. Component 6 is the scoring model, fitted on that pile of 240,000 applications. In one steady month, component 5 routed 602 files and component 6 determined the outcome of 5,981.

Neelima Rao read all 34 lines of component 5 in 25 minutes. At the end of it she could explain any one of the 602 routed files to anybody who asked. The explanation was line 27, and line 27 said what it said.

Eleven working days. On the bank's own assumed working day of 420 minutes that is 4,620 minutes against 25, or roughly 185 times as long. The two activities were not the same activity performed at different speeds, so the ratio is the least interesting thing about them. None of those eleven days was spent reading the component. The time went on the four things in the scope note, and what came back at the end was a finding of a different kind. Not this line is wrong. The behaviour has moved.

The distinction between the two kinds of finding changes what an audit function can promise. A finding about a line is closed by editing the line, and the edit is dated and attributable. A finding about behaviour is closed by deciding whether the movement is acceptable, and if it is not, by running the fitting process again on a different pile. Running the fitting again is a project with a budget rather than an afternoon with an editor.

Does a learned component notice when the world it was fitted on changes?

No, and the word learning is what makes everybody expect otherwise. Think about a driver who memorised a route twenty years ago and has driven it on autopilot ever since. The road has been made one way. He has not noticed. Nothing about his memory of the route contains a mechanism for noticing. He is not broken. He is doing exactly what he learned, on a road that stopped matching it.

A fitted component holds perfectly still while the numbers arriving at it move underneath. The component has no sense that anything has changed, no comparison being run in the background, and no way of registering surprise. Surprise would require an expectation, and the component holds numbers rather than expectations. The stored values it does hold assume the inputs still mean what they meant on the day the pile was frozen.

What is a learned component genuinely better at?

None of this is an argument against learning. There is a real class of problem where writing a rule is not merely slower but impossible, and it is worth naming precisely.

Sumeru asked its document classifier to look at an uploaded sheet and say what kind of document it is. Writing that down as a rule is where the difficulty starts. Which rule separates a salary slip from a bank statement? An attempt begins with something about a table of dates and amounts, and then discovers that both have one. The attempt tries the presence of an employer name, and then meets a statement with a salary credit narration on every line. There are hundreds of layouts, dozens of formats, and the thing actually used to tell them apart is a loose resemblance that will not go into words. The answer is known and cannot be explained, and knowing without being able to explain is exactly the situation where fitting a component to examples earns its place.

The same is true of the liveness check on a selfie image. Nobody can write the procedure that separates a live face from a photograph of one, and yet a person can do it instantly. Where the pattern is real, plentiful in past examples, and stubbornly resistant to being written down, learning is not a shortcut. Learning is the only route.

Notice what these two share. Neither of them ever has to explain itself to a customer. The classifier's output never leaves the system; the liveness check produces an outcome a person can be asked to re-check. Never having to explain is not a coincidence, and it leads directly to the last question.

Where does a written rule stay the better answer?

Three questions decide it, and they are worth asking in this order. Can the pattern be written down by a person who understands the work? If yes, write it, and stop. Do enough past examples exist with an outcome attached? If not, learning is not on the table however attractive it looks. And will somebody, one day, have to state a reason for one individual answer to the person that answer was about?

The third question rules out more cases in practice than the first two put together, and it is the one most often asked last or not at all. Sumeru made this trade explicitly. On the document classifier the output never reaches a customer, so Sumeru accepted a build whose answers came with no nameable driver behind them. On the scoring model an applicant can be declined and somebody has to say why, so Sumeru refused the same trade. Same bank, same month, opposite decision, and performance was not what flipped it.

The household version of that trade is familiar. A neighbour can be told exactly why the flat above the bakery was turned down: it was the smell at four in the morning, and it can be pointed at. Why somebody at a wedding did not appeal cannot be explained to a cousin at all. Both judgements may be sound. Only one of them survives being questioned by a person who disagrees, and in a regulated business, being questioned is not the exception.

How does a lender, a reviewer or a board actually use this?