Credit Score vs Credit Decision: The Number and the Cut-Off

A credit score is a number a component produces about an application. A credit decision is what a lender does about that application. The decision needs the score, plus cut-offs the lender chose, plus everything else in the chain. At Sumeru Bank Limited, invented, 2,619 of one month's 8,600 files reached a decision without any score entering into them, being 30.5 per cent.

Start with what each of the two words points at. A score points at an application and says how it sits against one outcome somebody defined. A decision points at a person and changes what happens to them today. Sitting between the two is a set of choices the lender made and could change tomorrow without touching the fitted component at all. The lender's choices are the whole of the difference, and they are also where the answering sits.

What is a credit score a claim about?

The distinction is easier to see outside banking. A thermometer at home reads 101. The reading of 101 is a claim about one thing, measured one way, at one moment. The thermometer does not say keep the child home from school, it does not say ring the doctor, and it says nothing whatever about the child as a person. Somebody has to look at 101 and decide, and the deciding is a separate act with its own reasons behind it. Nobody at home confuses the reading with the deciding. Inside a bank the two get merged into one word constantly, and every problem below follows from that merger.

A credit scoreA number a component produces about an application, stating how that application sits relative to an outcome somebody defined. at Sumeru Bank Limited comes out of component 6 of its retail loan intake chain. The component returns one value between 0 and 1000 on that bank's own scale, higher meaning it placed the application further from the outcome it was fitted to. The value is the whole of what comes back. One number, on a scale this lender made up for itself. The number is a claim about one application, measured against one definition of an outcome, among one population, and it carries nothing else.

Several things are not inside it. There is no outcome in it. A value of 552 is not an accept or a decline until somebody says where the line falls. There are no words in it, so nothing in the number explains itself. There is no route in it, so nothing indicates which desk the file goes to next. There is no name attached to it. Every one of those has to arrive from somewhere else, and where each of them arrives from is the whole subject.

What is a credit decision, and what makes it different in kind?

A credit decisionWhat the lender does about an application, which is an act with a consequence for the person waiting on it. is the act the lender takes on the application. In one steady month, 8,600 files reached this bank's decision engineThe whole arrangement that takes a file from arrival to an outcome, of which the scoring component is one part among nine. and every single one of them reached a decision. An accept, a decline, or a route into somebody's queue: each of those is an act, each has a time stamped on it, and each changes what a person waiting on a handset sees.

The difference between the two is not that a decision is a bigger or later version of a score. The difference is one of kind, and the test for that is what happens when each turns out to be wrong. A number can be wrong and nothing has yet happened to anybody: the error is found, a better number is produced, and the world is where it was. An act cannot be undone by a later correction. The person waited the days they waited, and the answer they were given is the answer they arranged the rest of their month around. A score is a statement that can be replaced, and a decision is an event that can only be followed by another event.

The difference in kind reaches the words used around each of them. A score can be argued with by arguing about the fitting, the inputs and the definition. A decision cannot be argued with in that way at all. About a decision the only available questions are who took it, out of what, and what would have to change for it to come out differently. Agrawal, Gans and Goldfarb, in Prediction Machines, 2018, put the same split as prediction on one side and judgement on the other: the fitted part gets cheaper and more plentiful, and everything about what to do with it stays exactly where it was.

What has to be added to a score before anybody has a decision?

Put a column of 5,981 numbers in front of somebody and ask them to run a lending arrangement with it. Nothing happens. A weighing scale in a shop reads 2.4 kilograms, and the customer still pays nothing until somebody applies a rate card that the shopkeeper wrote and can change on a Monday morning. The reading is the easy half. The rate card is where the shop's choices live, and the same is true here. Four things have to be added to a column of numbers before anybody has a decision, and not one of the four comes out of the component.

The first is a cut-offThe value at which an arrangement stops asking and acts. It is chosen by the lender and is never a property of the component., with somebody who chose it. At this invented bank there are two, set by Revathi Balan, head of retail credit: accept at 720 and above, decline below 580. The second is a route for whatever falls between the cut-offs. A number in that range has to go somewhere, and the component has no opinion about where. The third is a reason attached to the outcome. A value on a made-up scale answers nobody who asks why. The fourth is somebody accountable for all three, with the standing to stop the arrangement running.

The four additions come from the bank itself, not from any outside checklist. The bank's own credit decision record has nine items, and the four are items 4, 5, 6 and 8 of it. Five of the nine were documented, and at the independent validation in month 12 those exact four were the ones found missing. The five that existed all describe how the component was made. Everything describing how the arrangement treats a file, and who is able to stop it, sat in the four that nobody had written down. Buying a scoring component is not buying a decisioning arrangement, and the gap between the two is precisely these four items.

How many of one month's files reached a decision with no score at all?

One month's counts are where the distinction stops being a matter of careful wording. Of the month's 8,600 files, component 6 produced a score on 5,981 of them. The other 2,619 reached a decision without any score entering into them at all, or 30.5 per cent of the month. Not a score that was ignored, and not a score somebody overrode. No score. The field on those files was never filled. The file left the automatic path before the scoring component saw it.

Five separate exits produced that 2,619, and each is a point at which the chain stops and hands the file to a person. 1,264 files carried a document field the reading step could not read with enough confidence, being 14.7 per cent of the month. 602 had a declared income the chain could not corroborate from the statement, being 7.0 per cent. 452 tripped a fraud line, being 5.3 per cent. 189 carried an identity record that did not match across two sources, being 2.2 per cent. 112 had an incomplete consent record, being 1.3 per cent. The five exits sum to 2,619 and the shares sum to 30.5.

A decision does not require a score, and nearly one file in three at this bank is the demonstration of it. Sit with that for a moment. The 2,619 are the cleanest evidence available that the two words point at different things. An arrangement that is habitually described by its scoring component reached an answer on almost a third of its month without that component being involved in any way.

Who sent those 2,619 files out of the automatic path?

Two very different things did, and the split is worth holding. 1,264 of them were sent out by component 4, the field reading step, a part that learns from data. The other 1,355 were sent out by four written rule sets: income corroboration at 602, the fraud lines at 452, the identity match at 189 and the router itself at 112. 1,264 plus 1,355 is 2,619. So a fitted part and a set of lines somebody typed both produce files that reach a decision with no score attached, and they do it in roughly comparable volume.

Every one of those 1,355 was a routing decisionA decision about where a file goes next, rather than about what happens to the applicant. A file can collect several before anybody decides its outcome., a decision about where a file goes and not about what happens to the applicant. A hospital reception desk sending a patient to the right counter has decided something real, and it has not diagnosed them. In this chain the written rules move files and never hand out or refuse a loan; whatever happened to those 1,355 applicants was settled afterwards, by a person at the exception desk that Ismail Sheikh runs. Keeping the routing decision and the credit decision apart is the same habit as keeping the score and the decision apart, one level down.

Did the component determine 5,981 files or 5,590?

Both counts are correct, they are counts of different things, and a great deal of confusion lives in the gap between them. 5,981 is the number of files component 6 touched and had a hand in what happened next: 4,902 accepted, 688 declined, and 391 pushed into the range between the cut-offs. 5,590 is the number of files where the score settled the outcome and no person read the file at all. The difference between the two counts is exactly the 391.

The 391 files sit on the seam, and they are the most useful files in the month. Each was scored. Each then went to a person, who decided it, and from month 5 a second reviewer read each one as well. The number decided where those files went and settled nothing about what happened to the applicants. A file the component scored and a person then settled is a referralA file where the component produced a score and a person then decided the outcome, so the number routed the file rather than settling it., and a referral is a score doing the smaller of its two possible jobs.

So the honest sentence has two halves. A score settled the outcome on 5,590 files, being 65.0 per cent of the month, and 3,010 files reached a decision that no score settled, being 35.0 per cent. 5,590 plus 3,010 is 8,600. The other true sentence, that component 6 determined what happened to 5,981 files, being 69.5 per cent, counts routing and settling together. Which of the two is meant has to be said every time either is said. The two counts differ by 391 files and by the entire question of whether a person was involved.

Which of the two can be checked, and against what?

The tests available for each of them pull the two apart again. Testing the score means running the same application through the same component and seeing whether the same value comes back. The same value does come back, for as long as the fitted numbers stay where they are. Repeatability of that kind is something tested rather than something read, and it ends at the next refit. Nobody can open the component and read off why 552 rather than 561. The opacity is a property of the fitted part and not a fault of anybody's.

Testing what the number claims about the future needs outcomes, and this bank cannot have them for a long time. Its own definition of trouble needs 12 months of observation before an account counts either way, so the earliest a month's credit decisions can be scored against reality is 15 months after they were taken. Checking a decision needs nothing of the kind. Testing the decision means reading the cut-off, reading the value on the file, and confirming that the recorded outcome is the one the lender said it would produce at that value. The check takes a minute, needs no data, and can be done this afternoon.

The number is the half that cannot be read and can only be tested, and the act is the half anybody can read straight off the file. The usual talk runs the other way, treating the score as the transparent part because it is a number and the decision as the mysterious part because it involved a system.

Which of the two moves when the lender changes its mind?

Hold the same 5,981 scores completely still and change one thing the lender wrote down. At the deployed accept cut-off of 720, those scores produce 4,902 accepts, 391 referrals and 688 declines. Read the same column of numbers at an accept cut-off of 760 and it produces 4,142 accepts, 1,151 referrals and the same 688 declines. 760 applicants who would have had an answer in about four minutes are now waiting in a queue for a person to reach them.

Nothing about any score changed between those two readings. No refit, no new data, no new component, not one value in the column different from what it was. The scores are identical and the decisions are different, and no demonstration this subject offers is cleaner than that one. Whether that move is a good idea is a real question with real arithmetic behind it, and the arithmetic is set out under the decision threshold.

The two also move on completely different clocks. A score moves when somebody refits the component. A refit is a project with data, testing and a validation behind it, and at this bank the review of component 6 alone took eleven working days. A decision moves when somebody changes a number in a document. Changing a number in a document is a meeting. Anything that can be changed in a meeting is a choice, and anybody trying to work out who is responsible for an outcome should start with the things that move that fast.

What can each of them honestly be said to explain?

The score explains nothing on its own, and saying so is not a criticism of it. A score is a position on a scale this bank made up for its own use, and a position on a scale is not a reason for anything. The attributed reasonA cause produced after the fact for a file the component answered with only a number, by steps that run after the component has finished. that appears on a declined file at this bank is manufactured by steps that run after the component has finished, and it is a genuinely useful artefact, but it did not come out of the number and the number does not contain it.

The decision is the half that happened to somebody, so the decision is the half that has to be explainable. A person who is refused is entitled to ask what happened. The answer the lender is able to give back was fixed long before that call, by what the lender wrote down before the arrangement went live. At this invented bank the record item that would have said how a reason is produced was one of the four never written down, so the explanation existed as a practice and not as a documented one.

An explanation is owed for the act and not for the number, and a lender that can produce only the number has not answered the question it was asked. An arrangement can therefore be entirely fair, work exactly as designed, and still be unable to account for a third of what it did.

Who answers for the score, and who answers for the decision?

Revathi Balan, head of retail credit at this bank, is the named accountable personThe named individual who answers for a component or for a decision, written into the record rather than assumed. for component 6. The naming is item 7 of the bank's own nine item record, and item 7 was one of the five documented. She answers for the number: what the component was fitted on, what it claims, how it behaves, and when it should be refitted. The duty is a real one, and it is a bounded one.

The decision belongs to the lender, and the reason is arithmetic rather than sentiment. Every choice wrapped around the number was the lender's: where the two cut-offs sit, what happens to a file inside the referral range, what a refused applicant is told, and who is permitted to stop the arrangement running. Item 8 of the same record, covering what would have to be true for the component to be stopped and who may stop it, was among the four missing. The bank had written down who answers for the number and had not written down who is able to stop the act.

So pointing at the component when somebody asks about a decision sends the question to a person who did not make most of what is being asked about. Worse, no name is attached to the four choices, so they are left unattended. Merging the two words has a quiet cost: not that somebody is blamed unfairly, but that the parts nobody is answering for are exactly the parts that reach the applicant.

Where does treating the two as one thing go wrong in a report?

A monthly report at this invented bank described the arrangement as a scoring model deciding 8,600 applications. Every word of that sentence was defensible in a meeting, nobody who wrote it was careless, and it was wrong in a way that cost something twice over.

Set the sentence against the month. 5,590 files had their outcome settled by the score. 391 were scored and then decided by a person. 1,264 were routed out by the reading step and decided by a person. 1,355 were routed out by four written rule sets and decided by a person. The four counts sum to 8,600 with nothing counted twice, and the last three of them sum to the 3,010 files a person decided. One sentence putting a whole month behind one component makes 35.0 per cent of the month's outcomes invisible to the conversation that is supposed to be about them.

Follow what the meeting then does with its time. Because the sentence said the month was the component, the meeting reviews the component. The four written rule sets that moved 1,355 files get no review, the reading step that moved 1,264 gets none, and the desk that actually decided 3,010 outcomes is discussed as a cost line rather than as the place where more than a third of the answers were produced.

Where does treating the two as one thing go wrong for a person?

Take one file out of the 1,264. A document image was hard to read, one field could not be pulled from it with enough confidence, and the file left the automatic path at the reading step and went into somebody's queue. Component 6 never saw that application. The score field on that file is empty, and it is empty on the record where anybody at the bank can look at it.

Where a file like that ends in a refusal and the person is answered through the same process as everybody else, they are told that an assessment was made about their application. No assessment of that application was made at all, and the file itself says so. The mismatch is not an edge case somebody has to hunt for. By construction the mismatch sits on nearly a third of the month's files, and 2,619 out of 8,600 is what a third means written out as people rather than as a percentage.

Nothing in that paragraph is about the applicant. Whether any individual outcome at this bank was the right one is a separate question. The narrower question belongs entirely to the lender: a process that describes an act it took as an assessment it did not make cannot answer a straight question about its own file, and the person who asked is the one left holding that.

Which four questions keep the two apart?

The whole distinction collapses into four questions, asked in order, and the first one settles most cases before the second is reached. Was there a score at all? Did the score settle the outcome, or did it only decide where the file went? Which cut-off turned that number into this outcome, and who set it? Who answers for the outcome?

A score and a decision give different answers to all four, and the difference in the answers is exactly why the four work as a test. Applied to any sentence about an automatic arrangement, the four sort it. The sentence that survives all four is describing something real. The sentence that fails at the first question was never describing that file at all, and at this bank nearly one file in three would fail it.

How does somebody actually use this in a room?

Three people read this distinction differently and all three readings earn their keep. Somebody reviewing the arrangement, in the position Neelima Rao was in when she did the independent validation, does not begin with the component at all. She asks for two counts, files that reached a decision and files that were scored, and subtracts one from the other. The subtraction took a minute and produced 2,619, the single most informative number about this arrangement. No description of the arrangement carries it. Then she asks which cut-off produced each outcome, who set it, on what date, and where that is written down.

Somebody running a customer desk reads it as two queues instead of one. Before speaking to anybody, the useful thing to know is whether that file carries a score. The answerable question is different on each side. A file with a score and a cut-off has a readable path behind it. A file with an empty score field has a different path behind it, and describing the first path to somebody whose file took the second is how a confident answer turns out to be wrong.

And somebody who has been refused reads it from the other side entirely. Once the two words come apart, the question is no longer what my score was, but whether a score came into it at all, and which line the number was compared against. Both of those are the lender's to hold and the lender's to answer, and neither of them is a question about the person. Keeping the two words apart is worth the trouble for exactly that reason.