Adverse Action: Explaining a Declined Application

Adverse action is what a lender does and says when it refuses an application. At Sumeru Bank Limited, invented, 688 files a month were declined by a fitted component with nobody reading them. A reason code was written to every one of those files. The letter that reached the applicant carried the same sentence in all 688 cases, and it was not that reason.

A refusal is something that happens to a person, and the explanation is the only part of it they get to keep. One asymmetry inside this bank sits under all of it. The component produced an outcome the bank could act on in about four minutes, and a reason the bank had to construct afterwards, and only one of those two things was ever built on purpose. The decision was designed and the explanation was assembled, and the applicant received neither of them.

What is adverse action, and why is it not a document?

Start away from banking. A household asks the shop at the end of the lane for the month's groceries on credit, the way three neighbours already do, and the answer is no. Consider what has just happened. There was a decision. There was the moment somebody delivered it. There was whatever reason came with it, or did not. And there was what the household could do next: whether it could ask again, and whom it would even ask. Not one of those four things is a document. The whole event is the refusal, and if the shopkeeper hands over a printed slip, the slip is a part of it rather than the thing itself.

Adverse actionWhat a lender does and says when it refuses an application, taken as the whole event rather than as the letter alone. is that same event with a lender on the other side of it. Adverse action covers the decision, the telling, the reason offered, and who is able to answer when the person asks. Treating adverse action as a document turns a hard question into an easy one. A document raises questions about wording. The whole event raises a different question: what the lender can honestly say at all. The second question is much harder to answer, and that is exactly why it tends to get answered by whoever last edited the template.

At Sumeru Bank Limited, invented, the retail loan intake chain reached a decision on 8,600 applications in one steady month. 5,590 of those were decided with nobody touching the file. Inside that group sat 688 declines, and 688 people opened a message that said no. Whether any individual one of those 688 was rightly declined is a separate question from what each of them was told.

What did the applicant actually receive?

All 688 letters carried the same sentence, saying the application had been assessed as likely not to be repaid. The sentence was drafted once and printed on every decline of that kind. There is nothing unusual about standard wordingA sentence drafted once and printed on every letter of a kind, so no letter has to be written from the file. in itself. Every large lender has some, and a person waiting for an answer would rather have a clear sentence than a technical one. The trouble here is not that the sentence was standard. The trouble is what it claimed.

Component 6, the fitted part that produced the outcome, was built to predict one thing. The labelThe outcome a component was fitted to predict, defined by choices somebody made about what counts, over what period, and among whom. it was fitted to carries four choices, all of them Sumeru Bank Limited's own: that trouble means 90 days past due, that the observation runs for 12 months, that the population is applications this lender accepted, and that accounts closed early are set aside. Read those four next to the sentence in the letter. The letter said something about repayment in general, and the component knew about one definition of trouble, over one window, among people this lender had already said yes to. In the fitting window of 3,00,000 applications, the component learned from the 2,40,000 that were accepted. The other 60,000 were declined and their outcomes were never observed at all, so 20.0 per cent of the window was a blank to it.

Neelima Rao, in the risk function, found the gap at the month 12 validation by putting two documents side by side: the letter template and the written definition of what the component predicts. Finding the gap took reading the two documents next to each other, and nobody had done that since the wording was drafted. The sentence had been going out since go-live in month 4. Agrawal, Gans and Goldfarb, in Prediction Machines, 2018, separate the prediction a component makes from the decision a person then takes with it, and this bank had let the second one borrow the authority of the first: a narrow prediction went out wearing the words of a broad judgement.

One more number belongs here, and it needs a word attached to it. 688 declines a month across months 6 to 11 alone is 4,128 letters carrying that sentence. Months 4 and 5 sit before that stretch and their letters carried it too, so 4,128 is a floorA count that is at least this large, where the true total is not known. The opposite of an estimate that might be too high. rather than a total. At steady volume, months 4 to 11 would be 5,504, but months 4 and 5 were a single channel and then an extension across channels rather than steady state, so nobody can state the true count. Saying floor rather than total changes what a count is allowed to do in an argument, and a count with the wrong word attached is worse than no count.

What sat on the file that never reached the letter?

Here is the part that is easy to miss. A reason codeA short attributed cause written to the file when a component declines an application, held in the bank's own record. was written to every single one of the 688 files. Not to a sample, not to the difficult ones. All of them. So the bank was holding, for each person it declined, one short line of information specific to that application, at the same moment it posted them a sentence that was specific to nobody.

Think about what that means for the person on the phone the next morning. The caller is not asking for the workings. The caller is asking the plain human question about their own application. The answer existed, in the bank's own system, attached to their reference number. The answer simply had no route out. The one file specific thing the bank held never left the bank, and nobody remembers deciding that it would not. Most failures of this kind have that shape: not a decision anybody defended, but a gap between two systems that no single person was accountable for closing.

Which six reasons were written across one month of declines?

The six codes are worth reading slowly for what they are and what they are not. Each one names something the application record contained. Not a judgement about a person, not a character, not a failing. A record showing a length of credit history is a fact about a record, and a short one belongs to everybody who has just started out. Every young person is in that group, and so is every household that has never borrowed before. Read the six as descriptions of what a file held. A code can honestly be nothing else.

The shape matters more than the ranking. One code covers a third of the month and the last one covers 24 files. The small number at the bottom invites reading as the leftovers, the rounding, the ones handled later. The 24 are worth holding on to instead. A design that prints a reason on every letter has already made a promise about those 24 files, and what that promise can be kept with is the whole problem.

How does a reason get attached to a component that returns only a number?

Component 6 does not produce sentences. Component 6 produces one value on a scale Sumeru Bank Limited invented for itself, and that value carries no cause inside it, in the way a thermometer reading carries no explanation of the weather. So where did the code on the file come from? From two further steps, built afterwards, sitting between the component and the record.

The first step looks at one application and names which input moved that value furthest. The second turns that input into a short written cause from a list of six. Both steps are ordinary engineering and neither is dishonest. But both are full of choices somebody made: which inputs are eligible to be named, how far is far enough, what happens when two inputs are close, and which of the six sentences a named input maps to. An attributed reasonA cause produced after the fact from a component that returned only a number, rather than a cause the component itself produced. is manufactured after the number rather than produced with it, and every joint in that manufacturing is a decision that can be argued with. Item 6 of the credit decision record is exactly where those choices would be written down, and at the month 12 validation it was one of the four items found missing.

What happens on the 24 files where no single attribute could be named?

On 24 files a month, the attribution step ran and came back with nothing worth calling a cause. No single input had moved the value far enough to stand up as the reason. Nothing had gone wrong on those files. An empty result is what happens when many small things point the same way at once, and it is the honest answer: the component's view of that application rested on no one attribute a person could name.

Code 6 is the honest code, and it is also the one that cannot be turned into a letter. Every other code has a sentence waiting for it. Code 6 has an accurate record and nothing sayable. So what goes out on those 24? At Sumeru Bank Limited the answer was the same standard sentence that went out on the other 664, and the smallest code became invisible. 24 a month is 288 files a year, and 288 people is a full auditorium.

Why can one form of words not answer both kinds of refusal?

In the same steady month, 1,290 files reached an outcome the applicant would experience as a refusal, and they came from two completely different places. 688 were declined outright by the fitted component. The other 602 were stopped by component 5, the income corroboration step, a written procedureA rule a person can read as lines, which a refusal produced by a rule set has sitting behind it. of 34 lines somebody wrote by hand. A file stopped that way is not declined; it becomes a referralA file sent to a person because the automatic path stopped rather than answering, so a human decides what happens next. and a person picks it up. But the applicant experiences the same thing in the moment. The answer did not come.

Now look at what each kind leaves behind. For the 602, the bank can say exactly what happened: the rule compared the declared monthly income of Rs 45,000/- against a corroborated Rs 38,000/- from three months of statement, found a gap of 15.6 per cent, and the tolerance Sumeru Bank Limited chose for itself is 10 per cent. Every one of those numbers is readable, and Neelima Rao read the whole 34 lines in 25 minutes. For the 688, there is a value, an attributed cause, and no lines to read. Answering all 1,290 by quoting the written procedure describes something that did not happen on 688 of them, and 688 is 53.3 per cent, the larger share.

What does an explanation have to contain before it is worth sending?

Strip away the wording question and a small list is left. An explanation of a refusal has to say what was decided and by what. The explanation has to say what the decision rested on for this file rather than for files in general. The explanation has to say what the component behind the decision was actually built to predict, in the lender's own definitions rather than in borrowed general language. And it has to say who can be asked, so the person holding it has somewhere to go. Four parts, and none of them is long.

The standard sentence at Sumeru Bank Limited carried none of the four, and the file it was generated from could have supplied three of them without anybody writing anything new. The uncomfortable part is that the bank never lacked the information. The outcome was in the record, the code was in the record, the label definitions were written down and were among the five items of the credit decision record that did exist. Only the fourth part, who to ask, needed a decision from anybody.

What does the record have to hold so somebody can answer later?

Sumeru Bank Limited settled on nine numbered items as what documenting an automatic credit decision means for it. At the month 12 validation, Neelima Rao found 5 of the 9 documented, or 55.6 per cent. The count is not the interesting part. The five that existed all describe how the component was made, and every one of the four that did not describes what it does to a file and who can stop it. A reviewer reads that pattern, not the percentage.

Item 6 is the whole problem in one line. The missing item would have held the two attribution steps, the list of six codes, the rule for what happens when nothing can be named, and the route from the code on the file to the words in the letter. Without it, a person asked to explain a decline can confirm that it happened and can read out the code, and cannot say what the code was produced by or how far to trust it. Ashok Pillai, in technology risk, ran the month 10 sweep of what was actually in use across the bank and found more in use than had been registered. The same problem sits one layer up: a question about a decision cannot be answered if the record of what made it is thinner than the decision itself.

Who answers when the applicant asks, and what do they need in front of them?

Somebody rings the next morning and asks why. The call travels a fixed distance inside a bank. The person who picks up can read what the letter said. The desk behind them can see the file and the code. But the question underneath is what the code means and how far it can be relied on, and that question belongs to the person accountable for the component. At Sumeru Bank Limited the accountable person is Revathi Balan, head of retail credit, named in item 7. Item 7 existed. Item 6, the item she would need in order to answer, did not.

Ismail Sheikh runs the exception desk, and his 7 people are not the answer to this either. Their 3,010 files a month are the ones a person was always going to touch, and each one takes 19 minutes. The 688 declines were decided and closed in about four minutes without a person, so they never reach the exception desk. An arrangement where nobody in the answering chain holds the record of how the reason was produced leaves everybody in it able to confirm the outcome and nobody able to explain it. An applicant may ask for what the lender holds about their own application, and what comes back is set by what that lender wrote down before go-live rather than by who happens to answer the phone.

How does somebody reviewing this arrangement read all of it?

Three people read this differently and all three readings are useful. A reviewer inside the lender, in the position Neelima Rao was in, does not start with the model. She puts the letter template beside the written definition of what the component predicts and reads them in the same sitting. A twenty minute check of that kind found a sentence standing since month 4. Then she counts: how many declines, how many codes, how many files where the code says nothing nameable, and whether the record says how any of it was produced.

A person running a customer desk reads it as two queues rather than one. The refusals with a written procedure behind them can be answered on the call. The refusals from a fitted component cannot, and pretending otherwise is what produces a confident answer that turns out to be wrong. Knowing which queue a file is in, before speaking, is the single most useful thing a desk can hold. And a person who has received one of these letters is reading for something simpler: whether the letter says anything about their own application at all, and who they can ask if it does not. Neither is a large thing to want, and the first of the two sat on the file the whole time.