The Model Owner: The Named Person Accountable

A model owner is one named person who answers for a deployed component being what the record says it is. The accountability is six duties, and not one of them requires the ability to build the thing. Naming somebody changes nothing on its own. A name becomes real through a route from the monitoring to that person and a date on which the approval ends.

AccountabilityThe set of things one person can be asked to produce about something a firm is running. is a set of things one person can be asked to produce. The definition stops there, and everything difficult about accountability falls straight out of it. A duty nobody can produce evidence for is not a duty, and a name attached to duties whose evidence is posted to somebody else is a name that will fail at the first question asked of it. So the useful question is never whether a firm has named somebody. The question is what that person could put on the table if somebody walked up to them this afternoon, and at Sumeru Bank Limited, invented, that question turns out to have an exact answer.

What is a model owner, and what does the title actually mean?

The shape is clearer outside a bank altogether. In a block of flats in any Indian city, the lift has a name against it. Not the name of the engineer who designed the lift, and not the name of the contractor who services it. The name is one resident on the building committee, and what that name means is narrow and precise. If the lift stops, that resident is the person the other residents ring. The resident has to be able to say when the lift was last serviced, who services it, what the contract covers and what happens on the morning the contractor does not turn up. The same resident could not repair a lift, and nobody in the building has ever suggested they should be able to.

A model owner is that name, written against something a firm runs. A named accountable personOne person, written into the record, who answers for a deployed component being what the record says it is. is one person, in the record, who answers for a deployed component being what the record says it is. Not the person who built it. Not whoever happened to be most senior in the room when it was signed off. One person who can be asked, and who can answer with something other than an opinion.

The title carries no authority to change the component, no obligation to understand its internals and no share of the outcome it produces. The title carries one obligation only: to be able to answer. Authority, understanding, outcome and answerability are four separate things, and people fold them into one without noticing. Folding them in turns a job about records and arrangements into a job about mathematics, and mathematics is what makes the role sound impossible to fill. Pull them apart and the role becomes ordinary. Somebody has to be reachable. Somebody has to hold the paperwork. Somebody has to notice when the date passes.

Now the counting. A firm can say all of this in a policy and still have named almost nobody. At Sumeru Bank Limited, a mid-sized Indian bank, a deliberate search at month 10 found 14 separate uses of artificial intelligence running across the bank. Six of those 14 carried a named accountable person, being 42.9 per cent. Read that on its own and the bank looks careless. Read it three ways and it becomes something more interesting.

All 6 of those names sat among the 9 uses that were on the register at all, so measured against the list the bank actually kept, 6 of 9 carried a name, being 66.7 per cent. Measured against the 11 uses that pass the bank's own consequence testThe wording a firm chooses to decide which of its uses need the controls at all, based on what the output does rather than how it was built., the wording it wrote to decide which uses need controls in the first place, 6 of 11 carried a name, being 54.5 per cent. Three denominators, one numerator, and all three readings are true at the same time. The six uses never move. Which number a bank quotes decides whether it sounds negligent, adequate or middling, and the only honest thing to do with that is to say all three out loud and let the reader pick.

Does accountability require being able to build the thing?

Most arguments about naming somebody stall at the same point, and they stall there every time. A business head says the honest thing: I cannot be accountable for that, I do not understand how it works. The objection sounds like modesty and is usually sincere. The objection is also a category error, and a test settles it in about four minutes.

AuthorshipHaving built the thing. A different question from accountability, and rarely the same person. is having built the thing. Take each of the six duties in turn and ask one question of it: could this be discharged by somebody who could not have written a line of the component? Not should it be. Could it be. Run the test honestly down all six and the answer comes back the same every time. Checking an entry, writing one plain sentence, listing what a component consumes, receiving a report, arranging a route back and setting a date are all things a person can do without ever seeing the inside of the thing.

At Sumeru Bank Limited this is not a hypothetical. Revathi Balan, head of retail credit, is the named accountable person for component 6, the scoring model sitting inside the retail loan intake and decision chain. She built no part of it and could not have. Saying so plainly matters. The softened version, in which she helped a bit and understands the broad shape, quietly reintroduces the very requirement the test removes. She did not build it. She was still the right name on it, and the reason is the test.

There is a second reason to keep the two apart, and it has nothing to do with fairness. If accountability required authorship, then the accountable person for every bought component would sit outside the firm, at a supplier, beyond anybody's reach. Four of this bank's 14 uses sat inside a service bought rather than built. Requiring authorship would leave those four with no reachable name at all. Naming somebody is for exactly the opposite.

What are the six duties, and what is each one evidence of?

Here is the list Sumeru Bank Limited settled on, in its own numbering. Read it once for what it says. Then read the right hand column, the part that decides whether any of it is real. Every duty is paired with the thing the named person hands over when somebody asks. A duty with no artefact beside it is a sentiment, and a sentiment cannot be produced on request.

Now read the third column again on its own. Not one entry in it is a model, a measure or a line of code. Every single one is a document, a distribution list or a date. Five of the six are arrangements somebody makes once and then maintains. The sixth is a date somebody sits down and sets. Documents, distributions and dates are the entire job. The role can therefore be filled by the person who runs the business the component serves rather than by the person who wrote it.

Why does duty 1, that the entry is true, come first?

Duty 1 is first because it is the only one whose failure hides the other five. If the entry describes something other than what is running, then the sentence in duty 2 describes the wrong thing, the field list in duty 3 covers the wrong inputs, the monitoring in duty 4 watches the wrong behaviour and the route back in duty 5 restores the wrong process. Everything downstream of a false entry is a confident, well evidenced answer about something that is not there.

A false entry is not a theoretical risk at this bank, and the shape the failure took is worth sitting with. Entry 1, covering the intake and decision chain, counts as complete at the month 12 validation. The entry got there carrying eleven filled fields and one empty one, and it carried that empty twelfth field straight through the window in which a change made in month 7 sat unrecorded. In month 7 the waiting time before escalation in one of the chain's written components was changed, with no approval taken and no note made anywhere. The entry still read as true. Eleven of its twelve fields were true. The twelfth was the date of the next re-approval, and it was empty.

So duty 1 is not a filing duty. Duty 1 is a comparison duty, and the comparison has to be made against the running component rather than against the paperwork. The question a named person asks of their own entry is not whether the fields are filled. The question is whether somebody who read the entry, and then watched the component run for an hour, would recognise the second from the first.

Who does duty 4, the monitoring, actually have to reach?

Put a smoke alarm in a house and wire its bell into a room nobody sits in. Everything about that alarm works. The alarm senses smoke, it triggers, it rings. An alarm is defined by where the sound arrives rather than by where the sensor sits, so for every practical purpose the thing on that wall is not an alarm. Duty 4 is exactly this, and it is written the way it is for exactly this reason.

The duty is not that monitoring exists. The duty is that the monitoring reaches the named person, and the two are separate facts that firms report as one. A firm asked whether its components are monitored will say yes and be telling the truth. A firm asked who receives that monitoring, by name, and on what day of the month, often cannot answer at all. The second question is the one duty 4 asks.

Duty 4 is the commonest of the six to fail, and the reason is that nobody has to do anything wrong for it to fail. The monitoring gets built by the team that builds the component. Nobody else is in the room at the time. The report lands in that team's inbox because the team asked for it. Nobody makes a decision to exclude the named person. The name goes on months later, the distribution list is never revisited, and the accountable person is left answering for behaviour they are not sent.

Why does duty 6, the stated date, decide whether anybody ever looks again?

Five of the six duties describe a state of affairs. Duty 6 is the only one carrying a clock. A re-approval dateThe date on which an approval ends unless somebody deliberately renews it, so the record itself asks the question again. is the date on which the approval ends unless somebody renews it, and its whole function is to make the record ask a question without anybody remembering to ask it.

An approval with no end date is not a long approval, it is a permanent one, and it became permanent by accident rather than by decision. Nobody in the room at month 0 intended to approve anything forever. The date field was simply left empty. To everything that happens afterwards, an empty date field is indistinguishable from a decision that the component never needs looking at again.

At Sumeru Bank Limited the scoring component was approved at month 0 with no re-approval date set. Twelve months of live decisions followed. The month 12 validation was what finally set a date. Setting it there rather than at the approval is the wrong way round: the person doing the checking had to supply the date that was supposed to have triggered the checking. The failure is that ordinary, and it happens almost anywhere.

What can a named person be asked for, and what can they not?

A role is defined as much by its refusals as by its duties, and this one has four refusals worth writing down before anybody takes the name. Asking for the wrong thing produces either a bluff or a resignation, and both of those cost more than the naming saved.

A named person can be asked for the six artefacts and nothing else: the entry, the sentence and its limit, the list of what the component consumes, the most recent monitoring report addressed to them, the written route back with the date it was last exercised, and the date the approval ends. Every one of those can be produced on the day it is asked for, or it cannot, and either answer is useful.

A named person cannot be asked for anything requiring them to be the component: they cannot be asked to make it more accurate, to explain any individual output mechanistically, to validate it independently, or to carry the outcome personally. The first belongs to whoever builds it. The second is a question about the component and is answered by looking at the component. The third has to sit with somebody who did not build it and does not answer for it, and at this bank that is a separate person in the risk function. The fourth is a firm's accountability and cannot be handed to an individual, whatever a policy says.

Agrawal, Gans and Goldfarb, in Prediction Machines, 2018, put the useful frame on this. A learned component produces a prediction. A person or a rule still has to decide what to do with the prediction, and the judgement about what to do is a separate thing from the prediction itself. The named accountable person sits on the judgement side of that line. Asking them to improve the prediction is asking them to do a job that is not theirs. Asking them which decisions the prediction is allowed to feed is asking them exactly the right question.

Where should the name sit, and what are the two places it should not?

There is one question, and it settles almost every case. Can this person already be asked for the outcome the component produces? Not the component. The outcome. If somebody already answers to the board for the quality of retail credit decisions, then they can already be asked why decisions look the way they do, and the component is simply one of the things producing them. Naming that person adds no new accountability to the firm, it just writes down an accountability that already existed and points it at a specific thing.

Naming an accountability that already exists is why the role usually turns out to be fillable when a firm was convinced it was not. The bank was not looking for somebody to become accountable for the scoring model. The bank was looking for whoever was already accountable for credit decisions, and then writing that down.

Two places it should not sit. Not the team that built the component. Duty 4 asks that the monitoring reaches somebody, and a team receiving the monitoring on its own work leaves nobody outside the work to answer. And not whoever is most senior. Seniority does not make a person reachable for a particular outcome. A name three levels above the work is a name that will be told what to say. Being told what to say is the same as no name at all, except that it looks compliant on paper.

What could one named person actually evidence at one bank?

Everything above is a design. Here is what it produced when somebody independent walked in and asked. At the month 12 validation, carried out by a person in the risk function who had built no part of the chain, Revathi Balan was asked for all six of her duties. Four of the six she could produce there and then, being 66.7 per cent.

Duty 1, the entry, she produced and it was true. Duty 2, the plain sentence naming the component's job and its stated limit, she produced. Duty 3, what it consumes, she produced. Duty 5, the route back, she produced, and it was the strongest of the six. Manual underwriting existed at the bank before the chain did, and the route had genuinely been rehearsed. Four of six is not a poor result, and reading it as one blames a person for two addresses somebody else chose.

How many uses can one name carry before the name stops meaning anything?

Now the part firms almost never compute. The decision that produces it looks like good judgement at the moment it is made. A firm has six uses needing a name and one obviously capable person. Naming that person against three of them feels like quality control. Naming one person against three uses is also the single decision that concentrates the duties, and the concentration is arithmetic rather than opinion.

At Sumeru Bank Limited, the 6 uses carrying a named accountable person carried only 4 names. One person was named on 3 of them, and the other three uses took one name each. Three plus one plus one plus one is six. Put six duties behind each use and the arrangement contains 36 duties in total. One of those four names carries 18 of the 36, being 50.0 per cent of every duty the arrangement contains. Each of the other three carries 6. An even spread across four names would be 9 each. One person is at twice the even number and the other three are at two thirds of it.

The spanThe number of uses one name carries, and therefore the number of duties resting on that one person. matters because duties are not symbolic. Each one is an artefact somebody has to hold, maintain and produce on request. A name on three uses is not a title held three times. A name on three uses is eighteen documents, distributions and dates in one person's working life, sitting alongside whatever job they were hired to do.

Scale it once more and the picture gets sharper. A complete arrangement names somebody against all 14 uses the bank runs, and 14 uses at six duties each is 84 duties in total. At the four names the bank actually had, that is 21.0 duties a name. The bank is not currently carrying 21 duties a name. The bank is carrying 36 across four names and leaving the other 48 unattached, and both statements are the same fact from opposite ends.

What happens to the six duties when nobody is named at all?

The answer is the one thing people do not expect: nothing happens to them. The duties do not disappear when the name field is empty. The duties are properties of something running rather than properties of a job title, so a use with no name still needs its entry to be true, still needs somebody able to say what it does, still needs its inputs documented, still needs monitoring that arrives somewhere, still needs a route back and still needs a date. An empty name field changes exactly one thing: who can be asked. The answer becomes nobody.

Count it at this bank and the size of the gap is uncomfortable. Fourteen uses at six duties each is 84 duties that exist whether or not anybody has been named. Six uses carry a name, so 36 of those duties have somebody attached, being 42.9 per cent. The other 8 uses carry no name, so 48 duties, being 57.1 per cent, belong to nobody. Notice that 42.9 per cent is the same number as the share of uses carrying a name, and it is the same because every use carries the same six duties. The duty count adds no new information about coverage. The duty count adds a sense of size instead, and 48 is a very different thing to look at than 8.

None of the 5 uses the sweep found unregistered carried a name. A use nobody has written down is a use nobody has been asked to answer for. The five unregistered uses were not concealed. Every one of them was set up by somebody solving a real problem, and four of the five had told somebody. None of them had a line in a record with a name on it, and until that line exists there is no question anybody can be asked.

How is somebody named, in order, and what goes across the table with the name?

The order matters, and almost everybody does it backwards. The instinct is to find a name first. A name is what the policy asks for and what an auditor counts. Writing the name first creates a person who is accountable for arrangements that do not yet exist. The named person at this bank was put in exactly that position. Do the arrangements first and the name is the easy part.

Step one is to agree the six duties before any name is discussed, so the conversation is about a job rather than about a person. Step two is to find who can already be asked for the outcome the component produces. Step three is to route the monitoring to that person and change the distribution list at the same time. Duty 4 is an address rather than an intention. Step four is to set the re-approval date, and to set it to a real date rather than to a review cycle. Step five, and only now, is to write the name into the entry.

The six artefacts go across the table with the name, and the handover is the whole of the point. The entry as it stands today, the one sentence naming the component's job and the use it is barred from, the list of what it consumes, the monitoring report with the new name already on its distribution, the written route back with the date it was last exercised, and the date the approval ends. Handing over six artefacts makes the name real on day one. Handing over a name alone arranges for somebody to fail a validation twelve months later, politely and through no fault of their own.

How does an auditor, a board member or a buyer actually use the name?

Four people use the name in practice, and each of them uses it differently. Knowing which four matters before the arrangement is designed for any of them.

An internal auditor uses the name as an address rather than as a judgement. An auditor does not arrive intending to assess whether the named person is any good. An auditor arrives with six questions and a stopwatch, and the measurement is how long it takes to get an artefact. A person who produces four in the meeting and says plainly that the other two were never routed to them has given a better answer than a person who produces six with a week's notice. The first answer is about the arrangement, and the second is about preparation.

A board member uses the name to find out whether a list is real. Fourteen identical lines on a register tell a board nothing about size or exposure. A real arrangement is separated from a tidy one not by how many uses carry a name, but by how many names carry the uses. Four names against six uses is a very different answer from six names against six uses, and only the second one survives somebody going on long leave.

Somebody buying a component from a supplier uses the name to find the boundary. The supplier will answer for availability and for the service's stated behaviour. Nobody outside the firm will answer for duties 1, 2, 4, 5 and 6. All five are about the buyer's own record, the buyer's own monitoring and the buyer's own route back. At this bank, 4 of the 14 uses sat inside something bought rather than built, and the naming question on those four is not softer for having been bought. The question is harder. Evidence for duty 3 has to be asked for rather than looked up.

And a business head about to be handed a name negotiates before signing rather than after. There are exactly two things to ask for: the name on the monitoring distribution today, and a re-approval date in field 12 today. The monitoring address and the date are exactly what failed at this bank. Asking for both before the name goes on moves four of six to six of six for the price of two emails.