Neural Networks and Deep Learning: When More Layers Help

A neural network is an input layer, one or more hidden layers, an output layer, and a set of weights between them that were fitted from past examples rather than written by a person. Deep learning means several hidden layers instead of one. Extra layers let a component handle harder inputs and cost the ability to state, in words a customer would accept, why it produced any particular answer.

Nothing inside that structure holds a reason. A weightA fitted number setting how strongly one unit's value carries into the next. is a number that survived fitting because it lowered the error across a great many examples, and no single weight and no single layer corresponds to a step in an argument. The missing reason is not a shortcoming of anybody's build. The missing reason is what a network is. A deployer either accepts that in a given place, or does not put one of these there.

So the interesting question is never whether a component ought to have more layers. The question worth asking is what the extra layers are buying, what they are costing, and whether the place they are going can afford the loss. Sumeru Bank Limited, an invented lender, answered that question twice on the same day, in opposite directions, for two parts of the same lending system.

How is a neural network actually put together?

Picture a room with four rows of clerks. The first row is handed the numbers read off a form. Every clerk in the second row reads all of the first row's numbers, multiplies each one by a figure written on a card in front of them, adds up the results and writes down a single number of their own. The third row does exactly the same to the second row's numbers. The last row produces four numbers, one for each answer available, and the largest of the four is the answer the room gives. Nobody after the first row ever sees the form.

The room is a neural networkA component built as layers of connected units with fitted weights between them. with nothing left out. A network has four named parts and no fifth one. An input layerOne rank of units, all reading from the rank before it and feeding the rank after. holds the numbers read off the thing being classified. One or more hidden layersAny layer between the inputs and the output, whose values nobody reads directly., called hidden for the unexciting reason that nobody reads their numbers on the way past. An output layer, one unit for each answer the component is allowed to give. And the weights: the figure on each clerk's card, one for every line running from one unit to the next.

All of the behaviour lives in the weights, and the weights were arrived at by fitting rather than written down by anybody. The layer count only decides how many times the numbers get combined before they come out the other end. Being slightly unimpressed by all of this is the right response. The awkward part of the subject comes from how ordinary the structure is, not from how clever it is.

Sumeru Bank Limited runs one of these as component 3 of its intake chain, the document classifierA component that decides which kind of document an image is. that decides whether the image a customer has just uploaded is a salary slip, a bank statement, an identity proof or an address proof. Numbers read off the image go in at one end and four numbers come out at the other, and the largest of the four is the kind of document the bank then treats it as.

What is a weight, and why does it hold no reason?

A weight is one number on one line, and nobody chose it. The weight settled where it settled during fitting, a process covered separately. No document anywhere records why it came out at one value rather than another. There is nothing to read. Nothing was ever written down, so there was never anything to read.

Against that sits the written half of the same system. Component 5 of the intake chain, the step that checks declared income against the statement, is a procedure 34 lines long. Neelima Rao in the risk function read all 34 of those lines in 25 minutes and could then say exactly what the component would do to any file put in front of her. There is no equivalent of that reading anywhere inside a network, and no amount of care by whoever built it can put one there.

The second thing catches deployers out. Every unit in a layer reads every value from the layer before it. So one number in the second row is already a blend of all five numbers in the first row, and after four rows the output is a blend of a blend of a blend of a blend. The structure never held that information in the first place. Nothing in it records which of the original numbers mattered.

The shape of this is familiar from ordinary life. A household closes the month Rs 3,000/- short. The shortfall is genuinely made of about four hundred small decisions across thirty days, and no single one of them caused it. A story can be told afterwards about the wedding gift or the extra autorickshaw fares, and the story may even be useful, but it is a story constructed after the event rather than a record kept during it. The network is in exactly that position on every single document.

What does deep learning add, and how many layers make something deep?

Deep learningBuilding with several hidden layers rather than one. means building with several hidden layers rather than one. Several hidden layers rather than one is the whole content of the word, and anyone who says otherwise is selling something. Deep learning is not a different kind of mathematics, not a different kind of component and not a different kind of governance problem. A deep build is the same four parts with more of the second one.

How many is several? There is no agreed count. The word was never defined against a threshold. Five practitioners asked where deep starts will answer three, five and ten, and none of them will be wrong. The layer count is a description of a build rather than a category a build falls into, and chasing the number at which several becomes deep is the most reliable way to waste a meeting.

The pair of questions the count hides is worth a meeting. Ask what the extra layers bought on the task actually at hand, and what they cost on that same task. Both of those can be measured on the build in question. The definition cannot. Sumeru answered both of them with a number, and its committee could then argue about the answer rather than about the word.

What do more layers actually buy?

Harder inputs, read correctly. The answer is honest, and it is narrower than it sounds.

Think about what actually arrives at Sumeru. An applicant photographs a salary slip on a handset at a kitchen table at ten at night. A thumb covers one corner. The paper is folded across the middle so one line of figures bends away from the camera. A bank statement arrives as a photograph of a laptop screen, with the room reflected in it. An identity proof arrives with an office stamp printed across the text, so the letters underneath are half ink and half paper. Every one of those is still, obviously, a salary slip or a statement or an identity proof to a person glancing at it for one second.

A component with a single hidden layer combines what it reads once, and then has to give an answer. A single combination is enough when the useful signals are more or less directly readable: this region is dark, this block sits in the upper third, this line has more digits than words. A single combination is not enough when the useful signal is itself a combination of combinations. A folded, stamped, half reflected document carries exactly that. Extra layers do not add information to the input; they add ways of combining the information that is already in it.

Which is also why the honest answer to what more layers buy is: it depends entirely on the task, and mostly on how messy the inputs are. On tidy inputs the first combination already found what there was to find. The extra layers then buy close to nothing. On the documents Sumeru's applicants actually upload they bought something, and the bank measured how much rather than assuming it.

What do more layers cost, and who pays it?

One sentence. Specifically, the sentence somebody at the bank can say about one particular answer.

At two hidden layers, a reviewer at Sumeru could look at a single classification and name the feature that drove it: this was read as a salary slip mainly because of the stamped block in the upper third. At four hidden layers the bank could not produce that sentence for any one document. Not that the sentence was hard to produce, or expensive, or needed a specialist. A value four layers deep is a blend of a blend of a blend, and the structure never kept a record of which strand was which. There was nothing to produce the sentence from.

So the cost of depth is not money and it is not accuracy: it is a reportable reasonA statement about why one answer came out, in words the recipient can act on., and whether that cost is bearable depends entirely on who was going to ask for one.

In most of life nobody asks. If a photo application on a handset picks the wrong filter for a family picture, no explanation is owed to anybody and none is missed. Finance is not most of life. A person refused a loan can ask why, a supervisor can ask why, and the answer has to be in words the person on the other end can do something with. The obligation to answer is the whole reason this trade is a governance question in a bank and a shrug almost everywhere else.

What did one bank measure when it built the same classifier twice?

Sumeru built component 3 twice before deploying either version, on the same four kinds of document and the same fitting examples, changing the layer count and nothing else. The shallow build used two hidden layers and read 91.5 per cent of documents correctly, and a reviewer could name the feature that drove any given classification. The deeper build used four hidden layers and read 96.2 per cent correctly, with no equivalent statement available for any one document.

Both figures belong to one bank, one task, and that bank's own four kinds of document. Neither figure says anything about what such components achieve anywhere else. A different bank with different documents would measure something different, and measuring it was therefore the whole point rather than looking the number up.

The gap is 4.7 percentage points, and 4.7 is the price of the explanation on this task. Not the price of explanations in general, and not a rate anybody can quote. One number, one task, one committee, one afternoon.

Percentage points are hard to feel, so translate them into the month. Sumeru's applicants send 4 documents on each of the 8,600 files that reach the decision engine, for 34,400 documents in a steady month. At 91.5 per cent, 2,924 of those are read as the wrong kind of document. At 96.2 per cent, 1,307 are. The deeper build misreads 1,617 fewer documents a month, more than half of what the shallower one got wrong.

Which rule decided where that cost was accepted, and where it was refused?

The same committee, in the same session, took the deeper build for the document classifier and refused it for component 6, the scoring model. The pair of decisions looks inconsistent for about four seconds. The pair is consistent, and what separates the two cases has nothing to do with layers, methods or how modern anything is.

The question is: can this output refuse somebody, and does a reason have to be stated?

For the classifier, no. No applicant is ever refused by a document classifier. The classifier decides what a document is. Its output never leaves the system. When the classifier gets one wrong, the file either fails a later check or lands on Ismail Sheikh's exception desk, where a person opens it and looks. Nobody is owed a sentence, so giving up the sentence costs nothing that anyone will ever ask for.

For the scoring model, yes. The scoring model declined 688 applications in that same month, and a declined applicant can ask why. Revathi Balan, head of retail credit, is the named accountable person for that component, and she is the one who would have to say the sentence out loud. Sumeru refused the trade there and kept a structure it could report reasons from, accepting whatever accuracy that cost.

The rule is about the person on the other end, not about the method. One committee could therefore take the accuracy in one place and refuse it in another on the same afternoon without contradicting itself once.

Bias vs variance: what does the layer count actually trade?

Underneath the layer count sits a trade with a name older than any of this, and it is worth having the name because it stops the argument going in circles.

Set the count too low and the component cannot represent what it is looking at. The component is not confused by particular documents. The error runs in a consistent direction on nearly all of them, in the way a straight ruler is wrong about a curve everywhere at once. Error of that consistent kind is called biasError that comes from a component being too simple to represent what it is reading., and adding layers reduces it.

Set it too high and the opposite thing happens. The component now has enough structure to follow the particular examples it was fitted on very closely, including the accidents in them: the lighting in one channel's photographs, the way one form's boxes happen to line up. The component carries those accidents forward as though they were the point. A build like that is right on documents like the ones it learned from and wrong on documents that arrive a little differently. Error of that particular kind is called varianceError that comes from a component tracking the particular examples it was fitted on., and adding layers increases it.

One dial, two failures, pulling in opposite directions, and no setting that removes both. Every layer count anybody ever chooses is a position on that trade, whether or not the person choosing knew they were choosing.

How that trade is measured, how a set of examples is put together to measure it on, and the statistics underneath the two words are covered separately. A deployer needs something narrower, and it holds regardless: the dial exists, it has a bad end in both directions, and the position on it is found by measuring on the task at hand rather than by reasoning about the count.

Too few layers and too many: how does one component fail in two opposite ways?

Sumeru did not settle the layer count by argument. The bank built the classifier at every count from one hidden layer to six and took two readings each time. The first reading is the share of documents read correctly on documents like the ones it was fitted on. The second is the share read correctly on the same four kinds of document arriving from a channel that was not in the fitting set: same documents, same task, different origin.

Before the readings appear, a guess is worth committing to. The shape of the second reading is what almost nobody predicts.

The first reading climbs the whole way, from 88.0 per cent at one hidden layer to 98.1 at six. The second reading climbs to 93.1 at four layers and then falls. By six layers it is down to 87.2, worse than the two layer build managed. A deployer watching only the first reading would have taken six layers and shipped the worst performance on new documents of any build on the table. Every number on the report in front of him went up.

O'Neil's point in Weapons of Math Destruction arrives in a very ordinary form: a component's errors do not fall evenly. The errors collected on documents from one channel. In practice they collected on whichever group of applicants happened to use that channel. Nobody chose that, and nobody would see it from the headline reading either.

Why does asking which layer refused the application have no answer?

Six weeks after go-live, an applicant asked Sumeru which part of the system had refused the application. A manager answered by naming the third layer of the model.

There is no sense in which that is true, and it fails in three separate ways at once. No layer holds a reason, so there is nothing in the third layer that could be the reason. Every value in every layer is a blend of everything before it, so no layer produced the answer on its own. And the same answer would have come out with the weights arranged quite differently, so even the arrangement is not the explanation.

The real danger is not that the answer was wrong; it is that it sounded mechanical enough that nobody checked it. An honest answer, that the structure does not hold a reason and the decision rests on something else, invites the next question. Naming a layer closes the conversation. Closing the conversation is precisely what made the answer spread.

How does a lender, a reviewer or a board actually use this?