Generative AI: What Makes It Different From Earlier Models

A generative component produces new material, usually text, rather than a value drawn from a fixed set of answers. The difference in what comes back decides everything about deploying one: the output is read by a person rather than acted on by a system, it reads well whether or not anything supports it, and evaluating it means checking each assertion against something else rather than comparing one number to a right answer.

Scoring is where the difference bites. An earlier model was asked a question that had a right answer, so somebody could score it by holding what came back against what should have come back. A generative componentOne that produces new material rather than selecting a value from a fixed set of possible answers. is asked to produce something, so scoring it means a person deciding whether what it produced is supported by the material in front of them. A person's decision takes minutes rather than milliseconds, and any honest account of what one of these arrangements saves has to subtract that time.

What does generative actually mean here?

Picture two desks in the same office. At the first desk, a clerk reads a file and ticks one of four boxes: accept, decline, refer, or send back for documents. Whatever the clerk was thinking, the output is one of four things. Six months later somebody can pull the file, see what actually happened to that customer, and say the ticked box was right or wrong. At the second desk, a clerk reads the same file and writes a paragraph explaining what happened and why. There is no box. There is no list of correct paragraphs sitting somewhere to hold it against. Two careful people would write two different paragraphs and both could be perfectly correct.

Generative means the second desk. The component is not choosing from a set that somebody wrote down in advance; it is assembling material that did not exist until it assembled it. How the text gets put together is a separate subject, and none of it is needed here. The shape of the output changes everything downstream.

Take Sumeru Bank Limited, an invented mid-sized Indian bank whose retail personal loan intake chain runs through this whole subject area. Two of its nine components make the contrast cleanly. Component 6 is the scoring model, and what comes back from it is a value: high enough to accept, low enough to decline, or inside the bank's own referral range. Component 9, the workflow router, acts on that value straight away and no person is involved. Component 8 is the drafting assistant, and what comes back from it is English prose about a file. Nothing acts on it. Somebody reads it.

Why is that a difference in kind rather than a difference in capability?

The tempting reading is that a generative component is simply a cleverer version of what came before, further along the same road. It is not. A generative component is on a different road, and the difference shows most clearly in what it takes to find out whether one is working.

Sumeru Bank Limited can measure component 6, the scoring model, over a month of files without anybody reading anything. There are 8,600 decided files, each carries an outcome, and the comparison runs overnight. Measuring component 8 was a different exercise entirely. Two hundred exception notes were pulled, and people sat and read them beside the files they described. The question is not whether a number matched but whether a sentence was supported, and nothing but a person answers that, so there is no way to run it overnight.

Ajay Agrawal, Joshua Gans and Avi Goldfarb make the point across Prediction Machines that any learned component produces something a person still has to act on. With an earlier model the acting is often another machine, so the point is easy to miss. With a generative component the acting is unavoidably a person, so the point is unmissable and expensive.

What does the drafting assistant actually do in one deployed chain?

Nine components carry the artificial intelligence label inside the intake chain at Sumeru Bank Limited. Component 8 is the drafting assistant, and it is the only generative one. The assistant produces the first draftA first version produced for a person to check, change or reject before it takes effect. of exactly two things: the exception noteThe written record a desk officer makes on a file that the automation stopped, saying what happened and what should happen next. a desk officer writes on a file the automation stopped, and the explanation paragraph that sits inside a decline letter. The list ends there. Of the month's 8,600 files, 3,010 were routed to a person, so the assistant runs on 3,010 exception notes a month.

Notice what is missing from that list. Component 8 decided nothing. By the time it writes anything the decision has already been taken by something else: the scoring model, an income rule, a fraud rule, an identity check. Of the month's 8,600 decisions it determined none. Component 8 writes about a decision, never towards one. The distinction between writing about a decision and writing towards one is the reason the bank could place it where it did, and it is the first thing to establish about any generative component anybody proposes.

Who acts on the output, and where does the person stand?

Ismail Sheikh runs the exception desk at Sumeru Bank Limited, and the arrangement he signed up to has six stages, of which the component occupies exactly one. The decision has already been taken. The material from the file is assembled. Component 8 drafts. A desk officer reads the draft against the file, and the bank calls the stage verificationThe checking a person does on a draft before it takes effect, held against the material the draft is supposed to describe.. The desk officer signs. Only then does anything reach the applicant.

Drawn out as a path, the time and the accountability land in the same picture, and they turn out to sit in the same two stages. The person is not reviewing the component; the person is producing the document, using a draft. That sounds like a distinction without a difference until something goes wrong, at which point the question asked is who signed, and the answer has a name attached to it rather than a version number.

What did it save, before anybody subtracted anything?

The bank measured it properly. Not every bank does. Two hundred exception notes in month six, timed end to end. Before component 8, a desk officer wrote the note from the file in about 18 minutes. After, the officer read a draft, corrected it and finished it in about 4. The fall is 14 minutes, and it is a real, measured, honest number.

The 14 minutes is also half a sentence. The other half is a stage that did not exist before and now does. The officer has to check the draft against the file, and that took 9 minutes. Before, one person did one thing for 18 minutes; after, a component does one thing for 4 minutes and a person does a different thing for 9, so the file costs 13 minutes instead of 18.

What does the saving look like once verification is subtracted?

Five minutes a note. Five minutes is the number to quote, and it survives contact with anybody who asks how the desk's capacity actually changed. Work it forward across the month: 5 minutes on each of 3,010 exception notes is 15,050 minutes, or about 250.8 hours. Those hours are real. A desk of seven people gets roughly a third of a person's month back, and gets it back on the most tedious part of the job.

Now work the other number forward, the one that leaves verification out. Fourteen minutes on 3,010 notes is 42,140 minutes, or about 702 hours. The gap between the two is not a rounding difference: the first figure is very nearly three times the second, and only one of them describes anything a desk head could actually plan around. The 9 minutes is not overhead somebody could sharpen away. Those nine minutes are the stage that finds the faults, and what they catch has been counted.

How is a draft checked before somebody's name goes on it?

Review it is not an instruction. Review it is a hope. Somebody handed a draft and told to review it will read it, find it reads well, and pass it on, and that is not a failing of the person; it is a failing of the instruction. A list turns a hope into a task somebody can actually finish. Sumeru Bank Limited settled on five checks, and the 9 minutes is what those five checks take.

How to Evaluate a Generative AI Output for Financial Work

The five, in the order the desk runs them. Check one: trace every figure back to the supplied material. Not to a remembered version of the file, and not to the number the draft repeats twice. A number repeated twice repeats the same error twice. The field itself is opened and looked at. Check two: trace every assertion back to a passage. Each claim in the draft should be pointable at something: a line in the statement, a rule that fired, a document that was or was not there. Check three: look for anything asserted that the material does not contain at all. Not a wrong figure and not a wrong reading, but a sentence describing something that never happened. Check four: confirm the required format and the required statements are present. A decline explanation that is accurate and missing the wording the desk is obliged to include is still not finished. Check five: where the material genuinely did not support an answer, confirm the draft handled that properly rather than filling the gap.

Which of the five catches most of what is wrong?

Of the 200 drafts, 23 carried a statement that was not in the file. The rate is 11.5 per cent, or about one note in nine. Twenty-one of the 23 were caught, and every one of them was caught by check three. Two were not caught, and those two were signed and sent. The check that finds almost everything is the only one of the five with nothing in the document to point at it. Having nothing to point at makes it the slowest, and the first to be dropped when the queue is long.

Proofreading a letter somebody else wrote about a meeting both of them attended makes the difficulty plain. A wrong date is easy to spot. The date is right there and can be checked. A described conversation that never happened is much harder. Nothing about the sentence looks wrong. The invented sentence reads exactly like the sentences either side of it. Only holding the whole letter against what was actually said catches it, and that is slower and more tiring than checking a date. Nothing in the document helps.

Why does a draft that reads perfectly settle nothing?

FluencyText reading correctly and confidently, which is a property of the writing rather than of the content. is a property of the writing and support is a property of the content, and the two are not connected. Every one of the 200 drafts read as though it had been written by somebody sitting with the file open. All 200. Twenty-three of them contained something the file never said, in exactly the same steady tone as the sentences around it. There was no wobble, no hedge, no change of register at the point where the text left the material behind.

The disconnection runs against everything a career of reading documents teaches. A reader learns to trust the confident memo and to slow down at the one that reads oddly. The faults arrive dressed exactly like the rest. An instinct trained on how a document reads is not merely useless here. It points the wrong way. An unsupported statementAn assertion in the output that nothing in the supplied material contains. is not a property that shows up in the prose. The fault is a measured rate instead: at this bank, in month six, 11.5 per cent of drafts.

Here is one of them, reconstructed from the desk's own template. The income corroboration rule routed the file. The rule compares a declared monthly income against the median monthly salary credit over three months of statement. The declared figure was Rs 45,000/- and the corroborated figure was Rs 38,000/-, a gap of Rs 7,000/- or 15.6 per cent. The gap is outside the tolerance the bank chose, so the file stopped and went to the desk. Four of the sentences below can be traced to the file in seconds. One cannot.

Where should a generative component never be placed?

Two questions settle it, and neither of them is about the task. The first is whether a person will actually read every output before it takes effect. Not whether somebody could, not whether a policy says they will, but whether the volume, the staffing and the clock allow it on the worst day of the month. The second is whether that person will have the material to check it against, open in front of them, at the moment they are reading. Two noes and the component does not belong there. One no and there is work to do before it does.

Both questions are about what happens after the output. Teams that argue about the task instead get the placement wrong. A household version makes it plain. Somebody might happily let another person draft a reply to a difficult letter on their behalf, so long as they were going to read it before it went, with the original letter beside them. The same person would not let it be posted unread, and could not read it usefully without the original letter to hand. Nothing about that judgement depends on how good a writer the drafter is.

How does a desk head, an internal auditor or a board member actually use this?

Ismail Sheikh, running the exception desk, uses it to staff. His number is 5 minutes a note, never 14. Five is what actually appears in his capacity. Fourteen is what appears in somebody else's paper. When a proposal reaches him promising that a component will take work off his desk, the first question he asks back is what the checking costs, and if nobody has measured it, the honest position is that the saving is unknown rather than large.

Neelima Rao, in the risk function and having built no part of the chain, uses it differently. She is not interested in the saving at all; she is interested in whether the checking exists as a task rather than as an instruction, whether there is a list, and whether the list has a check three on it. An arrangement where the person in the path has no written list is one where nobody can say what was checked, and that is a governance finding whatever the fault rate turns out to be.

A board member, or anybody reading the paper rather than writing it, uses one habit: never let the saving and the fault rate arrive in separate sentences. The deployment at Sumeru Bank Limited saved 5 minutes a note and produced an unsupported statement in about one note in nine, and the checking that finds them is exactly what the 9 minutes buys. Said together, those facts describe a working arrangement that somebody chose deliberately. Said apart, the first is a sales pitch and the second is a scare, and neither is the truth of it.