Tool Calling: Letting a Model Use Systems Rather Than Guess

Tool calling means giving a component a way to ask another system for something during a request, instead of producing it from text. The distinction that matters is between fetching and producing. A figure held in a system of record has a definite right answer, so it should be fetched. A characterisation of a situation has no such answer anywhere, and it has to be produced and then checked.

The split between fetching and producing is worth setting out on its own. Every unsupported figure in the case this sequence has been working through existed because a component was asked to write out a number it had no way of looking up. Guessing a number is not really a limitation of the component. Guessing is a design decision, made by whoever supplied the component with prose and no route to the record. Guessing is also the one part of the whole problem that can be removed completely rather than merely reduced. Almost nothing else in a deployment of this kind can be removed completely, so this small stretch of plumbing carries more weight than its size suggests.

What is tool calling, once something is actually running?

A kitchen table is the plainest place to start. Somebody is filling in a form and asks what last month's electricity bill came to. There are two ways to answer. About Rs 2,400/- sounds like what the bill usually is, so the first way is to say that. The sentence produced will be fluent, confident and roughly the right shape. The second is to get up, open the drawer, find the bill and read the figure off it. Both routes end in a sentence. Only one of them ends in a sentence with a right answer standing behind it, and the difference between the two is not how careful the answerer was. The difference is whether somebody got up and opened the drawer.

Tool callingGiving a component a way to ask another system for something during a request, rather than writing it out from its own text. is the drawer. A component that produces text runs a request straight through from the material it was given to the words it hands back. Give it a connectionA defined route from the component to a system that holds something it needs. and the shape of the request changes: it runs until it reaches a point where it needs a value, stops, asks the system that holds that value, receives an answer, and carries on writing with the answer in hand. The pause in the middle is the whole of the idea, and the pause sits exactly where the guessing used to be.

The pause leaves a great deal unchanged. The component still writes the sentence. The component still chooses the words, the order and the tone, and it can still be wrong about what the sentence means. The pause changes one thing only: where the number inside the sentence came from. An account that treats tool calling as a general upgrade has missed this. Tool calling is not general at all. Tool calling is narrow, and its narrowness is the reason it can be measured.

Why is fetching different in kind from producing?

Filing the split under reliability is tempting, as though fetching were simply the more accurate way of getting to the same place. Reliability is the wrong shelf. FetchingObtaining a value from a system that holds it, rather than writing it out. and producingWriting something out from the component's own text generation, with nothing to look it up against. answer different questions, and one thing separates them: whether a right answer exists anywhere at all.

A wedding has the same shape. Somebody asks the caterer how many guests have confirmed. There is a list. The list is kept by one person, everyone agrees it is the list, and if two people disagree about the count then one of them is wrong and the list settles it. Now somebody asks the same caterer whether the crowd will be happy with the menu. There is no list for that. A considered answer is possible, and a good caterer gives a much better one than a bad caterer, but there is no drawer anywhere in the building that can be opened to settle it. The first question has a right answer sitting somewhere and the second does not, and that is a difference in kind rather than a difference in how reliable the answer is.

In a bank the list is called a system of recordThe system whose copy of a value is the one that counts when two systems disagree.. The system of record holds the copy that counts. When the loan origination screen says one thing and a spreadsheet says another, everybody knows which one to believe. If a value has a system of record then it has a definite right answer, and a component producing it from prose is being asked to reproduce something that already exists, from memory, in a room next door to the filing cabinet. If a value has no system of record then there is nothing to connect to, and no amount of connection helps.

Which parts of an output have a right answer sitting somewhere?

The useful thing about that test is how fast it runs. Applying it needs no knowledge of the component at all. One element of the output is taken, the question is whether some system in the building holds the right answer for it, and the element sorts itself. Asked of a customer's account number, the answer is immediate. Asked of the date a file arrived at the desk, the answer is immediate. Asked of whether the file looks like a case worth a second look, there is no system in the building that holds an answer, and no amount of engineering will produce one.

One question sorts almost every element of a financial output onto one side or the other in a couple of seconds, and the elements that resist sorting are the interesting ones. A figure that three systems hold differently is not a fetch problem, it is a records problem wearing a fetch problem as a disguise, and connecting a component to one of the three does not settle which of them is right.

What can never be fetched, and what follows from that?

Take one exception note apart. At Sumeru Bank Limited, invented, an exception note is what a desk officer writes on a file that stopped: what stopped it, what the numbers were, what the file amounts to and what should happen next. The drafting assistant, component 8 of the intake chain, produces the first draft of that note and a person reads, corrects and signs it. Sort its elements with the one question and the note falls into two unequal halves.

Five of the seven elements have a right answer somewhere in the building. The applicant's declared monthly income is a structured fieldA value held as a field in a system rather than described inside a paragraph of text. on the file. So is the corroborated monthly salary credit that the income rule compares it against, so is the amount requested, so is the coded exception cause that stopped the file, so is the date it reached the desk. The remaining two are of a different order entirely. A sentence saying what the file amounts to is not held anywhere. Neither is what should happen next and why. The two unfetchable elements are exactly the part a person has to read and think about, and removing the five fetchable elements does not remove that reading, it just stops the reading being spent on arithmetic.

The uneven split settles what a check is for, and it matters more than it looks. If the whole note were fetchable, the check would be a comparison and could eventually be done by another machine. If none of it were fetchable, the check would have to cover everything and would never get cheaper. The real note sits between the two, so connecting the fetchable parts makes the check smaller and more focused rather than unnecessary. A desk that expects the check to disappear will be disappointed. A desk that expects the check to concentrate on the two elements no system can settle will find that is exactly what happens.

What did this bank actually connect, and what did it read?

Now the case. The reading step of the intake chain pulls 14 fields from each file, and those 14 fields are the material a note gets written about. Until month 9 the drafting assistant saw them only as prose: the file's contents were handed to it as words, and any figure it put in a note was a figure it had written out from those words. In month 9 a connection was added letting it read the structured fields directly instead.

Take the income case the intake chain works with. An applicant declares Rs 45,000/- a month. The corroborated figure, being the median monthly salary credit over three months of statement, is Rs 38,000/-. The gap is Rs 7,000/-, or 15.6 per cent of the declared figure. The bank's own chosen tolerance is 10 per cent, so the file is routed to a person. All three of those numbers are fields. Every one of them has a system that holds it, every one of them has a right answer, and until month 9 all three arrived at the component as sentences and left it as sentences it had written itself.

Three fields out of the 14, and they were chosen by how often a note restated them rather than by anything about the systems behind them. The ordering is the whole trick: connect in the order the desk actually reaches for a value, and the work runs out long before the field list does. Each connection took about 2 working days of somebody's time, so the three cost about 6 working days in total. At this bank's own 7 hour working day that is about 42 hours, and it is arithmetic on the bank's assumed working day rather than a separately measured amount.

What did the one connection deliver, and why can it be attributed?

Component 8 was measured twice on 200 notes each time. In the first measurement, at month 6, 23 of the 200 drafts carried a statement that was not in the file, being 11.5 per cent. Six of those 23 were restated figures and 17 were other kinds of statement, and 6 plus 17 is 23. Of the 200 notes, 31 restated a figure at all, so 6 of 31 were wrong, being 19.4 per cent. In the second measurement, at month 10, the whole rate had fallen to 7 of 200, being 3.5 per cent, and 34 notes restated a figure with none of them wrong.

Three changes sat between those two measurements and nobody at this bank separated them. In month 8 a retrieval arrangement was added, so the component could be handed passages from the bank's own document store. In month 9 part 6 was added to the instruction, telling the component what to do when the material does not support an answer, and the connection to the structured fields was added. The improvement of 8.0 percentage points therefore belongs to all three together, and every honest statement of it says so.

Except in one place. Split the 23 by category and the arithmetic goes further than the headline can. Restated figures fell from 6 of 200, being 3.0 percentage points, to 0. Other statements fell from 17 of 200, being 8.5 points, to 7, being 3.5. The two movements are 3.0 points and 5.0 points and they sum to the 8.0 points of the headline. Now ask which of the three changes could have touched the figures, and the answer is exactly one.

Retrieval brings back passages from a store of policies, procedures and product terms. Nothing in that store holds this applicant's declared income, so retrieval could not have moved a figure about one file in either direction. Part 6 covers the case where the supplied material does not support an answer, and for these six the material did support an answer: the figure was there, in the file, in prose, and the component restated it wrongly anyway. Part 6's condition never fires on a case like that. The connection is the only change of the three that reaches this category at all, and it removed the category entirely.

So the honest ledger reads: 16 of the 23 unsupported statements were removed, 6 of them by the connection and 10 shared between the retrieval and the instruction with no way to split them. The connection accounts for 37.5 per cent of what was removed, and it is the only 37.5 per cent in this whole case with one cause standing behind it. Everywhere else in this deployment, an improvement has to be reported as the joint work of things nobody separated.

One detail in that second measurement is easy to skim past and worth stopping on. Before the connection, 31 of 200 notes restated a figure. After it, 34 did. More notes reached for a figure once reaching was cheap and safe, not fewer. A connection does not only make an existing behaviour more accurate, it changes how often the behaviour happens at all, and a team that budgets only for accuracy will be surprised by the volume.

Two things this case does not settle. Of the 23 unsupported statements in the first measurement, 21 were caught in verification and 2 reached a customer, but the bank never recorded which category those 2 came from, so nobody can say whether either was a restated figure. And a measured zero in 34 notes is a measured zero, not a promise about every note thereafter. The bank did not claim it was one, and the ledger above holds either way.

How many fields had to be connected before the benefit ran out?

Here is the part that changes how a team sizes this work. The reading step pulls 14 fields. Connecting all 14, at about 2 working days each, is about 28 working days. At this bank's 20 working day month, 28 days is about a month and a half of somebody's time. A month and a half is a real project, and a real project is what the field count makes the work look like.

Connecting all 14 was never the work. All 6 of the first-measurement figure errors sat in just 3 of the 14 fields, and they sat there in the order the desk reaches for them: 3 errors in the most restated field, 2 in the next and 1 in the third. Connect the first field and 3 errors remain. Connect the second and 1 remains. Connect the third and there are none left, and every connection after that removes nothing at all, at 2 working days each. Errors follow usage and usage is concentrated, so six working days of the twenty eight removed the whole category and the other twenty two would have bought nothing.

The concentration is not a coincidence and it is not special to this bank. A field a note restates often is a field with many chances to be restated wrongly, and a field a note almost never touches has almost no chances at all. So an error count sorted by usage falls away fast, and the practical instruction that follows is blunt: never scope this work by counting fields. Scope it by counting where the faults actually were, and connect in that order until the count reaches zero.

What happens when a connection can change something rather than read it?

A second connection was proposed at this bank and refused, and the refusal teaches more than anything the bank built. The proposal was small and sensible: let the component update the file's status when it has finished a note, so the desk officer does not have to. The proposal would have saved a step. Saying so is not a polite concession but the honest position. Across 3,010 notes a month, a step is a step, and the desk had every reason to want it.

The kitchen table has the same shape again. There are two things a customer can ask a bank to do. The first is to report the balance. Reporting leaves every decision about that balance exactly where it was, with the customer. The second is to set up an instruction that pays a bill automatically each month. The instruction is genuinely convenient, and it also means that from then on, for that one bill, nobody decides anything. The payment happens because the date arrived. Nothing announced the change. No meeting was held about removing the customer from the decision. The customer was removed from it by an arrangement made for convenience, and that is the whole of what separates a read connection from a write connection.

A read connectionA connection that can only obtain values from another system and cannot change anything there. leaves the arrangement exactly as it was. The component drafts, the person reads and signs, and the words only reach anybody because somebody put their name to them. A write connectionA connection that can change something in another system, so its effect does not wait for anybody to sign. is a different animal. The status would have moved when the component decided it had finished, not when anybody signed. For that one field, in that one system, the person would no longer be standing between the output and its effect.

What does a connection need recorded before it is allowed to exist?

Four things, and the fourth is the one usually missing. First, the systems a connection may reach. Second, what it may do there, meaning read only or read and change. Third, whose permissions it acts under: a connection acts as somebody, and if nobody has said who, it acts as whoever built it. And fourth, what happens to the output when the connection is unavailable.

The fourth item does more work than the other three put together. A recorded fallback decides what an ordinary outage produces. If the answer is recorded, a note comes back saying the value could not be obtained and naming which value. The refusal has the same shape as part 6 of the instruction, telling the component what to do when the material does not support an answer. If the answer is not recorded, the component does what it did before the connection existed: it produces the figure from the prose, fluently, and nothing in the note says the drawer was locked. An unrecorded fallback does not remove the connection's benefit on a bad day, it silently returns the arrangement to the state the connection was built to fix, with a month of measurement saying it was fixed.

The first two items look like engineering questions and are not. Permissions and access, covered separately, settle what a connection may reach and who decides it. All four items have to be written down somewhere a reviewer can find them. Neelima Rao, who does the independent validation here and built no part of the chain, asks for exactly this list, and the item she finds missing is the fourth one more often than any of the others.

What does this change for the person who has to check the note?

The desk is where this lands. Verification at this bank is five numbered checks, and the connection does not touch all five equally. Check 1 traces every figure back to the supplied material and it is the check the connection reshapes: a fetched figure carries where it came from, so tracing it is a comparison instead of a search. Check 3 asks whether anything has been asserted that the material does not contain. Check 3 is the slow one, and the connection does nothing for it at all: noticing something that is not there is not a lookup.

Ismail Sheikh, who heads the exception desk, reads that table as a rostering question rather than a quality one. His people verify every note the component drafts, and verification across the two measurements fell from 9 minutes to 6. He cannot attribute that fall to the connection either, for the reason that governs every measured improvement in this deployment: three changes, one measurement, and nobody separated them. What he can say is which check got structurally easier and which did not, and that is enough to know where the remaining minutes are going.

Scale it, carefully, and say what kind of number it is. At the first measurement's rates, 3,010 notes a month with 15.5 per cent restating a figure is about 467 notes reaching for a figure, and 19.4 per cent of those wrong is about 90 notes a month carrying a figure that was written out rather than looked up. About 90 notes a month is arithmetic on measured rates, not a count anybody at this bank made. The figure is still the honest size of what six working days of connection work was aimed at, and it is why the desk cared.

The same reasoning travels beyond a bank. A household treasurer filling a form, a street vendor pricing from memory instead of the wholesale slip, a caterer counting a guest list: wherever a number is written out because opening the drawer was inconvenient, the fix is the drawer and not more care. Care is what goes on the sentences no drawer can settle.