The Calibration Loss Function: What You Are Minimising

A calibration loss function is the rule that says what it means for a model to be close to its targets. The person doing the fitting chooses it, nothing in the observed prices selects it, and it decides the answer as surely as the targets do. Three defensible rules on one set of five prices return three different volatilities.

A fit needs three things before it can run. The first is a set of targets. The second is a model that can produce a number comparable with each target. The third is a rule saying which of two near misses is the better one. A model with one free number and five targets is never going to hit all five, so something has to decide where the search stops. The first two are always written down. The third decides the answer and almost never appears in the record.

The quantity being modelled is the standard process, written S with a time subscript, starting at Rs 100/-, over a horizon of one year, with a rate of 5 per cent a year. The targets are five invented observed prices at strikes of Rs 80/-, Rs 90/-, Rs 100/-, Rs 110/- and Rs 120/-, quoted as implied volatilities of 0.240000, 0.220000, 0.200000, 0.190000 and 0.185000.

The five prices never move. Fitting one constant volatility to them under three rules that a careful person could each defend returns 0.198202, 0.204420 and 0.207000. The gap between the outer two is 0.008798, close to a full point of volatility. Nothing observable moved to produce that gap, and nothing observable chooses between the three answers.

What is a calibration loss function?

The case sits away from any market entirely: one thermostat and five rooms. The rooms are at different temperatures and the aim is to have every one of them at twenty degrees. There is a single dial. Whatever it is set to, at most one room lands where it was meant to, and the other four sit above or below. So the question is no longer what the right setting is. The question is what the best setting means, and there is more than one honest answer. One aim is for the total discomfort across all five rooms to be smallest. Another is for the worst room to be as good as possible. A third disregards the storeroom entirely. Each of those is a different rule, each is defensible, and each turns the dial to a different place.

A loss functionThe rule saying what it means for a model to be close to its targets, chosen by the person fitting rather than read off anything. is that rule, written down. The loss function takes a candidate parameter value, works out what the model says under it, compares the result against every target, and returns one number saying how bad the whole fit is. The fit then searches for the parameter that makes that one number as small as it goes. Notice what has happened: five separate comparisons have been collapsed into one number, and a collapse of five things into one is a choice about relative importance whether or not anybody made it deliberately.

The definition above already contains the whole difficulty, and it is worth pausing on the shape of it rather than the notation. Two of the three inputs to a fit are things that are found. The targets are found, the model is chosen from a small number of named alternatives and then stated. The third input is manufactured on the spot, usually without comment, often by whoever wrote the fitting routine rather than by whoever will read its output. The loss function is the only input to a calibration that is manufactured rather than found.

What are the usual choices?

Three rules come up constantly, and all three are worked below on one set of numbers. The first is squared price errorAdding up the squared differences between model price and observed price, with every strike counted the same way. It returns 0.198202 on these five prices.: compare prices to prices, count every strike the same way, add the squares. Squared price error is the plainest rule a person can write, and it needs no extra machinery.

The second is vega weightingDividing each price gap by that strike's vega before squaring it, which restates every gap in volatility points rather than rupees. It returns 0.204420 on these five prices.. The same move in volatility produces a different move in price at each strike, so a gap of Rs 0.50/- does not mean the same thing everywhere. So divide each price gap by that strike's vega first, and square only afterwards. The division restates every gap in volatility points. The precise form is the part that most often goes unrecorded, so the name is worth stating exactly. The weight on each squared price gap is one divided by that strike's vega squared.

The third is squared implied volatility errorComparing volatilities to volatilities directly and forgetting prices, which counts every strike the same way. It returns 0.207000 on these five prices.. The targets arrived quoted as volatilities in the first place, so compare volatilities directly and leave prices out of it entirely. Squared implied volatility error is the only one of the three that never evaluates the pricing formula at all.

Why does the choice change the answer?

At this point an intuition that sounds right turns out to be wrong, and the mistake is worth walking into fully. The plain squared price error contains no weights at all, so it looks like the neutral rule, the one with no opinion. And because the biggest prices sit at the low strikes, at Rs 25.227000/- against Rs 2.745149/- at the top strike, it looks as though those big numbers must dominate a sum of squared rupee gaps and drag the answer toward whatever fits them. Both halves of that reasoning are wrong, and the arithmetic says so plainly.

Differentiate the loss and look at what appears. Setting the derivative to nought is the condition the fitted value satisfies, and the term that shows up beside every residual is the vega at that strike. Vega was never asked for. Vega arrives because the model price is what is being moved, and how much the model price moves when the volatility moves is the definition of vega.

Now look at where vega is large and where it is small on these five strikes. Vega is largest where the log of the ratio of starting value to strike sits near the middle of the distribution and falls away on either side of that. On this invented set it runs 18.080359, 28.592755, 37.524035, 39.485723 and 32.862160 per one unit of volatility, so it peaks near the Rs 110/- strike and is smallest by a wide margin at Rs 80/-. The strike carrying the largest price carries the smallest vega, and those two facts point in opposite directions.

Work the exact weights out and the intuition inverts completely. Take the share of the total of the five observed prices that each strike carries. The wrong story says that share should drive the fit. Then take the exact share of the fitted answer that each strike actually commands, computed further below. The Rs 80/- strike carries 41.1 per cent of the total price and 4.2 per cent of the fitted answer. The Rs 120/- strike carries 4.5 per cent of the total price and 22.4 per cent of the fitted answer. The squared price error does not weight the largest prices; it very nearly discards them.

So the plain price loss returns the lowest of the three answers, and the reason is the opposite of the tempting one. The Rs 80/- strike carries the highest observed volatility on this set at 0.240000. Moving the volatility barely moves the price at Rs 80/-, so the price loss listens to that strike least of all and has almost nothing to gain by trying. The highest reading is nearly discarded, the answer falls, and the fit ends at 0.198202. The largest miss on the whole fit, at minus Rs 0.662462/-, sits at exactly the strike the wrong story claims the fit is being dragged toward.

What does each choice implicitly weight?

There is a way of stating all three answers that makes them comparable, and it removes the last of the mystery. Every one of the three fitted volatilities is a weighted average of the same five observed implied volatilities. Not approximately, and not as a rough description. Exactly, to every decimal place. The three rules differ only in the weights they put on those five readings, and once the weights are written down there is nothing left to explain.

Read the third column down and the shape is unmistakable. Under the plain price loss the weight climbs from 0.041890 at the strike carrying the highest volatility to 0.308446 at the Rs 110/- strike, then eases back at Rs 120/-. The weight profile is not the price profile and it is not flat. The weight profile is the vega profile, squared and then bent slightly by the curvature of price in volatility. The heaviest weight sits where vega peaks, and on this set vega peaks near the middle of the strike range, firmly on the low volatility side of it.

Read the fourth column and something different is happening. The vega weighted rule puts 0.148766, 0.199854, 0.211964, 0.212630 and 0.226786 on the five readings. The five weights are nearly equal. Near equality is precisely what dividing by vega squared is for: one vega term in the effective weight is cancelled by the stated weight and the other very nearly is, leaving a rule that treats the five readings almost alike. Vega weighting does not concentrate the fit anywhere; it is a way of undoing the concentration the plain price loss carries. Vega weighting therefore lands at 0.204420, only 0.002580 from the plain mean, while the price loss sits 0.008798 away.

The fifth column is exact by construction and needs no computation at all. Comparing volatilities directly puts 0.200000 on each of the five readings, and a weighted average with equal weights is the plain mean. The five readings of 0.240000, 0.220000, 0.200000, 0.190000 and 0.185000 sum to 1.035000, and a fifth of that is 0.207000. The volatility loss returns the plain average of the five readings because equal weighting and a plain average are the same statement said twice.

How much does it change the answer?

The three answers span 0.008798 of volatility. The span is the whole distance between the plain price loss and the volatility loss. The vega weighted rule sits inside that span at 0.204420, or 0.002580 below the plain mean. Set that against the effects this subject treats at length and it is large. The spread is bigger than the gap between a twelve step lattice and the closed form. The spread is bigger than most numerical error a careful implementation carries. The choice of loss is not a small residual sitting under the real answer; on this set it is one of the largest single movements available.

In rupees at the middle strike the same spread is Rs 0.330352/-, the difference between what the Rs 100/- contract prices at under 0.198202 and what it prices at under 0.207000. On a model price of about Rs 10.45/- that is roughly three per cent of the value of the thing, produced entirely by a rule nobody wrote down.

The last row across is the one to take first. The root mean squared miss rises from Rs 0.471464/- to Rs 0.547387/- across the three answers, and that looks like a ranking. The rise is not a ranking. Root mean squared miss in rupees is the squared price error rule wearing a different label, so of course the squared price error rule wins on it. Scored by the vega weighted rule, the same three answers put 0.204420 first, at 0.001875 against 0.002044 and 0.001905. Scored by the volatility rule, 0.207000 wins, at 0.002080 against 0.002467 and 0.002113. Three different questions are on the table and each rule answers its own, so each of the three fits is the best answer available and each of the three is beaten by the other two.

How should the choice be made and recorded?

Nothing in the five prices has a preference, so the choice follows the use, and the use is the only thing that can sensibly decide it. Ask what the fitted model is for and the rule usually falls out. If the model exists to reproduce prices in rupees across a book, and a rupee of error is a rupee of error wherever it lands, then the plain squared price error is asking the matching question and its concentration on the high vega strikes is a feature rather than a distortion. If the model exists to be compared against volatility readings, or to be handed to somebody who will quote it as a volatility, then the volatility loss asks the matching question and its equal treatment of the five readings is the point.

Vega weighting sits between the two and is the honest choice when the targets are prices but the answer will be read as a volatility. Dividing by vega restates every gap in the units the answer will be quoted in, and that is why vega weighting lands close to the volatility loss. The rule is chosen by what the fitted number will be used for, and only afterwards by anything about the numbers themselves.

Recording it costs one line and there is no defensible reason to leave it out. The line names the rule, the weights, the units the residuals were measured in, and the set of targets they were measured over. If the weights were anything other than one, they are written out rather than described. A phrase like weighted by vega has at least two readings that give different answers, and a reader six months later cannot tell which was meant. The name of a rule is ambiguous in a way the weights never are, so the written form of the rule is the weights themselves.

What is worth carrying away from all of this?

Three sentences hold the whole subject. The first is that a fit has three inputs and only two of them are ever found: the targets and the model are read off something, and the rule that decides what closeness means is manufactured. The second is that the manufactured input is not small. On five invented prices the manufactured input moves the answer by 0.008798 of volatility, or Rs 0.330352/- at the middle strike, and it does so with every observable quantity held completely still.

The third is the one worth holding on to longest, and it is the one that generalises past this arithmetic. None of the three rules is wrong. Squared price error, vega weighting and squared volatility error each state a coherent view of what it means to be close, and each returns the best answer available under its own view. The failure is a report that states an answer without stating which view produced it. A fit that does not say which question it asked has not reported its own inputs, and the answer it prints is arithmetic without a statement of what was being asked.