Portfolio Optimisation: What the Optimiser Actually Solves

A portfolio optimiser solves for the set of weights that makes portfolio variance as small as it can be for a stated expected return, subject to whatever conditions the mandate imposes. An optimiser is handed expected returns, volatilities and correlations, and it hands back weights. Every one of those inputs is an assumption written down by the holder, so the answer reports what those assumptions imply and nothing more.

A set of weights that arrives out of a machine carries two decimal places and nobody in the room wrote it, so it reads like a measurement of the world in the way a thermometer reading does. The arithmetic behind it is fine. Move one assumed number by a point, once, and watch the answer shift.

A portfolio optimiserA calculation that searches the possible sets of weights and returns the one scoring best on a stated measure. runs below on the Anantara Multi-Asset Portfolio, an invented discretionary mandate of Rs 500 crore run by Faiz Ahmad Ansari for an invented charitable endowment whose investment committee is chaired by Rukmini Deshpande. Its policy mix is equity 60.0 per cent at Rs 300 crore, fixed income 30.0 per cent at Rs 150 crore and cash 10.0 per cent at Rs 50 crore. Every figure below is one of that mandate's own stated assumptions, chosen by the holder rather than read off a market.

What is the optimiser actually being asked to solve?

The problem has three parts and only three. The quantities that may be changed are the decision variablesThe quantities a calculation is permitted to move while it searches. Everything else in the problem is fixed for it., here the three class weights: how much equity, how much fixed income, how much cash. The thing being made as small as possible is the objective functionThe single number a calculation is trying to make as large or as small as it can., here the portfolio variance. The requirement that must hold true throughout is an expected return of 10.05 per cent. The policy mix itself expects exactly that figure.

Three unknowns, one quantity to minimise, one equation that must hold. The problem is small and entirely mechanical, and everything difficult about portfolio optimisation sits in the numbers fed into it rather than in the solving. A student who can compute a standard deviation can check every step of it by hand.

Why variance rather than something else? Because a portfolio's expected return is the weighted average of its parts and its variance is not, and that asymmetry is the only reason a combination can be worth more than the sum of its pieces. Harry Markowitz set that out in Portfolio Selection in 1952. The mean-variance argument is worked through under risk and return; what matters for an optimiser is that the covariances carry information no list of individual volatilities contains.

How many numbers does the problem need before it can run?

Count them. The count is the lesson. For three classes the optimiser needs three expected returns, three volatilities and three pairwise correlations, nine numbers in all. Not nine thousand observations, not a data feed, not a model of the world. Nine numbers, and it will not start without all nine.

Here is what the Anantara mandate wrote down. Equity is assumed to return 12.0 per cent at a volatility of 18.0, fixed income 7.5 per cent at 5.0, and cash 6.0 per cent at 0.5. Equity and fixed income are assumed to move together with a correlation of 0.20. Cash is assumed uncorrelated with both, two more correlations of 0.00. The input set is complete at nine numbers.

All nine are assumptions the holder chose, so the weights that come out restate those nine numbers and report nothing about any market. The optimiser finds out very fast what a set of assumptions implies, and it does nothing else. The weights contain no information the nine numbers carried in, so nine numbers that could not be defended in a meeting produce weights that cannot be defended either.

The expected return the policy mix already implies is the yardstick everything else is measured against. Multiply each weight by its assumed return and add: 0.60 times 12.0 is 7.20, 0.30 times 7.5 is 2.25, and 0.10 times 6.0 is 0.60. The three contributions sum to 10.05 per cent. A portfolio's expected return really is the weighted average of its parts, so the arithmetic is simple. The volatility is not a weighted average of anything, and that is where the interesting behaviour lives.

The variance is worth writing out once. Square each weight, multiply by the square of that class volatility, and then add a cross term for every pair that is correlated. On the policy mix those four terms sum to 125.3725, and the square root of that is 11.20 per cent. Not one of the four terms is a weighted average of anything, and a combination can carry less risk than its pieces suggest for precisely that reason.

Set that 11.20 per cent against the weighted average of the three volatilities, 12.35 per cent. The parts would have given 12.35 per cent had they all moved as one. The 1.15 point difference is the whole of the diversification, and it exists only because the correlation was assumed at 0.20 rather than at 1.00. Change that one assumption to 1.00 and the difference vanishes entirely.

What does the answer look like when nothing is constrained?

Give the optimiser only the requirement that the three weights add up to the whole portfolio, and nothing else. On these nine assumptions the lowest variance way of reaching 10.05 per cent is equity at 38.24 per cent, fixed income at 117.02 per cent and cash at minus 55.27 per cent, with a volatility of 9.89 per cent. Read the middle number again: it holds more fixed income than there is portfolio, paid for with a negative amount of cash.

Nothing has gone wrong. The problem said the three weights must add to one and the expected return must be 10.05 per cent, and that answer satisfies both at a lower variance than anything else does. An unconstrained answer is not a wrong answer; it is the correct answer to a question nobody in the room meant to ask. The fix lives in the problem statement, never in the solver.

The everyday version is a household planning a wedding. Tell somebody the only rule is that the total must come to Rs 8 lakh and they can hand back a plan with minus Rs 2 lakh of catering, funded by an extra Rs 2 lakh of hall hire. The rule that food cannot cost a negative amount was never written down, so the plan adds up and is useless.

What is a portfolio constraint, and what does adding one cost?

A portfolio constraintA term written into a mandate that any answer must satisfy. is a condition the answer has to satisfy before it counts as an answer at all: the weights must sum to one, no weight may fall below nought, a class must sit between two stated bounds, no single holding may exceed a stated share. Each one narrows the set of weight combinations the optimiser may consider, and none of them changes how it scores the combinations that remain.

Two facts about constraints cover most of what anybody needs. A constraint removes options and does nothing else, so it can only make the objective worse or leave it exactly where it was, never better. And a constraint that never binds costs nothing whatsoever, so the useful question is never whether a constraint exists but whether it is a binding constraintA condition that is actually stopping the answer from moving where it would otherwise go..

Think of a queue at a ticket counter. A rule saying nobody may buy more than fifty tickets changes nothing on a day when the largest order is four, and it is the only thing that matters on the day a tour operator arrives wanting eighty. The rule did not change. On one day somebody was pressing against the rule and on the other nobody was.

Watch the cost arrive on this mandate. With only the adding up rule, the volatility at a 10.05 per cent expected return is 9.89 per cent. Add the rule that no weight may be negative and the answer moves to 56.67 per cent equity and 43.33 per cent fixed income at 10.84 per cent. The no negative rule cost 0.95 points. Now add the mandate's equity band of 50 to 70 per cent: the answer already sits inside it, so nothing moves and the band costs nothing. Add instead a requirement that cash be held at 10.0 per cent and the answer is forced back to 60, 30 and 10, at 11.20 per cent, a further 0.35 points.

The third bar is the one to stare at. The equity band is a real, written, enforceable term of the mandate. The answer the optimiser wanted was already inside the band, so on these assumptions the band costs nothing. A slack constraint looks exactly like that, and moving one assumption turns the same band into the most important sentence in the document.

Which of this mandate's constraints touch the class level problem?

The Anantara mandate carries five stated conditions. The weights must sum to the whole portfolio. No weight may be negative. Equity must sit between 50 and 70 per cent. No single holding may exceed 5 per cent of the portfolio. And no unlisted holdings, with a minimum credit standing on the fixed income sleeve stated as a policy rather than as a rating symbol.

Hand that list to the three class problem and see which items it can even read. The problem has three variables in it, an equity weight, a fixed income weight and a cash weight, and no individual holdings anywhere. The problem contains nothing for the 5 per cent holding cap or the ban on unlisted holdings to constrain, so neither one enters the class level problem at all. The common assumption is that every constraint in a mandate binds on every decision taken under it, and that assumption is expensive.

The two absent conditions are not unimportant. Both govern the selection decision inside the equity sleeve, where the holdings actually live, and that decision is taken separately. Feeding the two conditions to a three variable class optimisation feeds it nothing, and reporting the answer as satisfying all five conditions is a claim nobody tested.

Is the policy mix the optimiser's answer?

Run the optimiser on this mandate's own nine assumptions, ask for the minimum variance portfolioAmong all the weight combinations that satisfy every stated condition, the one whose variance is smallest. that reaches an expected return of 10.05 per cent, and require only that the weights sum to the whole portfolio and that none of them is negative. The answer is 56.67 per cent equity, 43.33 per cent fixed income and cash at nought, at a volatility of 10.84 per cent.

An answer that breaks the mandate is not an answer, so check this one against the mandate first. Equity at 56.67 per cent sits inside the 50 to 70 per cent band, no weight is negative, and the three sum to the whole portfolio, so it is admissible. The mandate permits both this mix and the policy mix of 60, 30 and 10, and both reach exactly 10.05 per cent on the same assumptions.

The policy mix is admissible and it is not the lowest volatility way of reaching its own expected return: the same 10.05 per cent is available at 10.84 per cent volatility rather than 11.20, a difference of 0.35 points. The reason for the gap is mechanical. Reaching a 10.05 per cent target while carrying cash forces more equity in to make up the shortfall, and equity is the expensive part. Once the optimiser can choose freely it therefore has no use for a class assumed to return 6.0 per cent. Every rupee of the Rs 50 crore cash weight is paid for in equity risk somewhere else.

Neither mix is a recommendation. The two points on a plane show that the policy mix was chosen for reasons the optimiser was never told about, and that those reasons carry a measurable price on these assumptions.

How far does the answer move when one assumption moves by a point?

Hold eight of the nine numbers perfectly still. Do not touch a volatility, do not touch a correlation, do not touch the fixed income or cash returns. Move the equity expected return alone, by one point in either direction, and watch the equity weight. Before reading on, guess which way it goes when the assumption rises.

It falls. With cash at nought, the equity weight that reaches 10.05 per cent is 2.55 divided by the gap between the equity assumption and the fixed income assumption of 7.5. At an assumption of 12.0 that is 2.55 over 4.5, or 56.67 per cent. At 13.0 it is 2.55 over 5.5, or 46.36 per cent. At 11.0 it is 2.55 over 3.5, or 72.86 per cent. Nine numbers went in, one of them moved by a single point in each direction, and the answer moved across 26.5 points of the portfolio. On Rs 500 crore that is Rs 1,32,50,00,000/-.

Moving one input a little and watching the answer move a long way is what input sensitivityHow much the answer to a calculation moves when one of the numbers fed into it moves a little. means. Nobody in any room can say whether equity will return 12.0 per cent or 11.0 per cent over any period, and the two claims sound almost identical when spoken aloud. On these assumptions the two claims produce answers 16.19 points of the portfolio apart. On Rs 500 crore that is Rs 80,95,00,000/-. The precision of the output is not evidence about the precision of the inputs. The output is arithmetic doing exactly what it was asked, on numbers that were never that firm.

Look at where the curve leaves the pale band at each end. The curve crosses the 70 per cent ceiling at an assumption of about 11.14 per cent and crosses the 50 per cent floor at 12.60 per cent. Outside that window the two asset answer is not admissible at all. An assumption of 13.0 per cent produces 46.36 per cent equity, below the mandate floor just as surely as 72.86 per cent is above its ceiling. Both bounds of the band bind, at opposite ends of a two point move in a single input.

What happens when a constraint puts the target out of reach?

Take the equity assumption down to 11.0 per cent. Reaching 10.05 per cent now needs 72.86 per cent equity, and the mandate ceiling is 70, so there is no set of weights that satisfies both the return requirement and the mandate. The problem has an unreachable targetA requirement that no combination satisfying every other condition can meet., and the right thing for the machine to hand back is nothing at all, accompanied by a sentence saying why.

Can cash rescue it? No, and it is worth checking rather than assuming. Cash is assumed to return 6.0 per cent, below the fixed income assumption of 7.5 per cent, so swapping fixed income for cash lowers the expected return further. The best available expected return inside the mandate is therefore equity pinned at its ceiling with fixed income taking the rest: 0.70 times 11.0 is 7.70, 0.30 times 7.5 is 2.25, and those sum to 9.95 per cent. The target of 10.05 per cent misses by 0.10 points. On Rs 500 crore that is Rs 50,00,000/- of expected return the mandate as written cannot produce on that assumption.

The correct response is to change the target or to change the mandate, in the open, with somebody's name against the change, and a system that quietly relaxes a constraint in order to return an answer is doing the most dangerous thing a system can do. An answer always arrives, it always looks reasonable, and nothing in the output shows that the mandate was overridden to produce it. Rukmini Deshpande's committee can only argue with a decision it can see.

The floor end of the range behaves differently. At an assumption of 13.0 per cent the two asset answer of 46.36 per cent equity is below the mandate floor, but the target is still reachable: pin equity at 50.00 per cent, and the return equation then fixes fixed income at 36.67 per cent and cash at 13.33 per cent, at a volatility of 9.54 per cent. Cash comes back, not because anybody formed a view about liquidity, but because the equity floor forced a weight the return arithmetic then had to pay for. And 9.54 per cent against the 9.26 per cent the answer without the floor would have carried is the ladder rule holding once more: the floor cost 0.28 points.

What does the optimiser know, and what does it not know?

The optimiser knows nine numbers exactly, it knows every combination of three weights that satisfies the conditions it was handed, and it knows to as many decimal places as anybody cares to print which of those combinations carries the smallest variance. On its own terms it is perfect, and the arithmetic is not what is at issue.

The list of what it does not know is longer. Uncertainty was not one of the inputs, so the optimiser does not know that the nine numbers are uncertain. The optimiser was handed one number for the equity and fixed income correlation and not a description of how that number behaves, so it does not know that correlations tend to move in exactly the periods when a holder most needs them to hold still. Nor does it know that the endowment may need cash for a reason that never appeared in the problem, such as a commitment falling due. And it does not know that somebody has to live with the answer through a year in which it looks wrong.

The output is the beginning of a discussion and never the end of one, and the moment it is treated as the end, the nine assumptions stop being arguable. That is the damage. Nobody argues with a machine in a meeting; they argue with each other, and the assumptions were where the argument belonged.

How does a committee actually use one of these outputs?

Not by adopting it, and not by ignoring it either. An analyst preparing a paper for Rukmini Deshpande's committee reruns the optimiser across a stated range for each assumption, one at a time, and reports the range the answer moves across rather than the single answer. Alongside that goes a list of which conditions were binding and which were slack. A slack condition had no effect on the decision and a binding one decided it.

A lender does the same work under a different name when it stress tests a borrower's covenant headroom: the interesting output is never the base case number, it is which covenant bites first and how far the inputs have to move before it does. A household does it with a pen when it works out what happens to the monthly plan if one salary stops for three months, and again what matters is which line of the plan breaks first.

The exercise found out how little of the answer was determined by anything anybody actually knows, and reporting one set of weights hides exactly that. The reporting artefact that survives contact with a committee has three parts: the answer, the range it moves across when each input moves, and the binding list.

What is portfolio optimisation not?

Optimisation is not a forecast. The expected returns were handed to the solver and it did arithmetic on them. No market entered the calculation at any point, so optimisation is not a measurement of any market. The objective being minimised was chosen by somebody, and a different objective gives a different answer, so the output is not a ranking of what a holder should carry. And it is not a substitute for the conversation about the nine numbers. The judgement sits in that conversation.

Optimisation is, precisely, a fast and exact way of finding out what a set of assumptions implies, and an unusually clear way of finding out how firmly those assumptions are held. The second use is the one worth having. If a room cannot agree whether the equity assumption is 11.0 or 12.0, the optimiser has told everyone in about a second that the room does not agree about 16.19 points of the portfolio. No other part of the process surfaces that as quickly.