Vintage Benchmark: Comparing a Fund to Its Own Cohort

A vintage benchmark is a set of figures for funds of one vintage, one strategy and one currency, assembled by a named provider and published with a sample size and an as-at date attached. A benchmark number stripped of its provider, its sample size and its as-at date is not a weaker figure but a different kind of object. The method is the thing to hold on to: what has to match before a comparison carries information, and what a quartile with nothing behind it actually is.

Begin with something visible from any doorstep. Two tea stalls, one street apart. One opened nine years ago and the owner has long since paid off the loan, bought the second cart and put a son through college. The other opened five months ago and is still working through the first sack of sugar. Somebody hands over a sheet saying stall one turned over more money last month than stall two. Nothing on that sheet is false. The sheet has compared a business in its ninth year with a business in its fifth month, and reported the difference as though it were about the two owners, so the sheet has said almost nothing. What has gone wrong is instinctively clear. The comparison did not control for age.

A fund's age is not printed on the front of the figure, so private funds have exactly that problem and worse. A report saying a fund stands at 1.50 times paid in looks like a fact about the manager. The 1.50 times is at least as much a fact about how far through its own life the fund happens to be, when the figure was struck, and how much of the value in it has actually been paid to anybody. The whole apparatus of vintage benchmarking exists to hold the age still so that the rest can be looked at.

What is a vintage benchmark actually made of?

A vintage benchmarkA set of figures for funds of one vintage, assembled by a provider from funds that reported. is not a fact about private markets in the way that a temperature is a fact about a room. A vintage benchmark is a set of numbers somebody collected. Four separate choices sit inside it before a single figure is read, and every one of the four moves the answer.

The first choice is which funds go in. A provider decides that this fund belongs in the set and that one does not, on a definition of vintage, of strategy and of geography that the provider wrote. The group that results is the cohortThe set of funds a benchmark treats as comparable, defined by the provider., and it is a definition rather than a discovery. The second choice is where the figures come from, and the answer is almost always the managers themselves. The third choice is the moment the whole set is struck at. The fourth is the arithmetic that turns a column of numbers into positions.

A reader who knows how the set was built can use it, and a reader who has only been handed the rank cannot, and will not be able to tell that they cannot. That last clause is the dangerous half. A rank arrives looking finished. The rank has no visible seams, it fits in a sentence, and it survives being repeated by somebody who never saw the underlying table. The four choices above are all invisible by the time the number reaches a third reader.

Every performance figure in the worked case that follows belongs to two invented funds run by one invented manager, Nilgiri Alternatives Advisors Private Limited, and each figure is stated with the fund it belongs to and the period it covers attached. A figure with neither of those attached is not a smaller fact but a different one.

What has to match before two funds can be compared at all?

Seven things, and a comparison missing any one of them is not a comparison. The seven work the way two students' marks work. Same paper, same day, same marking scheme, same duration, same syllabus, and marked by somebody using the same rules for both. Remove any single one of those and the two marks stop being about the two students. The comparison does not become slightly weaker. The comparison becomes a sentence about something else.

Here are the seven, and each one is a condition on both funds at once, never on one of them.

One, the vintage, on a stated convention. A fund can be dated by more than one rule, and two rules can put the same fund in two different years. Which convention does the dating is covered separately; what matters here is only that both funds were dated by the same rule. Two, the strategy, meaning what the fund set out to buy. Three, the geography. The conditions a business trades in are local to a place. Four, the currency the figures are struck in, before anybody applies a translation.

Five, the measure itself. A multiple has a denominator, and the same fund on the same day produces different multiples against what its holdings cost, against everything its investors paid in, and against everything they promised. Nilgiri Growth Partners Fund II, invented, stands at 1.80 times on the cost of its holdings, 1.50 times on everything paid in, and 1.44 times on everything committed, all three as at the end of its own Year 9 Quarter 2. The three figures are one fund, one day, three denominators. How each of the three is built is covered separately, and the only thing needed from it here is the rule: the denominator is named, every time. Six, the as-at dateThe moment every figure in a set is struck, the same moment for all of them.. The as-at date carries more weight than the rest and gets its own section below. Seven, whether the figure is gross or netBefore or after fees and carried interest, stated with the figure., before or after fees and carried interest have come out of it.

The reader cannot tell from the number which condition is missing or which way it pushed, so a missing condition is not a small problem. Where two funds are known to differ only in currency, the effect of a translation can at least be thought about. A rank handed over with nothing else says nothing about whether the difference is between two managers, between two years, between two measurement bases, or all three at once pulling in different directions.

Why does the as-at date carry more weight than the rest?

Because it is the condition most likely to be quietly untrue, and the only one that can be untrue inside a single published table.

Here is the mechanism, stated plainly and then stopped. Private funds do not report on a shared timetable. Each one strikes its own figures on its own cycle and sends them out when its own process finishes, and those processes finish at different times. A provider assembling a set therefore receives figures of different ages, and unless the provider forces every fund in the set to one common as-at date, some rows are more recent than others. The effect on the number is precise: part of any gap between two rows is the gap between the two dates the rows were struck at, and none of that part is visible in the rows. A stale row does not make the set useless and does not make the provider careless. A difference then has more than one possible source, and the table cannot say which.

The same thing happens in a household. Two neighbours are asked what their monthly grocery bill is. One gives last month's figure and the other gives the figure from the month of a wedding in the house. Both are honest, both are their own real numbers, and the difference between them is partly about the two households and partly about the two months. Nobody is lying and the comparison is still carrying something it did not declare.

A figure and the moment it was struck are one object, and pulling them apart makes the figure unreadable rather than merely less precise. A performance figure is therefore written with its fund and its period attached in the same sentence, without a shorthand on second mention. Written that way, the number says what it means.

How much does a fund's own age move a fund's own figures?

The cleanest comparison available is a fund against itself. Six of the seven conditions are met automatically and the seventh can be held still. Same vintage, same strategy, same geography, same currency, same measure, same fee basis. Only the as-at date is allowed to move. Whatever changes between the two readings is therefore about age and about what happened in between, and about nothing else.

Nilgiri Growth Partners Fund II, invented, read 0.95 times paid in and minus 1.8 per cent net at its own Year 5 year end. The same fund read 1.50 times paid in and plus 8.3 per cent net at its record date, the end of its own Year 9 Quarter 2. The same manager, the same nine holdings and the same set of documents sit behind both readings. Between the two sit 3.50 years, being fourteen quarters, and five realisation events, in which three holdings became cash in full, one left the portfolio producing no cash at all, and 40 per cent of one more became cash.

The headline figures hide the more important change. Look at what was underneath each of those two readings. At Nilgiri Growth Partners Fund II's Year 5 year end the fund had paid in Rs 4,55,50,00,000 and had distributed nothing whatsoever to its investors. Not one rupee. So the entire 0.95 times was a carrying value: an estimate of what the unsold holdings were worth, divided by what had been called. At its record date the same fund had paid in Rs 4,80,00,00,000 and had distributed Rs 4,38,00,00,000, so 0.91 of the 1.50 times is cash that investors have actually received and 0.59 of it is still an estimate. Both fractions use the same denominator, being the Rs 4,80,00,00,000 paid in.

One reading was made entirely of estimates and the other is made mostly of cash, so the two differ in kind and not only in size. The denominator itself also moved: Rs 24,50,00,000 more was called between the two dates, so even the bottom of the fraction is not the same bottom. A comparison of a fund with itself at two ages is the only comparison in this guide that holds anything still, and even it is comparing two different mixtures.

Exactly two points of that fund's path are drawn here and no others: its Year 5 year end and its record date. The year by year shape those points sit on is covered separately.

What can this manager's two funds be compared on, and what can they not?

Nilgiri Alternatives Advisors Private Limited runs two growth funds, and the two figures look like they belong in the same column, so the temptation to put them side by side is enormous. The two figures do not belong in one column, and working out precisely why matters more than the comparison itself ever would.

Nilgiri Growth Partners Fund I, invented, wound up at the end of its own Year 10 having returned 1.80 times paid in, net to its investors, with every one of its seven holdings realised and every rupee of that figure received as cash. Nilgiri Growth Partners Fund II, invented, stands at 1.50 times paid in as at the end of its own Year 9 Quarter 2, 8.50 years into a ten-year term with six quarters left to run.

Four of the seven conditions fail at once between those two sentences, and it is worth naming each failure rather than gesturing at them.

The vintage fails. Nilgiri Growth Partners Fund I's final close is four years before Nilgiri Growth Partners Fund II's. Every date is therefore written with its fund in front of it: Fund I's Year 10 Quarter 4 and Fund II's Year 6 Quarter 4 are the same afternoon. Under no convention do two funds four years apart sit in one cohort. The as-at date fails, and not by a quarter: one figure is struck at the end of a completed fund's life and the other is struck partway through a running one. The measure fails in a way that is easy to miss. Both 1.80 times and 1.50 times are multiples on paid in and still are not the same quantity: one counts money received and the other counts money received plus an estimate. And the fee basis fails. Nilgiri Growth Partners Fund I's 1.80 times is after carried interest of Rs 48,00,00,000. Nilgiri Growth Partners Fund II's 1.50 times has no carried interest in it at all, for the plain reason that none has ever been paid to its manager.

Now the part that makes this the clearest illustration of all. Nilgiri Growth Partners Fund II's total value at its record date is Rs 7,20,00,00,000, being Rs 4,38,00,00,000 of cash already distributed to investors plus Rs 2,82,00,00,000 of carrying value on holdings still held. Against the Rs 4,80,00,00,000 paid in, that is 1.5000 times, of which 0.9125 is cash and 0.5875 is the carrying value, and those two add to 1.5000 exactly. Take the carrying value as a share of the total value and Rs 2,82,00,00,000 divided by Rs 7,20,00,00,000 is 39.2 per cent, so nearly two fifths of what this invented fund is said to be worth at its record date is an estimate rather than a transaction. The same number arrives from the other direction: 0.5875 divided by 1.5000, both of them multiples on the same Rs 4,80,00,00,000 paid in, is also 39.2 per cent. The cash share is Rs 4,38,00,00,000 divided by Rs 7,20,00,00,000, being 60.8 per cent, and the two shares add to 100.0.

Be careful with the count. Three counts sit in this fund's record and only one of them is five. Four of the nine holdings are gone from the portfolio entirely. Five are still held, being holdings 4, 6, 7, 8 and 9, and it is those five that carry the whole Rs 2,82,00,00,000. There have also been five realisation events across five holdings, and that is a different five. Holding 9 sits on both sides: 40 per cent of it was converted to cash in Nilgiri Growth Partners Fund II's Year 8 Quarter 3 and the remaining 60 per cent is still held and still carried at an estimate. Not one rupee of the Rs 2,82,00,00,000 has been paid by anybody to anybody.

Think of two flats in the same building. One has been sold and the money is in the bank. The other has a valuer's letter saying what it should fetch. Both appear on a statement of the household's worth, and only one of them has been tested against a person willing to hand over money. Nobody is claiming the valuer is wrong. The point is that the two entries are different kinds of thing, and a total that adds them together has quietly mixed a fact with an estimate.

So what can these two funds be compared on? The things that are genuinely the same. Both were raised by the same manager on the same economic terms, both ran ten-year terms, both drew 96.0 per cent of what was committed to them, and both bought unlisted companies. The comparisons that hold are about structure and about process. One outcome is finished and one is not, so the outcomes cannot be compared.

What is a quartile, and what does it need behind it to be a measurement?

A quartileA rank within a set, meaningful only when the set and its date are stated. is a position within an ordered set, arrived at by lining the members of the set up on one measure and cutting the line into four. The arithmetic is covered separately. A quartile needs two things behind it before the position means anything at all: the set it was cut from, and the moment that set was struck.

A rank in a class test works the same way. Fourth of forty means something that can be acted on. Fourth, with nobody willing to say fourth of how many, out of which class, sat on which day, is a word that sounds like a measurement and behaves like an opinion.

The fund's own figure and its rank are two different objects, and only one of them belongs to the fund. Nilgiri Growth Partners Fund II's 1.50 times paid in as at the end of its own Year 9 Quarter 2 is a property of that fund, computed from its own record, and it does not change when somebody assembles a table. A rank is a property of a set. Move the set and the rank moves with it. The fund carries on holding exactly what it held before.

Where must a benchmark number come from, and what has to travel with it?

A benchmark figure has to come from a named provider, for a stated vintage dated on a stated convention, for a stated strategy and geography, with a count of the funds in the set and the as-at date every figure in it was struck at, and a statement of whether the figures are before or after fees and carried interest. Six fields, travelling with the number. Strip any of them off and what is left is a number that cannot be checked by the person who receives it.

None of the six can be stated from two invented funds run by one invented manager with no cohort behind them, and inventing a cohort around them would produce something far worse than a gap. A reader remembers a shape. A plausible table of quartile boundaries leaves those boundaries surviving in the reader's head six months later, long after the label saying they were illustrative has fallen off. A plausible wrong number will be remembered and cannot be checked, so a plausible wrong number is the single worst thing a teaching text can hand somebody.

There is a real place to get these numbers. Providers assemble and publish vintage benchmark sets commercially, and their published methodology documents state how they define a vintage, what they include and how they treat funds that stop reporting. The methodology is read before the table. In India, the Indian Venture and Alternate Capital Association at ivca.in is an industry body publishing material on private capital.

What does a count of funds in a set say, and what does it leave out?

A count gives the size of the set. The size is genuinely useful and it is the whole of what the count says.

Two mechanisms sit behind that sentence and each does something specific to the number. The first is that benchmark figures are self-reportedSupplied by the manager rather than collected independently.. A provider does not have a right of access to a private fund's books. The provider asks, managers supply, and the set is therefore assembled from what was sent in. The effect on the figure is exact: the set describes the funds whose managers chose to be in it, and says nothing whatsoever about funds whose managers did not.

The second is survivorshipThe distortion produced when funds that stopped reporting drop out of a set.. A fund that stops sending figures is not marked absent in the published table. The fund simply is not in the table, and the set is recomputed on the funds that remain. The effect on the number is also exact: the boundaries between one part of the set and the next are drawn through a different collection of funds than the collection they were drawn through the last time, and nothing in the table records which funds left or when.

Neither of those is an accusation of bad faith and neither means a set should be ignored. Self-reporting and survivorship are two facts about how a set is assembled, and they change what a count means. A count of the funds in a set is a count of the funds in the set.

The same shape appears in a shopkeepers' association that publishes average takings for its members. Shops that closed last year are not members any more, so they are not in the average, and the average is computed on the shops that are still open and still filling in the form. Nobody has done anything dishonest. The number is still an average of the shops that stayed.

What can a vintage benchmark say about private funds against public markets?

Nothing, in either direction. A vintage benchmark sets a fund against funds, and a public index is not one of them.

The claim that one of them does better than the other cannot be established by showing a figure from one invented fund, and there is no evidence behind this worked case to state it. Pick a different fund, a different period or a different measure, and a figure that appeared to settle it would settle it the other way. The two invented funds here are described against their own records and against nothing else. How a private fund's cash flows are set against an index at all is a method covered separately.

How would somebody actually read a benchmark line handed to them on a Tuesday?

Reading is the practical end of the method. Far more people read a benchmark line than ever assemble one, and the reading takes about four minutes if it is done as a sequence rather than as an interpretation.

Picture the analyst inside an institutional investor's private markets team. A line arrives inside a manager's presentation, or inside a report from a consultant, and it says something about where a fund sits in its vintage set. The analyst's job at that moment is not to form a view about the fund. The job is to work out whether the line can be reproduced, and to write down what it would take to reproduce it. The output of this reading is a list of questions and a note of which ones the line could not answer, and it is never a verdict.

The sequence has five rungs and there is a stop at the first one that cannot be answered from what was handed over. Which convention dated the vintage. How many funds are in the set. Who supplied the figures. As at what date they were struck. Whether they are before or after fees and carried interest. If rung two cannot be answered, the reading stops at rung two, and what goes in the file is that the line arrived without a sample and could not be reproduced. Six months on, that note is the only thing that will stop the same unreproducible line being quoted as though it had been checked, so the note is worth far more later than a remembered rank.

The same discipline works on a fund's own reported figures, and here the questions have real answers. Which multiple is this, and on which denominator: Nilgiri Growth Partners Fund II, invented, is 1.80 times on the cost of its holdings and 1.50 times on everything paid in, as at the end of its own Year 9 Quarter 2, and a report quoting one of those without saying which has not said much. How much of the stated value has actually been received: Rs 4,38,00,00,000 of the Rs 7,20,00,00,000 total value, being 60.8 per cent, with the remaining 39.2 per cent an estimate on five positions nobody has bought. How far through its own life is the fund: 8.50 years of ten, with six quarters left. All three questions are answerable from that fund's own record, and answering them takes the reader further than any rank would.

A position in a set belongs to the set and moves whenever the set moves, so a position cannot carry a decision. What a number can support is a much narrower question than what it is usually asked to support.