Case 097Systematic research and dataCore
The Qadira value backtest uses book value from annual reports dated to the fiscal year-end, although reports appear about 60 days later. The backtest earns 11% a year. What bias is this, how does it inflate returns, and how do you fix it?
1The situation
A researcher presents the Qadira value strategy: each year on 1 April, rank listed companies by book value to price, buy the cheapest fifth and short the dearest fifth, and hold for a year. The book values come from a database that stamps each annual figure with the fiscal year-end, 31 March, although companies file their annual reports about 60 days later, around the end of May.
The backtest earns 11% a year over 15 years. As a check, the researcher reruns it with book value lagged by 30, 60, 90 and 120 days; the results are 9.4%, 7.9%, 7.6% and 7.5%. These are the invented results of this exercise.
2Your task
What bias is this, how does it inflate the 11%, and how do you fix it?
Quick check
What is wrong with trading on 1 April using book values dated 31 March?
Worked solution
Try it on paper, then open one step at a time.
30-second answerThe answer to give first
This is look-ahead bias: the backtest trades on book values about 60 days before they were published. Stocks whose annual numbers turn out well tend to rise in those 60 days, and the backtest buys them before the move. Lagging book value to its release date cuts the return from 11.0% to 7.9%, so about 3.1 points, 28% of the return, was never available. Fix it with point-in-time data stamped by filing date.
Step 1Why is a correct number still the wrong number to trade on?
Think of betting on a cricket match after reading the score in tomorrow's newspaper. The score is accurate; the problem is when you knew it. A backtest may use a number only from the day it was public, and Qadira's book values were stamped with the date they describe, 31 March, not the date they became known, about the end of May. That is {term('look-ahead bias', 'Using, in a backtest, any information that was not available at the moment the simulated trade was made.')}, and it hides inside databases that store figures by the period they belong to.
Step 2How does the bias inflate the return?
Between year-end and filing, the market learns a lot about the year: quarterly results, management comments, sector data. Stocks whose annual book value will come in strong tend to rise in that window. The biased backtest ranks on the final numbers and buys on day 1, so it holds those stocks through the rise that the real investor could only have seen afterwards. The short side works the same way in reverse. Every rebalance gets a 60-day head start, and it compounds across 15 years.
Step 3How much of the 11% is real?
Read the lag test. Return falls from 11.0% with no lag to 7.9% at 60 days, then barely moves to 7.6% at 90 and 7.5% at 120; the fall stops where the information became public. So about 3.1 points, 28% of the headline, came from the future. The flat part after 60 days is the reassuring sign: the strategy's remaining return does not depend on the exact lag, which is what a genuine effect looks like.
Step 4How do you fix it for every field, not just this one?
Store every data point with the date it became available, a point-in-time database, and let the backtest see only what was public at each simulated moment. Use each company's actual filing date where you have it; where you do not, apply a conservative uniform lag, such as 90 or 120 days, and check that results survive it. Two relatives of the same bias need the same care: restated figures, where the database has overwritten the first reported number with a later correction, and index membership, where today's constituents are used for past years. Say the limitation: a lag rule cannot recover information about late filers that the database never recorded.
Where candidates lose it
The common miss is naming survivorship bias because it is the bias everyone remembers. Survivorship is about which companies are in the sample; this is about when the data were known.
The second is fixing only book value. Every field, earnings, sales, analyst estimates and index membership, needs its own release date, and a backtest is only as clean as its dirtiest field.
What the interviewer asks next
- Why might the true bias be larger for small companies than large ones?
- The database only has fiscal year-end dates. What lag would you choose, and how would you defend it?
- How would you check a vendor's claim that its data are point-in-time?
Company names and figures are illustrative.
