The Morlen momentum strategy was tuned over 200 combinations of look-back and holding period. The best has a backtest Sharpe ratio of 1.8 over five years. How much of that should you believe, and what test would you run before allocating?

Believe part of it: pure luck across 200 tries would typically produce a best Sharpe of about 1.2, so 1.8 probably contains a real effect but is biased well upwards. Even a single untuned test over five years would carry a 95% range of about 0.9 to 2.7. Before allocating, run the strategy on data it has never seen, check that neighbouring settings also work, and plan on a live Sharpe far below 1.8.

Ask 200 people to toss a coin ten times and the best will probably get eight or nine heads. Nobody concludes that person is skilled at tossing coins. Choosing the best of many backtests picks the luckiest result, so its Sharpe ratio overstates what the strategy will do next, a problem called selection biasThe upward bias in a result chosen as the best of many attempts, because the choice rewards luck as well as skill.. Over five years, a skill-free strategy's measured Sharpe has a standard error of about 1 over the square root of 5, 0.45. The best of 200 such draws typically lands about 2.75 standard errors up, a Sharpe near 1.23.

Some of it. If all 200 settings were independent and skill-free, the chance of the best reaching 1.8 is small, about 0.6%. That makes pure luck an unlikely explanation, but it does not make 1.8 the expected live Sharpe: the result is still the best of 200, and the backtest period, costs and universe were chosen by the same researcher. A defensible planning number starts from how far 1.8 sits above the 1.2 that luck alone delivers, about 0.6, and treats anything above that as unproven. The settings are also not independent, since neighbouring look-backs give similar results, which lowers the luck hurdle but also means the 200 tries tell you less than they seem to.

Three tests, each aimed at a different way the backtest could be flattering. First, hold-out data: run the chosen setting, unchanged, on a period or a market the researcher never touched. Second, look at the neighbourhood. A real effect shows a plateau of good results across nearby settings; a fitted accident shows a single spike surrounded by noise. Third, stress the costs: double the assumed cost and see what survives, because momentum trades often and crowds into the same names. Then allocate small, with a Sharpe expectation well below the backtest and a pre-agreed rule for when to stop.