Risk Management puzzles, solved step by step

Puzzles: 100
Traced to a firm: 17
Topics: 13
Hard: 30

Showing 1–10 of 10 · filtered from 100Clear filters

012A Monte Carlo VaR at 99% uses 10,000 simulated paths. How many scenarios sit in the tail that defines the VaR, and how many paths do you need to halve the standard error of the estimate?Statistics and estimationHardMSCISan Francisco · 2018
Try it first
How many paths halve the standard error of the VaR estimate?
Show the worked solution
100 scenarios define the tail, and you need 40,000 paths to halve the error. At 99%, 1% of 10,000 paths is 100 scenarios, and the VaR is read at the edge of that group. Monte Carlo error falls as one over the square root of the number of paths, so halving it means four times the paths. For a book with Rs 10 crore of daily volatility, the error is about Rs 37 lakh at 10,000 paths.
Why does a 99% VaR rest on so few scenarios?
Picture an exit poll that interviews 10,000 voters but reports only on the one voter in a hundred who picked a small party. Your estimate for that party rests on about 100 people, not 10,000. A 99% VaR is read at the boundary of the worst 1% of outcomes, so only about 100 of 10,000 paths carry information about where that boundary is. For a normal book with Rs 10 crore of daily volatility the true VaR is Rs 23.26 crore, and the estimate wobbles around it by about Rs 37 lakh from one run of 10,000 paths to the next.
The relationship
$\text{SE}(\widehat{\text{VaR}}) \approx \frac{1}{f(q)}\sqrt{\frac{p(1-p)}{n}}$
p the tail probability, 1%
n the number of paths
f(q) the height of the P&L density at the VaR point
What it says in wordsThe error of a quantile estimate shrinks with the square root of the paths and grows where the tail is thin.
For a book with Rs 10 crore of daily volatility, the standard error of a 99% Monte Carlo VaR is about Rs 37.3 lakh at 10,000 paths and Rs 18.7 lakh at 40,000, because the error falls with the square root of the number of paths.
Why does halving the error cost four times the paths?
Because the paths sit under a square root. To halve the error, the square root of the path count has to double, which means the path count itself must quadruple. That is why precision in Monte Carlo is expensive: going from about 1.6% error to 0.8% of the VaR costs four times the computing time. It also explains why desks use variance reductionSimulation techniques, such as antithetic paths or importance sampling, that give a more precise estimate from the same number of paths., especially importance sampling, which pushes more paths into the tail where the VaR is decided.
Say the limit plainly. More paths shrink sampling error only; they do nothing about model error. If the simulated distribution has the wrong tail, a million paths give a very precise estimate of the wrong number. And expected shortfall at 97.5% averages the worst 250 paths, so it is usually more stable than the 99% VaR read from a single boundary.
Where candidates lose it
The trap is answering 20,000 paths, assuming error falls in proportion to the path count. It falls with the square root, and the difference is a factor of two in cost that a quant interviewer will not let pass.
The other miss is thinking all 10,000 paths inform the VaR. Say the number 100 out loud; it shows you know why tail estimates are noisy.
What the interviewer asks next
How many paths would you need for the same precision on a 99.9% VaR?
How does importance sampling reduce the error without adding paths?
Why is a Monte Carlo VaR often less stable from day to day than a historical VaR on 500 days of data?
Asked at MSCI, Risk Management, San Francisco, 2018 (Wall Street Oasis): the next round was paper test, materials were similar to CFA level 1, and it mostly focused on monte carlo, dividend, and risk
023You estimate a correlation of 0.5 between two assets from 36 monthly observations. Roughly what is its standard error, and is a later estimate of 0.3 from the next 36 months evidence that the relationship has changed?Statistics and estimationHardQuant riskAsset manager risk
Try it first
Is the drop from 0.5 to 0.3 statistically meaningful?
Show the worked solution
The standard error is about 0.125, and 0.3 is not evidence of a change. A rough rule is one minus the correlation squared, over the square root of n: 0.75 over 6. Using Fisher's transformation, the 95% interval for 0.5 runs from about 0.21 to 0.71, and 0.3 sits inside it. Testing the difference between two 36 month estimates gives z of about 0.97, well below 1.96.
Why is a correlation from 36 months so imprecise?
Think of judging whether two friends like the same films after watching three dozen films together. They might agree on 20 by chance even if their tastes barely overlap. A correlation is estimated from how often two series move together, and with 36 points random coincidences move the estimate a lot: the standard error of 0.5 is about 0.125, a quarter of the estimate itself. The rough rule is one minus the correlation squared over the square root of the sample size, 0.75 over 6.
The relationship
$\text{SE}(\hat\rho) \approx \frac{1 - \rho^2}{\sqrt{n}} = \frac{0.75}{6} = 0.125 \qquad z = \tanh^{-1}(\hat\rho), \;\; \text{SE}(z) = \frac{1}{\sqrt{n - 3}}$
rho the estimated correlation, 0.5
n the number of observations, 36
tanh^{-1} Fisher's transformation, which makes the sampling error close to normal
What it says in wordsThe error of a correlation shrinks only with the square root of the sample size, and Fisher's transformation gives a fair interval around it.
The 95% interval for a correlation of 0.5 from 36 months runs from 0.21 to 0.71, and the interval for 0.3 from another 36 months runs from -0.03 to 0.57, so the two overlap heavily and the drop is within sampling noise.
How do you test whether the relationship changed?
Transform both estimates with Fisher's z, which turns 0.5 into 0.549 and 0.3 into 0.310. The difference, 0.240, divided by its standard error of the square root of 2 over 33, about 0.246, gives a z of 0.97: well short of the 1.96 needed for 95% confidence. A gap that size would appear by chance about 33% of the time even if nothing had changed. The honest statement is that the data cannot tell 0.5 from 0.3.
Then say why a risk manager cares. Hedge ratios, portfolio volatility and diversification benefits are all built on correlations like this one, and treating 0.5 as exact makes a model look more certain than its data allow. The practical responses are longer or higher-frequency data, shrinking noisy estimates towards a sensible prior, and stress testing the portfolio at correlations across the whole interval rather than at the point estimate.
Where candidates lose it
The trap is reading the drop from 0.5 to 0.3 as a regime change. It is a 40% fall in the number but well inside the noise that three years of monthly data carry.
The second miss is quoting a standard error without saying it is rough. The simple formula is an approximation; Fisher's transformation is what you use for an interval or a test.
What the interviewer asks next
How many monthly observations would you need to bring the standard error down to 0.05?
Would weekly data over the same three years help, and what could go wrong?
How would you stress test a portfolio whose risk depends on this correlation?
025A Kalman filter tracks a random walk with process variance 1 and measurement variance 4. What gain does it settle at, and which simple smoother is it then equivalent to?Statistics and estimationHardUBSLondon · 2022
Try it first
Where does the gain settle?
Show the worked solution
The gain settles at about 0.39, and the filter becomes an exponentially weighted moving average. In steady state the prior variance P solves P squared minus P minus 4 equals zero, so P is about 2.56. The gain is P over P plus 4, about 0.39. Each new estimate is then 0.39 times the new reading plus 0.61 times the old estimate, which is exactly an EWMA.
What is a Kalman filter doing, in one picture?
Think of estimating how many people are in a stadium from a noisy turnstile count that you update every few minutes. Your last estimate is useful but the crowd keeps changing, and each new count is useful but noisy. A Kalman filter blends the old estimate and the new reading, weighting each by how much you trust it; the weight on the new reading is the Kalman gainThe share of the gap between a new measurement and the prior estimate that the filter accepts as news.. Process variance of 1 says the true value drifts by about one unit each step; measurement variance of 4 says each reading is off by about two units.
The relationship
$P^- = P + Q, \quad K = \frac{P^-}{P^- + R}, \quad P = (1-K)P^- \;\Rightarrow\; (P^-)^2 - Q P^- - QR = 0$
Q the process variance, 1: how much the true value moves each step
R the measurement variance, 4: how noisy each reading is
P^- the variance of the estimate just before a reading arrives
K the Kalman gain
What it says in wordsIn steady state the uncertainty added by the drift each step exactly balances the uncertainty removed by each reading.
Starting from a vague prior the Kalman gain begins at 0.96, drops to 0.55 and 0.44, and settles at 0.390 by about the sixth update, after which the filter is an exponentially weighted moving average with weight 0.39 on each new reading.
Why does the gain settle rather than fall to zero?
Because the thing being tracked keeps moving. If the true value were fixed, every reading would add certainty and the gain would shrink towards zero, like a running average; with a random walk, each step adds one unit of variance back, so certainty stops improving at a balance point. Plugging in Q of 1 and R of 4, the prior variance solves P squared minus P minus 4, giving (1 plus the square root of 17) over 2, about 2.56, and a gain of 0.390. Only the ratio of R to Q matters: a noisier measurement lowers the gain, a faster-moving state raises it.
That is the link worth saying in a risk interview. An EWMA volatility or correlation estimate, the kind many risk systems use, is a Kalman filter in steady state with a particular noise ratio, whether or not anyone calls it that. The filter's advantage is that it chooses the weight from stated assumptions about noise and drift, and adapts it early on; the limit is that those assumptions, a linear model with normal noise, have to be right for the weight to be the best one.
Where candidates lose it
The trap is describing a Kalman filter in general terms and never producing a number. The interviewer wants to see you set up the variance recursion and solve the steady state.
The second miss is saying the gain goes to zero. That is true only for a constant state; for a random walk it settles at a positive value, and that is why the filter reduces to an EWMA.
What the interviewer asks next
What steady-state gain do you get with measurement variance 1 instead of 4?
What EWMA decay factor corresponds to this filter, and what is its half-life in updates?
How would you estimate Q and R from data?
Asked at UBS, Risk, London, 2022 (Wall Street Oasis): Explain what a Kalman filter is
037Forecaster A has a bias of 1 point and a forecast error standard deviation of 2. Forecaster B is unbiased with a standard deviation of 2.5. Using the mean squared error decomposition, which forecaster is better?Statistics and estimationCoreBlackRockNew York · 2026
Try it first
Which forecaster has the lower mean squared error?
Show the worked solution
Forecaster A, with a mean squared error of 5 against 6.25. Mean squared error splits into bias squared plus variance. A pays 1 squared for its bias and 2 squared for its spread, 5 in all. B pays nothing for bias but 2.5 squared, 6.25, for its spread. A's typical error, the square root, is 2.24 against 2.50.
How can a biased forecaster beat an unbiased one?
Two archers. One groups every arrow tightly but slightly left of centre; the other is centred on average but scatters arrows all over the target. Ask which one lands closer to the bullseye on a typical shot, and the tight grouping wins. Mean squared error charges for two things, how far off you are on average and how much you scatter, and a small, steady bias can cost far less than a large scatter. Being unbiased only removes the first charge.
The relationship
$\text{MSE} = \text{bias}^2 + \text{variance}: \quad A = 1^2 + 2^2 = 5, \qquad B = 0^2 + 2.5^2 = 6.25$
bias the average forecast error, forecast minus actual
variance the spread of the errors around their own average, the standard deviation squared
What it says in wordsSquared error on average equals the squared average error plus the spread of errors around it.
Forecaster A's mean squared error is 1 of bias squared plus 4 of variance, 5 in total, while unbiased Forecaster B carries 6.25 of pure variance, so A's small bias buys a larger cut in variance and gives the lower error.
When would your answer flip, and what would you do with A?
Solve for the tie: A matches B when bias squared plus 4 equals 6.25, so a bias of 1.50. Below that, A wins. More useful still, a bias that is stable can be measured and subtracted: correct A by one point and its MSE falls to 4, better than either original. That is the practical lesson for a risk team: a model that is consistently off in one direction is fixable, while a noisy model is not. The trade-off is also why risk teams use shrinkagePulling a noisy estimate towards a simpler, steadier target, accepting a little bias in return for much lower variance. on covariance matrices built from short histories.
The limit: MSE punishes large errors heavily because it squares them, and it treats over-forecasts and under-forecasts alike. A risk manager forecasting losses may care more about under-forecasting than over-forecasting, in which case a symmetric score is the wrong yardstick and the ranking could change.
Where candidates lose it
Candidates pick B on reflex because unbiased sounds like correct. The question is built to see whether you know that MSE has two parts and can do the two-line arithmetic.
The quieter miss is stopping at 5 against 6.25. Add that A's bias can be corrected, taking its MSE to 4, and you have turned a statistics answer into a model-risk judgement.
What the interviewer asks next
What bias would make the two forecasters exactly equal?
Why might a regulator prefer the unbiased forecaster even with a higher MSE?
How would you test whether A's bias is stable over time?
Asked at BlackRock, Restructuring, New York, 2026 (Wall Street Oasis): Which equities have duration ? multiple stocks vs value stocks MSE Forecasting equation
048You estimate a desk's daily P&L variance from five observations, once dividing the sum of squared deviations by 5 and once by 4. Which estimator is unbiased, which is consistent, and how large is the bias?Statistics and estimationCoreUBSZurich · 2021
Try it first
Which statement is true?
Show the worked solution
Dividing by 4 is unbiased; both estimators are consistent; dividing by 5 is low by one fifth. The sample mean is estimated from the same five points, which uses up one degree of freedom, so the divide-by-n estimator averages (n - 1)/n of the true variance, 80% here. If the true variance is 4, it centres on 3.2, a bias of -0.8. As n grows that factor tends to 1, so both estimators converge on the truth.
Why does dividing by n come out too low?
Measure how spread out five friends' heights are by comparing each to the group's own average, and you will understate the spread, because that average was pulled towards those five people. Deviations measured from the sample mean are smaller on average than deviations from the true mean, so their sum of squares understates the spread by exactly one observation's worth. Dividing by n minus 1, the degrees of freedomThe number of independent pieces of information left after estimating something from the same data; estimating the mean uses up one., corrects it exactly.
With five observations and a true variance of 4, the divide-by-4 estimator is centred on 4 while the divide-by-5 estimator is centred on 3.2, 80% of the truth, yet the biased version is narrower and has a lower mean squared error, 5.76 against 8.00.
What is the difference between unbiased and consistent?
Unbiased is about the average over many repeated samples of the same size; consistent is about what happens to one estimate as the sample grows. Divide-by-5 fails the first: repeat the five-day exercise many times and the estimates average 3.2, not 4. It passes the second: with 250 days the bias is only -0.016, and it shrinks to zero with more data. An estimator can be unbiased but inconsistent too, such as using only the first observation to estimate a mean: right on average, never improving.
The relationship
$E\!\left[\tfrac{1}{n}\textstyle\sum (x_i - \bar{x})^2\right] = \frac{n-1}{n}\,\sigma^2 = 0.8 \times 4 = 3.2$
n the number of observations, 5
\bar{x} the sample mean, estimated from the same five points
\sigma^2 the true variance, 4 in the illustration
What it says in wordsDividing by n recovers only (n minus 1) over n of the true variance on average.
Now the twist a model validator should add. For normal data the unbiased estimator has variance 8.00 here, while the divide-by-5 version has 5.12 plus a squared bias of 0.64, a mean squared error of 5.76. The biased estimator is closer to the truth on a typical sample. Which you prefer depends on the use: unbiasedness matters when estimates are averaged across many desks; a smaller typical error matters for a single desk's limit. With five data points, neither is reliable, and that is the more important thing to say.
Where candidates lose it
The usual slip is to treat unbiased and consistent as the same thing, and so to call the divide-by-5 estimator inconsistent. The interviewer asked both words together precisely to hear you separate them.
The second miss is answering from memory without the reason. One sentence on the sample mean using up a degree of freedom shows you know why n minus 1 exists, not just that it does.
What the interviewer asks next
Give an example of an estimator that is unbiased but not consistent.
Is the sample standard deviation, the square root of the unbiased variance, itself unbiased?
With 250 days of P&L, does the choice between n and n - 1 matter for VaR?
Asked at UBS, Risk Management, Zurich, 2021 (Wall Street Oasis): And several other questions on econometrics - what is an unbiased estimator vs consistent estimator?
050An EWMA volatility model with a decay factor of 0.94 had yesterday's daily volatility estimate at 1.2%. Today's return is minus 3%. What is the updated volatility estimate?Statistics and estimationCoreBank market riskQuant risk
Try it first
Roughly where does the new daily volatility land?
Show the worked solution
About 1.38%. EWMA updates the variance, not the volatility: new variance is 0.94 x 1.2 squared plus 0.06 x 3 squared, which is 1.3536 plus 0.54, or 1.8936. The square root is 1.376%. The shock has only a 6% weight but enters squared, so it supplies 29% of the new variance.
Why blend variances rather than volatilities?
A household tracking how much its grocery bill swings would be misled if it averaged the size of the swings in rupees but ignored that one big swing matters far more than several small ones. Variance is the average squared move, so the model updates squared returns, and a move 2.5 times normal size counts 6.25 times as much. An EWMAExponentially weighted moving average: each day the estimate keeps a fixed share of yesterday and adds the rest from today, so older days fade geometrically. of variances is what makes one large shock move the estimate quickly. Blending the volatilities directly gives 1.31%, understating the jump.
Today's minus 3% return carries only a 6% weight in the EWMA update, but because it enters squared it supplies 28.5% of the new variance of 1.8936, lifting daily volatility from 1.20% to 1.38%.
The relationship
$\sigma_t^2 = \lambda\,\sigma_{t-1}^2 + (1-\lambda)\,r_t^2 = 0.94(1.44) + 0.06(9) = 1.8936, \quad \sigma_t = 1.376\%$
\lambda the decay factor, 0.94
\sigma_{t-1} yesterday's volatility estimate, 1.2%
r_t today's return, minus 3%
What it says in wordsKeep 94% of yesterday's variance, add 6% of today's squared return, then take the square root.
What happens next, and what are the model's limits?
If tomorrow is flat, the estimate decays to the square root of 0.94 x 1.8936, about 1.33%. Each day of calm keeps 94% of the variance, so a shock's influence halves in about 11 trading days: ln 0.5 over ln 0.94. That is the design choice behind 0.94: fast enough to react to a new regime within days, slow enough not to swing on every move.
The limits: EWMA has no pull towards a long-run average, so after a calm spell it can drift very low and understate risk just before volatility returns, which GARCH-type models address with a mean-reversion term. It also treats up and down moves alike, while falling markets often raise volatility more than rising ones. And the choice of 0.94 is a convention for daily data, not a law, so the estimate should be backtested against realised moves.
Where candidates lose it
The usual slip is to blend the two volatilities directly, 0.94 x 1.2 plus 0.06 x 3, and answer 1.31%. It looks like the formula but applies it to the wrong quantity and understates the effect of the shock.
The second is forgetting the square root at the end and quoting 1.89% as a volatility. Say the units at each step: variance in, variance out, then volatility.
What the interviewer asks next
What would the estimate be with lambda of 0.97 instead?
How many quiet days until the estimate is back below 1.25%?
Why might a risk manager prefer GARCH to EWMA for a ten-day VaR?
062Your prior estimate of a hidden fair price is 10 with variance 4. A noisy measurement comes in at 12 with measurement variance 1. After one Kalman filter update, what is your new estimate and its variance?Statistics and estimationCoreUBSLondon · 2022
Try it first
Where does the new estimate land?
Show the worked solution
The new estimate is 11.6 with variance 0.8. The Kalman gain is the prior variance over the total, 4 over 5, or 0.8. The estimate moves 80% of the way from 10 towards 12, landing at 11.6. The variance becomes (1 - 0.8) x 4 = 0.8, smaller than either the prior's 4 or the measurement's 1.
How does the filter decide how far to move?
Two friends guess your commute time. One has ridden with you a hundred times, the other once. You would average their guesses, but lean heavily on the first. A Kalman filterA method that updates an estimate of something you cannot see directly each time a noisy measurement arrives, weighting old estimate and new data by how precise each is. does exactly that: it weights the prior and the measurement by their precision, one over variance. Precision 0.25 against 1 gives the measurement 80% of the weight.
The prior centred at 10 is wide, with variance 4, and the measurement at 12 is narrow, with variance 1. The update lands at 11.6, four fifths of the way to the measurement, and its variance of 0.8 makes it narrower than either source.
Why is the new variance smaller than both inputs?
Because two independent pieces of evidence together know more than either alone. Precisions add: 1 over 4 plus 1 over 1 is 1.25, and one over 1.25 is a variance of 0.8. That is the part candidates skip. The filter does not just move the estimate; it becomes more confident with every measurement, until new data carry little weight and the estimate settles.
The relationship
$K = \frac{P}{P+R} = 0.8 \qquad \hat{x} = 10 + 0.8(12-10) = 11.6 \qquad P' = (1-K)P = 0.8$
P the prior variance, 4
R the measurement variance, 1
K the Kalman gain, the weight on the new measurement
\hat{x} the updated estimate
What it says in wordsMove from the prior towards the measurement by the gain, and shrink the variance by the same share.
Say the limitation: this single step assumes both errors are normal and the hidden price did not move between the prior and the measurement. A full filter adds a prediction step that lets the price drift and widens the variance before each update. Risk teams use the idea to track hidden quantities such as a hedge ratio that changes over time.
Where candidates lose it
The trap is averaging the two numbers and answering 11. That treats the measurement and the prior as equally trustworthy, which the variances say they are not.
The second trap is getting 11.6 and then saying the variance is somewhere between 1 and 4. Combining evidence always reduces uncertainty, so the new variance must be below both: 0.8.
What the interviewer asks next
A second measurement of 11 arrives with variance 1. What is the estimate now?
What happens to the gain as the number of measurements grows?
How would you use a Kalman filter to estimate a time-varying hedge ratio?
Asked at UBS, Risk, London, 2022 (Wall Street Oasis): Explain what kalman filter is.
073A rating grade carries a predicted default rate of 1%. It has 500 obligors, and 9 defaulted last year. At 95% confidence, is the grade miscalibrated?Statistics and estimationHardRating agencyModel validation
Try it first
Nine defaults against five expected. What does an exact one-sided test say?
Show the worked solution
No, not at 95% confidence, though it is close. With 500 obligors at 1%, you expect 5 defaults. The exact binomial chance of 9 or more is about 6.7%, above the 5% cut-off; the smallest count that would reject is 10. A normal approximation without a continuity correction gives z = 1.80 and wrongly rejects. One year of data rarely proves a grade wrong.
How surprising are nine defaults if the 1% is right?
If a coin lands heads 7 times in 10, you would not call it biased; 70 in 100 you would. Small samples swing. With 500 obligors at 1%, the expected count is only 5, and counts that small are lumpy. The question is not whether 9 is bigger than 5, but how often a correct 1% grade produces 9 or more by chance, and the exact answer is about one year in fifteen.
At a true 1% default rate across 500 obligors the count centres on 5, and 9 or more defaults happens 6.7% of the time. The one-sided 95% critical value is 10 defaults, so the observed 9 sits just inside the acceptance region.
Why does the normal approximation get it wrong here?
The standard deviation of the count is the square root of 500 x 0.01 x 0.99, about 2.22. Nine is 1.80 standard deviations above five, past 1.645, so a quick z-test rejects. But the binomial is skewed and discrete at small counts; with a continuity correction z falls to 1.57 and the conclusion flips back. For counts under about ten, use the exact binomial.
The relationship
$P(X \ge 9) = \sum_{k=9}^{500} \binom{500}{k}(0.01)^k(0.99)^{500-k} \approx 0.067 > 0.05$
X the number of defaults in the grade
500 obligors in the grade
0.01 the predicted default rate under test
What it says in wordsAdd up the chance of every outcome at least as bad as nine; if that total is above 5%, you do not reject.
Two further limitations make the case for caution stronger. Defaults are correlated through the economy, so a single bad year moves many obligors together and the true spread of counts is wider than the binomial says. And a validation team would look at several years and at neighbouring grades before recalibrating. The right answer is to flag the grade for watching, not to declare it broken.
Where candidates lose it
The trap is the quick z-test: (9 - 5) / 2.22 = 1.8, above 1.645, reject. It is the answer most candidates give, and it is wrong because a normal curve is a poor stand-in for a skewed count with a mean of five.
The second miss is failing to mention correlation. Independent defaults are the most favourable assumption for rejecting; real defaults cluster, which makes nine even less conclusive.
What the interviewer asks next
How many defaults would reject at 99% confidence?
The grade shows 8, 9 and 7 defaults in three consecutive years. What now?
How does default correlation change the test?
087A desk's daily returns have a volatility of 1%. Over 250 days its average daily return is 0.05%. How precisely is that mean estimated, and can you say the desk has skill?Statistics and estimationCoreAsset manager riskQuant risk
Try it first
Is 0.05% a day, measured over one year, clearly different from zero?
Show the worked solution
Not precisely enough to claim skill. The standard error of the mean is the daily volatility over the square root of the number of days: 1% over the square root of 250, about 0.063%. The estimate of 0.05% is only 0.79 standard errors from zero, and a two standard error band runs from -0.076% to 0.176%. You would need about 1,600 days, 6.4 years, to clear zero.
Why is a year of daily data not enough?
Weigh yourself on a bathroom scale that jumps by two kilos each time you step on it. If you lost 100 grams last month, a week of readings will not show it; the jumps drown the signal. The precision of an average improves only with the square root of the number of observations, so a small edge buried in large daily noise takes years to show. Here the daily noise is 1% and the edge is 0.05%, twenty times smaller.
The relationship
$SE = \frac{\sigma}{\sqrt{n}} = \frac{1\%}{\sqrt{250}} = 0.063\% \qquad t = \frac{0.05}{0.063} = 0.79$
sigma daily return volatility, 1%
n number of daily observations, 250
t how many standard errors the mean is from zero
What it says in wordsThe mean's uncertainty is the daily volatility divided by the square root of the number of days, and the edge is less than one of those units from zero.
After 250 days the two standard error band around the 0.05% daily mean runs from -0.076% to 0.176% and straddles zero; after about 1,600 days, 6.4 years, the band narrows to 0.000% to 0.100% and only just clears zero.
How long would it take, and what does that mean for judging desks?
Set the t-statistic to 2 and solve for n: n equals (2 times 1% over 0.05%) squared, which is 1,600 days. An annual Sharpe ratio of about 0.79 needs more than six years of data before it is statistically distinguishable from zero. That is longer than most desks keep the same strategy, so a risk team cannot rely on the P&L record alone; it looks at whether the edge has a reason, whether it survives out of sample, and how much the result depends on a few days.
State the assumptions. The calculation treats daily returns as independent with constant volatility. Fat tails and volatility clustering make the true uncertainty larger, so six years is a floor, not a promise.
Where candidates lose it
The common error is annualising the mean to 12.5% and declaring skill, as if a big annual number were proof. The annual volatility grows too, to about 16%, and the ratio of the two is what matters.
The other slip is dividing by 250 instead of its square root, which makes the mean look fifty times more precise than it is.
What the interviewer asks next
The desk's volatility is 0.5% instead. How many days now?
Why is the mean so much harder to estimate than the volatility?
How would you judge a new desk that has only six months of history?
098A delinquency model was trained on a sample oversampled to 50% bad accounts, while the true bad rate is 2%. It scores an applicant at 30%. What is that applicant's probability of going bad in the real population?Statistics and estimationHardNeuberger BermanChicago · 2024
Try it first
Roughly what is the applicant's real-world probability of going bad?
Show the worked solution
About 0.87%. Oversampling inflates every score by the same factor on the odds. The sample's odds of bad are 50 to 50; the population's are 2 to 98, one forty-ninth as high. The applicant's sample odds are 30 to 70, 0.429; divide by 49 to get 0.00875, which is a probability of 0.87%. The model ranks correctly but must be recalibrated.
Why does oversampling change the score?
Suppose a doctor learns to spot a rare illness from a teaching ward where half the patients have it. In a village clinic, where only one patient in fifty has it, the same symptoms mean far less. A model trained on a 50% bad sample has learned a base rate 25 times too high, so every score it produces is inflated, even though the ordering from safest to riskiest is still right. Oversampling is done on purpose, to give the model enough bad accounts to learn from, so the correction is a routine step, not a sign of a broken model.
On a log-odds scale, oversampling shifts every score by the same amount, the log of 49, so a sample score of 30% maps to 0.87% in the real population and a sample score of 50% maps back to exactly the 2% base rate.
The relationship
$\text{odds}_{pop} = \text{odds}_{sample} \times \frac{\pi/(1-\pi)}{s/(1-s)} = \frac{0.3}{0.7} \times \frac{0.02/0.98}{0.5/0.5} = 0.00875$
pi true bad rate in the population, 2%
s bad rate in the training sample, 50%
odds probability of bad divided by probability of good
What it says in wordsMultiply the model's odds by the ratio of the population's odds of bad to the sample's; then convert odds back to a probability.
How do you sanity-check the answer?
Take an applicant the model scores at exactly 50%, the sample average. After correction that applicant should sit at the population average, and the formula gives exactly 2%, which confirms the factor. Then note that 30% is below the sample average, so the corrected figure should be below 2%, and 0.87% is. The shortcut of scaling the probability by 2% over 50% gives 1.2%, which is close for low scores but breaks down badly for high ones: a sample score of 90% would scale to 3.6%, while the correct answer is about 15.5%.
Say what a validator would do next. The correction assumes the good and bad accounts were each sampled at random within their class. If the bad accounts were drawn from a different period or channel, the model's ranking may also be off, and the fix is to check calibration on a recent, unsampled holdout, comparing predicted and actual bad rates by score band.
Where candidates lose it
The first trap is reporting 30% as the applicant's risk, which overstates it about thirty-fivefold and would, in a pricing or provisioning model, charge far too much for the loan.
The second is correcting the probability instead of the odds. It is close at low scores and wrong at high ones; say you adjust the odds, and check with the 50% applicant.
What the interviewer asks next
What does the corrected probability become for an applicant the model scores at 90%?
Does oversampling change the model's Gini or only its calibration?
How would you recalibrate if the true bad rate itself shifts in a downturn?
Asked at Neuberger Berman, Risk, Chicago, 2024 (Wall Street Oasis): How would you approach building a delinquency model?

p	the tail probability, 1%
n	the number of paths
f(q)	the height of the P&L density at the VaR point

rho	the estimated correlation, 0.5
n	the number of observations, 36
tanh^{-1}	Fisher's transformation, which makes the sampling error close to normal

Q	the process variance, 1: how much the true value moves each step
R	the measurement variance, 4: how noisy each reading is
P^-	the variance of the estimate just before a reading arrives
K	the Kalman gain

bias	the average forecast error, forecast minus actual
variance	the spread of the errors around their own average, the standard deviation squared

Try it first

Why does a 99% VaR rest on so few scenarios?

Why does halving the error cost four times the paths?

Where candidates lose it

What the interviewer asks next

Try it first

Why is a correlation from 36 months so imprecise?

How do you test whether the relationship changed?

Where candidates lose it

What the interviewer asks next

Try it first

What is a Kalman filter doing, in one picture?

Why does the gain settle rather than fall to zero?

Where candidates lose it

What the interviewer asks next

Try it first

How can a biased forecaster beat an unbiased one?

When would your answer flip, and what would you do with A?

Where candidates lose it

What the interviewer asks next

Try it first

Why does dividing by n come out too low?

What is the difference between unbiased and consistent?

Where candidates lose it

What the interviewer asks next

Try it first

Why blend variances rather than volatilities?

What happens next, and what are the model's limits?

Where candidates lose it

What the interviewer asks next

Try it first

How does the filter decide how far to move?

Why is the new variance smaller than both inputs?

Where candidates lose it

What the interviewer asks next

Try it first

How surprising are nine defaults if the 1% is right?

Why does the normal approximation get it wrong here?

Where candidates lose it

What the interviewer asks next

Try it first

Why is a year of daily data not enough?

How long would it take, and what does that mean for judging desks?

Where candidates lose it

What the interviewer asks next

Try it first

Why does oversampling change the score?

How do you sanity-check the answer?

Where candidates lose it

What the interviewer asks next