Risk Management puzzles, solved step by step

Puzzles
100
Traced to a firm
17
Topics
13
Hard
30
Topic
All topicsCapital and leverage6Compounding and drawdowns8Correlation and diversification8Counterparty exposure and collateral7Credit risk arithmetic10Duration and rates7Liquidity and balance sheet7Logic, estimation and brainteasers7Operational loss and fraud7Options and Greeks7Probability and base rates8Statistics and estimation10VaR and expected shortfall8
Level
AnyWarm upCoreHard
Source
AnyReported at a firmStandard
Showing 1–2 of 2 · filtered from 100Clear filters
  1. 020An auditor samples 60 of 3,000 trades and finds no booking errors. What error rate can you rule out at 95% confidence?Operational loss and fraudHardOperational riskBig Four risk advisory

    Try it first

    What can you conclude from zero errors in 60?

    Show the worked solution

    You can rule out error rates above about 4.9%, and nothing below. If the true error rate were p, the chance of 60 clean trades would be one minus p, to the power 60. That falls to 5% at p of about 4.87%. The rule of three gives the same answer quickly: 3 divided by 60 is 5%. With a real error rate of 1%, a clean sample happens 55% of the time.

    Why does a clean sample not prove a clean book?

    Think of tasting one spoonful from a large pot to check the salt. A good spoonful tells you the pot is not badly over-salted, but a few salty patches could easily be missed. Zero errors in a sample puts an upper bound on the error rate; it never proves the rate is zero. At a true error rate of 1%, one trade in a hundred is wrong, yet 60 random trades would all be clean 55% of the time. The sample simply is not large enough to see errors that rare.

    A clean sample of 60 rules out rates above about 5%, and nothing below0%25%50%75%100%0%2%4%6%8%10%True error rate in the populationChance of seeing zero errors in 605%1% errors: 55% chance of a clean sample4.87%: the 95% boundruled outnot ruled outRule of three3 / 60 = 5%exact: 4.87%
    The chance that a sample of 60 trades shows no errors is 55% when the true error rate is 1% and falls to 5% only at an error rate of 4.87%, so a clean sample rules out rates above about 4.9% but nothing below.
    The relationship
    (1−p)60=0.05  ⇒  p=1−0.051/60≈4.87%rule of three: p≈3n=5%(1-p)^{60} = 0.05 \;\Rightarrow\; p = 1 - 0.05^{1/60} \approx 4.87\% \qquad \text{rule of three: } p \approx \frac{3}{n} = 5\%
    pthe true error rate in the population of trades
    60the sample size
    0.05the chance you accept of being wrong, for 95% confidence
    What it says in wordsFind the error rate at which a clean sample would be a one-in-twenty event; anything higher is ruled out.

    Where does the rule of three come from, and does the 3,000 matter?

    The chance of zero errors is roughly e to the power of minus n times p, and e to the minus 3 is about 5%. So n times p equal to 3 marks the 95% bound, which gives the rule of three: divide 3 by the sample size. The population of 3,000 barely matters here because the sample is only 2% of it. Sampling without replacement tightens the bound slightly: counting exactly, the book could hold at most 144 errors, about 4.80%, instead of the 4.87% the simple formula gives.

    Then give the practical point. If the firm's tolerance for booking errors is 1%, a sample of 60 cannot confirm it; you need about 300 clean trades, 3 divided by 1%, to rule out 1%. The limit is that the sample must be random; a sample of the easiest trades to check says little about the ones that go wrong.

    Where candidates lose it

    The trap is reporting that the error rate is zero, or that it is below one in 60. Neither follows. A clean sample of 60 is quite likely even when one trade in a hundred is wrong.

    The second miss is freezing on the exact formula. Give the rule of three first, then refine it to 4.87% if asked.

    What the interviewer asks next

    • How many trades must you sample, all clean, to rule out a 0.5% error rate?
    • The sample of 60 finds one error. What upper bound can you now give?
    • Why might a random sample still understate errors in complex trades?
  2. 045A bank records only operational losses above Rs 10 lakh. True losses follow an exponential distribution with a mean of Rs 20 lakh. What is the average recorded loss, and what goes wrong if you fit a severity model to the records as if they were complete?Operational loss and fraudHardOperational riskModel validation

    Try it first

    What is the average of the recorded losses?

    Show the worked solution

    Recorded losses average Rs 30 lakh, 50% above the true Rs 20 lakh. Because the exponential is memoryless, losses above Rs 10 lakh exceed it by an average of Rs 20 lakh, so they average Rs 30 lakh. Fitting the records as if complete overstates the size of a typical loss by half, pushes the 99th percentile from Rs 92 to 138 lakh, and misses the 39% of losses below the threshold.

    Why does a threshold raise the recorded average?

    A school that only records exam scores above 60 will report a class average far above the real one, because the weak scores never enter the register. A reporting threshold removes the small losses from the data, so any average taken from what is left overstates the typical loss. For an exponential, the shift is exact: it is memorylessFor an exponential distribution, knowing a value already exceeds some level tells you nothing new about how much further it goes; the excess has the same distribution as the original., so the excess above Rs 10 lakh again averages Rs 20 lakh, and recorded losses average Rs 30 lakh.

    A reporting threshold hides the small losses and inflates the averagereporting threshold, Rs 10 lakhneverrecorded39.3%true mean 20recorded mean 30020406080100Size of a single loss, Rs lakhFit the records as if complete:mean 30, not 20 (+50%)99th percentile 138, not 92and 39% of events missing
    With a Rs 10 lakh reporting threshold, the 39.3% of losses below it are never recorded, and the recorded losses average Rs 30 lakh against a true mean of Rs 20 lakh.
    The relationship
    E[L∣L>u]=u+μ=10+20=30,P(L>u)=e−u/μ=e−0.5=0.607E[L \mid L > u] = u + \mu = 10 + 20 = 30, \qquad P(L > u) = e^{-u/\mu} = e^{-0.5} = 0.607
    uthe reporting threshold, Rs 10 lakh
    \muthe true mean loss, Rs 20 lakh
    What it says in wordsAbove the threshold the average excess is unchanged, so the recorded mean is the threshold plus the true mean, and about 61% of losses are recorded.

    What exactly goes wrong in the fitted model?

    Two errors that pull in opposite directions. Severity is overstated: fit an exponential to the records and you get a mean of 30, so every quantile is 50% too high, with the 99th percentile of a single loss at Rs 138 lakh instead of Rs 92 lakh. Frequency is understated: only 60.7% of events are recorded, so the true count is 1.65 times the recorded one. A model that fits both naively gets the size of losses and the number of losses wrong at once, and the errors do not cancel in the tail.

    The fix is to fit a truncated distribution: treat the records as losses known to exceed Rs 10 lakh and estimate the parameters of the whole curve from that conditional shape. For an exponential that means fitting the excesses over 10, which recovers the mean of 20. The limit is that heavier-tailed distributions are not memoryless, the truncated fit becomes unstable when the threshold is high relative to the data, and the missing small losses still matter for frequency.

    Where candidates lose it

    The common wrong answer is Rs 20 lakh, assuming the threshold only removes data without changing the average. The next most common is Rs 15 lakh, from averaging the threshold and the mean.

    The deeper miss is answering the number but not the modelling consequence. The interviewer wants to hear that severity is inflated, frequency is understated, and that the fix is a truncated fit, not simply adding the small losses back by guesswork.

    What the interviewer asks next

    • What fraction of the total rupee value of losses falls below the threshold?
    • If true losses were lognormal instead, would the recorded mean rise by more or less?
    • How would you combine internal data with a threshold and external loss data?