Derivatives Foundation puzzles, solved step by step

Puzzles
100
Traced to a firm
66
Topics
12
Hard
29
Topic
All topicsMental maths and estimation9Random walks and Markov chains7Conditional probability and Bayes7Volatility and correlation7Option pricing intuition7Expected value and optimal stopping10Market making11Option payoffs and no-arbitrage10Probability and counting11Distributions and statistics8Games and logic8Betting and sizing5
Level
AnyWarm upCoreHard
Source
AnyReported at a firmStandard
Showing 1–1 of 1 · filtered from 100Clear filters
  1. 003A logger stamps every event to the nanosecond, nine decimal places, and whenever the timestamp is missing it writes nine zeros instead. In 100,000 records you find 15 whose fractional part is exactly nine zeros. What is the probability that at least one of those 15 is a filled-in missing value?Conditional probability and BayesHardJump TradingAnonymous interview candidate in · 2022

    Try it first

    First instinct: roughly how many genuine timestamps, out of 100,000, should end in nine zeros by chance?

    Show the worked solution

    Essentially 1; the 15 are gaps. A genuine stamp ends in nine zeros with probability 10^-9, so in 100,000 records you expect 0.0001 such endings. The chance that 15 or more arise genuinely is about 10^-72, which no reasonable prior on missing data can overcome: even a missing rate of one in ten thousand would produce about 10 filled-in endings. So the probability that at least one of the 15 is a filled-in value is 1 to every decimal place you could print.

    What is the question really asking you to compare?

    A shopkeeper who finds 15 notes with the same serial number does not ask what the chance of a coincidence is; she asks which explanation makes 15 identical notes likely. This is a Bayes question in disguise: compare how likely 15 all-zero endings are if nothing is missing against how likely they are if some values are missing, then weight by a prior. The first likelihood is astronomically small; the second is ordinary. The prior would need to be more extreme than anything a real system justifies to change the answer.

    Expected against observed, on a log scale: five orders of magnitude apart0.0000010.00010.011100count of records ending in nine zeros (log scale)expected if genuine: 100,000 x 10^-9 = 0.0001observed: 15genuineseenChance of 15 or more genuine all-zero endings: about 10^-72Even if only one record in 10,000 were missing you would expect 10 filled-in endings.So the 15 are almost all gaps, and at least one of them certainly is.
    Genuine nanosecond stamps should produce 0.0001 all-zero endings in 100,000 records, while 15 were observed, five orders of magnitude more, and the chance of 15 or more genuine ones is about 10^-72, so the observation is explained only by filled-in missing values.

    How do you put a number on the genuine case?

    Each of the 100,000 stamps ends in a specific nine-digit string with probability one in a billion, so the count of genuine all-zero endings is Poisson with mean 0.0001. The probability of exactly 15 is e^(-0.0001) times 0.0001^15 over 15 factorial, which is about 10^-72. You do not need the exact figure in the room; say that 0.0001 to the fifteenth power is 10^-60 before dividing by 15 factorial, and the interviewer has what they need. The point is to show you can set up the count, not to print 72 zeros.

    The relationship
    P(≥1 missing∣15)=1−P(15∣none) P(none)P(15)≈1−10−72 P(none)P(15)≈1P(\geq 1 \text{ missing} \mid 15) = 1 - \frac{P(15 \mid \text{none}) \, P(\text{none})}{P(15)} \approx 1 - \frac{10^{-72} \, P(\text{none})}{P(15)} \approx 1
    P(15 | none)the chance of 15 genuine all-zero endings when no value is missing, Poisson with mean 0.0001
    P(none)your prior that the data set has no missing values at all
    P(15)the overall chance of seeing 15, dominated by the missing-value explanation
    What it says in wordsThe probability that none are missing is the genuine likelihood times its prior, divided by the total, and the genuine likelihood is so small that the result rounds to 1 whatever prior you hold.

    What does the interviewer want to hear about the prior?

    The honest answer is that the question is underspecified: without a prior on how often values go missing, you cannot write a single number. Say that, then show it does not matter: for the posterior to drop even to 99.9% you would need a prior of no missing values more than 10^69 times stronger than the alternative, and no logging system earns that confidence. For contrast, a modest missing rate of one record in ten thousand would give an expected 10 filled-in endings, right where the observed 15 sits. A limitation worth adding: the argument assumes the genuine fractional digits are uniform, which breaks if the clock quantises to microseconds and pads with zeros itself.

    Where candidates lose it

    Candidates reach for the binomial probability of 15 genuine zeros and stop, reporting a tiny number as if it were the answer. The question asks for the probability of a missing value given the data, which needs the comparison with the alternative, not a single likelihood.

    The second loss is freezing because no prior is given. The strong move is to name the missing input, then show that the likelihood ratio is so lopsided that the prior cannot matter. That is what a desk wants: a conclusion that survives the unknown.

    What the interviewer asks next

    • Now the logger stamps to the microsecond, six digits, and pads with three zeros. Does the argument survive?
    • Suppose only 1 record ends in nine zeros. What would you conclude then, and what would you need to know?
    • How would you check the data itself rather than reason about it?

    Asked at Jump Trading, Prop Trading, Anonymous interview candidate in, 2022 (Wall Street Oasis): What is the probability of at least 1 missing value given that we see 15 data points with 0's in the end