Case 047Systematic research and dataCore
A take-home asks you to regress 500 daily returns on the Selvara signal. You get a slope of 1.8, but three days with returns above 20% drive the fit. After winsorising at the 1st and 99th percentiles the slope is 0.4. Which do you report and why?
1The situation
A take-home exercise gives you 500 days of a stock's next-day returns and the value of the Selvara signal, a sentiment score, on the day before. Most signal values fall between about minus 0.6 and plus 0.6. You are asked to estimate how much next-day return each unit of signal predicts and to present the result.
An ordinary least squares regression gives a slope of 1.8. Plotting the data shows three days with returns of 22%, 25% and 23%, all on days with high signal values; every other day's return lies between about minus 5% and plus 5%. After winsorising both variables at the 1st and 99th percentiles, the slope is 0.4.
2Your task
Which slope do you report as your estimate, what checks do you run before deciding, and how do you present the choice?
Quick check
Which slope should lead your write-up?
Worked solution
Try it on paper, then open one step at a time.
30-second answerThe answer to give first
Report the winsorised slope of about 0.4 as the estimate, and show the naive 1.8 and the outlier check beside it. Three days out of 500 set the naive fit, so 1.8 describes those days, not the signal. Before deciding, check whether the three returns are data errors, such as an unadjusted corporate action, and how the slope moves when each is removed. The honest result is a weak relationship, not a strong one.
Step 1Why can three days out of 500 decide the answer?
Ask ten people their monthly income, include one billionaire, and the average describes nobody in the room. Least squares minimises squared errors, so a point far from the rest pulls the line towards itself with a force that grows with the square of its distance; three returns above 20% among days that otherwise stay within 5% dominate the fit. They also sit at high signal values, where a point has the most leverageIn a regression, how far a point sits from the average of the explanatory variable. High-leverage points can swing the slope on their own., so they tilt the whole line. The naive slope of 1.80 is the slope of three days, not of the signal.
Step 2What do you check before choosing?
First, whether the three days are real. A 22% one-day move in a stock is rare; a stock split or bonus issue that was not adjusted in the price series produces exactly this kind of jump. Look up the three dates: if they are data errors, fix the data and the question disappears; if they are real events, such as a takeover bid, ask whether the signal could plausibly have predicted them. Second, how fragile the estimate is. Rerun the regression without each of the three days and without all three, and try a robust method such as winsorising or a regression that down-weights large errors.
| Estimate | Slope | t-statistic | What it describes |
|---|---|---|---|
| Naive least squares, all 500 days | 1.80 | 5.2 | Mostly the three extreme days |
| Winsorised at 1st and 99th percentiles | 0.40 | 1.7 | Normal days, extremes capped |
| Least squares without the three days | 0.25 | 1.1 | The other 497 days only |
Step 3How do you present the choice?
Lead with the robust number and put the evidence for the choice next to it. A research write-up that reports 0.4 and shows that 1.8 came from three days reads as careful; one that reports 1.8 reads as a result that will not survive a live test. Say what the robust estimate means in plain terms: a one-unit move in the signal predicts roughly 0.4 percentage points of next-day return on an ordinary day, with a lot of noise around it: its t-statistic of about 1.7 is weak evidence on its own. Then say what you would do next: test the signal on a later period it has not seen, because a relationship estimated on one sample, however carefully, is still one sample.
Say the limitation of winsorising too. It is a choice, and the 1st and 99th percentiles are a convention, not a law. If the three days were real and the signal genuinely warns of jumps, capping them throws away the most valuable information in the data set, so the right conclusion is not that the jumps do not matter but that three examples cannot prove the signal predicts them. The right amount of care is modest: check the data, choose the estimate, show the evidence, and move on to testing the signal.
Where candidates lose it
The first loss is reporting 1.8 because it came out of the regression and has a respectable t-statistic. A plot of the data takes ten seconds and shows the three points; not plotting is the mistake the exercise is built to catch.
The opposite loss is deleting the three days quietly and reporting 0.4 as if nothing happened. The reviewer wants to see the check, the reason for the choice and the naive number alongside, so that the decision can be judged.
What the interviewer asks next
- The three dates turn out to be a 1:1 bonus issue that was not adjusted. What do you do?
- How would you choose the winsorising percentiles if the reviewer challenged 1% and 99%?
- What would a rank correlation between signal and return tell you here?
- How would you test whether the signal predicts large moves specifically?
Asked at Balyasny Asset Management, Quantitative Research, New York, 2024 (Wall Street Oasis): The take home exam is pretty untraditional, but not difficult. You need to take care of data outliers and do not overthinking.
Company names and figures are illustrative.
