Case 068Regression and model reviewWarm up
An analyst adds 12 macro variables to a 3-variable return model estimated on 120 months, and R-squared rises from 8% to 17%. Compute adjusted R-squared before and after and an F-test on the added block. Did the twelve add anything?
1The situation
A Jalvika Capital analyst has a model of a strategy's monthly returns on three factors, estimated over 120 months, with an R-squared of 8%. Hoping to explain more, the analyst adds twelve macro variables: inflation surprises, rate changes, oil, the rupee and so on. R-squared rises to 17%, and the analyst reports that the macro block more than doubles the model's explanatory power.
You are asked to review the claim before it goes into the strategy's monthly report.
2Your task
Compute adjusted R-squared for both models, run the F-test for the twelve added variables, and say whether the macro block has earned its place.
Quick check
R-squared rose from 8% to 17% when twelve variables were added to a model fitted on 120 months. What does that tell you on its own?
Worked solution
Try it on paper, then open one step at a time.
30-second answerThe answer to give first
Adjusted R-squared falls from 5.6% to 5.0%, and the F-test on the twelve added variables gives F = 0.94 against a 5% critical value of about 1.85, so the macro block adds nothing. R-squared rises mechanically with every variable; twelve noise series would have added about 9.5 points on 120 months, and these added 9. The p-value is about 0.51. Keep the three-factor model.
Step 1Why does R-squared rise even when the new variables are noise?
Give a student twelve extra guesses on a multiple-choice test and the score rises whether or not the student knows anything, because some guesses land. Least squares does the same: every added variable is fitted to the sample's quirks, so R-squared can only rise, and with n observations and k variables a noise variable removes about 1/(n - k - 1) of the remaining unexplained variance on average. Here twelve noise series would raise R-squared by roughly 12/116 of the 92% unexplained, 9.5 points. The observed rise is 9.0 points. On its face, the macro block has done exactly what random numbers would have done.
Step 2What does adjusted R-squared say?
Adjusted R-squared charges a price for each variable: 1 - (1 - R-squared) x (n - 1)/(n - k - 1). Before, it is 1 - 0.92 x 119/116 = 5.62%; after, 1 - 0.83 x 119/104 = 5.03%, lower than before, so by this measure the twelve variables cost more than they explain. The fall is small because the penalty is a crude one, but the direction is the finding. A report that quotes R-squared doubling while adjusted R-squared fell is the kind of thing a reviewer catches in one line.
| Model | Variables | R-squared | Adjusted R-squared | Residual degrees of freedom |
|---|---|---|---|---|
| Three factors | 3 | 8.0% | 5.62% | 116 |
| Plus twelve macro variables | 15 | 17.0% | 5.03% | 104 |
Step 3Does the F-test agree?
The F-test asks whether the improvement in fit per added variable is larger than the unexplained variance per remaining degree of freedom. F = ((0.17 - 0.08)/12) / ((1 - 0.17)/104) = 0.00750 / 0.00798 = 0.94; under the null of no effect F averages about 1, the 5% critical value with 12 and 104 degrees of freedom is about 1.85, and the p-value is about 0.51. A result this size or larger happens half the time when the twelve variables are pure noise. For the block to clear the 5% bar, R-squared would have needed to rise by about 16 points, to 24%, not 9.
| q | number of variables added, 12 |
| R_1^2, R_2^2 | R-squared before and after, 0.08 and 0.17 |
| n - k_2 - 1 | residual degrees of freedom of the larger model, 104 |
Step 4What should the review say?
Drop the block, and say why in one sentence: on 120 months, twelve extra variables buy nine points of R-squared for free, and these bought nine. If the analyst believes one or two of the macro series matter, the honest route is to pick them before looking at the fit, add them alone, and test them on months the model has not seen. Adding a dozen and keeping whichever looks best is the same mistake with a smaller block. State the limitation: the F-test assumes independent, equal-variance errors, which monthly returns often break, so a robust version should confirm the result; but a block that cannot pass the ordinary test will not pass a stricter one.
Where candidates lose it
Candidates read a doubling of R-squared as a doubling of explanatory power. R-squared cannot fall when variables are added, and on a short sample a dozen variables add many points by chance; the only honest comparisons are adjusted R-squared and the F-test on the block.
The second miss is computing adjusted R-squared, seeing a small fall, and calling it close. The F-test puts a number on the chance that noise produced the whole rise, and that number is about one in two.
What the interviewer asks next
- How would the answer change with 600 months of data and the same R-squared figures?
- Why is testing the twelve jointly better than keeping the ones with t-statistics above 2?
- What out-of-sample check would you run before accepting any macro variable?
Company names and figures are illustrative.
