Quant puzzles, solved step by step

Puzzles: 100
Traced to a firm: 71
Topics: 12
Hard: 30

Showing 1–2 of 2 · filtered from 100Clear filters

017The true model is y = x1 + x2 + noise, where x1 and x2 are standardised and have correlation 0.5. You regress y on x1 alone, then regress the residuals on x2. What coefficient do you get on x2, and how would you recover the true value of 1 in two stages?Correlation, regression and linear algebraHardQuant researchQuant trading
Try it first
What coefficient does the second stage give on x2?
Show the worked solution
You get 0.75, not 1. Regressing y on x1 alone gives a slope of 1 + 0.5 = 1.5, because x1 soaks up the half of x2 that moves with it. The residual is x2 - 0.5x1 + noise, whose slope on x2 is 1 - 0.5 squared = 0.75. To recover 1, residualise x2 on x1 as well and regress the residual of y on the residual of x2: the Frisch-Waugh-Lovell theorem.
Where does the missing quarter go?
Picture two salespeople who often work the same client. If you credit all joint sales to the first before looking at the second, the second looks worse than they are, because some of their work was already booked to the first. Stage one regresses y on x1 alone, and since x2 is correlated with x1, the coefficient on x1 rises to 1.5: it takes credit for 0.5 of x2. That piece has been removed from the residual, so stage two can only find what is left of x2's effect.
With a correlation of 0.5, x2 splits into 0.5 x1 plus an orthogonal part; stage one assigns the 0.5 x1 piece to x1, so regressing the residual on raw x2 gives 0.75, while regressing it on the orthogonal part of x2 recovers the true 1.
How do you get 0.75 exactly?
Write the residual out. y - 1.5x1 = x2 - 0.5x1 + noise, and the slope of that on x2 is its covariance with x2 over the variance of x2: (1 - 0.5 x 0.5)/1 = 0.75. The formula generalises to 1 - rho squared times the true coefficient, so the bias gets worse as the regressors get more correlated: with rho = 0.9 you would find only 0.19. A simulation of 100,000 observations gives 1.506 for stage one and 0.752 for stage two.
The relationship
$\hat\beta_{2,\text{seq}} = \frac{\operatorname{Cov}(x_2 - \rho x_1,\ x_2)}{\operatorname{Var}(x_2)} = 1 - \rho^2 = 0.75 \qquad \hat\beta_{2,\text{FWL}} = \frac{\operatorname{Cov}(x_2 - \rho x_1,\ x_2 - \rho x_1)}{\operatorname{Var}(x_2 - \rho x_1)} = 1$
\rho the correlation between x1 and x2, 0.5
x_2 - \rho x_1 the part of x2 left after regressing it on x1
What it says in wordsRegressing on raw x2 shrinks the answer by one minus rho squared; regressing on the part of x2 orthogonal to x1 gives the true coefficient.
What does Frisch-Waugh-Lovell tell you to do?
To get a variable's coefficient from a multiple regression in stages, partial the other regressors out of both y and that variable, then regress residual on residual. Here that means regressing x2 on x1 as well, keeping the orthogonal part x2 - 0.5x1, and regressing the stage-one residual on it. The slope comes back as exactly 1; the simulation gives 1.003. This is why factor-neutralising a signal before testing it, rather than after, matters in quant research: the order of the stages changes the answer.
Where candidates lose it
The common answer is 1, on the belief that regressing residuals step by step is the same as a multiple regression. It is only the same when the regressors are uncorrelated, and the question gives you a correlation of 0.5 precisely to break that.
The second loss is saying the answer is biased without saying which way or by how much. Give 1.5 for stage one, 0.75 for stage two, the 1 - rho squared rule, and the fix.
What the interviewer asks next
What would the stage-two coefficient be if the correlation were -0.5?
In the two-stage FWL regression, how do the standard errors compare with the full multiple regression?
You have a new signal correlated with a known factor. How do you test whether it adds anything?
078Let A be the 2 by 2 matrix with 2 on the diagonal and 1 off the diagonal. Compute A to the power 10 without multiplying it out ten times.Correlation, regression and linear algebraCoreQuant researchQuant trading
Try it first
What is the top-left entry of A^10?
Show the worked solution
A^10 has 29,525 on the diagonal and 29,524 off it. A has eigenvalue 3 along (1, 1) and eigenvalue 1 along (1, -1). Writing A = Q D Q^T with D = diag(3, 1), the tenth power is Q D^10 Q^T, and only the numbers 3 and 1 get raised to the tenth. The entries are (3^10 + 1)/2 and (3^10 - 1)/2.
Why look for eigenvectors at all?
Think of a photocopier set to 300% on one axis and 100% on the other. Copy a copy ten times and you do not need to simulate every pass: that axis is 3 to the tenth times longer and the other is unchanged. An eigenvector is a direction the matrix only stretches, so applying the matrix ten times along it is just multiplying by the eigenvalue ten times. Symmetric matrices always have a full set of such directions at right angles, which is what makes this matrix easy.
Find them by inspection. Adding the two rows of A gives 3 in each, so A(1, 1) = (3, 3): eigenvalue 3. Subtracting gives 1, so A(1, -1) = (1, -1): eigenvalue 1. The trace is 4 and the determinant is 3, and 3 + 1 = 4 and 3 x 1 = 3, which confirms both in one line.
The matrix stretches the direction (1, 1) by a factor of 3 and leaves (1, -1) unchanged, so A to the tenth stretches them by 59,049 and 1, and converting back to ordinary coordinates gives 29,525 on the diagonal and 29,524 off it.
The relationship
$A = Q\begin{pmatrix}3&0\\0&1\end{pmatrix}Q^{T},\; Q=\tfrac{1}{\sqrt2}\begin{pmatrix}1&1\\1&-1\end{pmatrix} \Rightarrow A^{10} = \tfrac12\begin{pmatrix}3^{10}+1 & 3^{10}-1\\ 3^{10}-1 & 3^{10}+1\end{pmatrix}$
Q the matrix whose columns are the unit eigenvectors
D the diagonal matrix of eigenvalues, 3 and 1
Q^T the transpose of Q, which is also its inverse
What it says in wordsRotate into the eigenvector directions, raise each eigenvalue to the tenth, and rotate back.
Is there an even faster route for this particular matrix?
Yes. Write A = I + J, where J is the all-ones matrix. J squared is 2J, so every power of J is a multiple of J, and (I + J)^n collapses to I + ((3^n - 1)/2) J. For n = 10 that is I + 29,524 J, which gives 29,525 on the diagonal and 29,524 off it: the same answer, and a good cross-check to say aloud. A brute-force multiplication in code agrees exactly.
Say why this matters on a desk. A covariance matrix with equal variances and one common correlation has exactly this shape, and its eigenvectors are the market direction and the spread directions. Powers of transition matrices in Markov chains are computed the same way, and the eigenvalue closest to 1 tells you how fast the chain forgets where it started.
Where candidates lose it
The fast wrong answer raises each entry to the tenth, giving 1,024 on the diagonal and 1 off it. Matrix multiplication mixes rows and columns, so entries do not power separately; A squared already has 5 on the diagonal, not 4.
The second loss is diagonalising correctly and then fumbling the conversion back. The Q matrix carries a 1/sqrt(2) on each side, which becomes the factor of one half in the final answer. Check with the trace: the diagonal entries of A^10 must sum to 3^10 + 1.
What the interviewer asks next
What is A^n as n grows large, after dividing by 3^n?
Compute the square root of A, a symmetric matrix B with B squared equal to A.
Generalise: an n by n matrix with a on the diagonal and b everywhere else. What are its eigenvalues?

\rho	the correlation between x1 and x2, 0.5
x_2 - \rho x_1	the part of x2 left after regressing it on x1

Q	the matrix whose columns are the unit eigenvectors
D	the diagonal matrix of eigenvalues, 3 and 1
Q^T	the transpose of Q, which is also its inverse

Try it first

Where does the missing quarter go?

How do you get 0.75 exactly?

What does Frisch-Waugh-Lovell tell you to do?

Where candidates lose it

What the interviewer asks next

Try it first

Why look for eigenvectors at all?

Is there an even faster route for this particular matrix?

Where candidates lose it

What the interviewer asks next