Case 029Signal research and data tasksHard
A social media sentiment signal on 150 consumer stocks goes long the top decile and short the bottom, with a 53% hit rate on two-day holds and an average move of 1.2%. Costs are 15 basis points a side and capacity is capped at 1% of daily volume. Is there a business, and how big?
1The situation
Zorvani Analytics has built a daily sentiment score from public social media posts about 150 listed consumer companies. Each day it buys the 15 names with the best score and sells short the 15 with the worst, holding each position for two trading days. In a three-year backtest the direction is right on 53% of trades and the average absolute two-day move is 1.2%.
Execution costs, spread plus impact, are about 15 basis points each way. The median stock trades Rs 15 crore a day, and the risk team caps trading at 1% of daily volume per name.
2Your task
Work out the edge per trade after costs, whether the 53% is even real, how much capital the idea can hold, and what would turn it into a business.
Quick check
Before costs, roughly what is a 53% hit rate on 1.2% moves worth per trade?
Worked solution
Try it on paper, then open one step at a time.
30-second answerThe answer to give first
Not as it stands: the signal earns about 7.2 basis points a trade before costs and pays 30 to trade, losing about 23 basis points a trade. It would need a 62.5% hit rate at these costs. Capacity is also small, about Rs 4.5 crore of gross positions. The business, if any, is as an overlay on trades already being made, or with a longer hold if the edge persists.
Step 1What is the edge per trade before costs?
Think of a coin that lands heads 53 times in 100 and pays Rs 120 either way. You collect 53 x 120 and pay 47 x 120, a net of Rs 720 over 100 tosses, Rs 7.20 a toss. The edge is the hit rate's excess over one half, doubled, times the average move: 0.06 x 1.2%, which is 7.2 basis points a trade. A 53% hit rate sounds like a strong signal; on symmetric moves it is a thin one. It also assumes winners and losers are the same size, which you would check in the data before anything else.
Step 2What would it take to clear the costs?
Reverse the arithmetic. To cover a 30 basis point round trip on 1.2% moves, the hit rate must satisfy (2p - 1) x 120 = 30, which is p = 62.5%. Or, at 53%, the round trip must cost under 7.2 basis points, which no one executing a two-day consumer stock rotation achieves. The two honest routes are a longer hold, if the signal keeps predicting beyond two days, so that one round trip pays for a larger move, and trading it where the cost is already paid.
Step 3Is the 53% even real?
The backtest has about 11,250 trades, and a naive standard error on a hit rate is the square root of 0.25 over that count, 0.47 percentage points, which makes 53% look like 6.4 standard errors. But the 30 trades placed on the same day share one market mood, so they are closer to one bet than thirty; counting one independent bet per holding period gives 375 bets and a standard error of 2.6 points, and 53% is only 1.2 standard errors from a coin. The truth sits between the two, and it needs checking with a test that clusters trades by date. Add that social media data invites many tried variants, and the 53% may be the best of several, which the multiple testingTrying many versions of a signal and keeping the best one. The winner looks stronger than it is because some of its edge is luck. problem shrinks further.
Step 4How big could it be, and where is the business?
At 1% of a Rs 15 crore daily volume, each position can be about Rs 15 lakh, so 30 names hold about Rs 4.5 crore of gross positions. Even if the costs vanished, 7.2 basis points on that book turning over every two days is about Rs 41 lakh a year, too small to pay for the data and the people on its own. The business is as an input: a fund already trading these stocks for other reasons can lean each order by the sentiment score and collect a few basis points on trades whose costs it pays anyway. That is how most thin signals earn their keep.
Where candidates lose it
The common loss is being impressed by the hit rate. Fifty-three per cent on its own says nothing about money; the edge is the 3 points over a half, doubled, times the move, and here it is a quarter of the cost of trading.
The second is stopping at the per-trade answer without capacity. Interviewers asking how big want the rupee figure: a signal that clears costs but holds only Rs 4.5 crore is a research note, not a business.
What the interviewer asks next
- How would you test whether the edge lasts beyond two days?
- The winners average 1.4% and the losers 1.0%. Redo the edge.
- How would you combine this signal with an existing momentum model?
- What biases does social media data carry that price data does not?
Asked at Two Sigma, Generalist, New York, 2024 (Wall Street Oasis): How would you make money with social media data?
Company names and figures are illustrative.
