Case 073Systematic research and dataCore
How would you make money from social media data on Fizzora Beverages? You have three years of daily brand mentions and sentiment, and quarterly sales. Design the signal, the test and the trade, and say how you would know it is not noise.
1The situation
A data vendor offers three years of daily data on Fizzora Beverages, a listed soft-drinks maker: counts of brand mentions on social platforms and an average sentiment score. You also have Fizzora's twelve quarterly sales reports and the analyst consensus for each quarter's sales growth.
A first look is encouraging: year-on-year growth in mentions over a quarter correlates 0.91 with that quarter's sales growth.
2Your task
Turn the data into a signal you could trade, design the test that would convince a sceptical PM, and explain why the 0.91 correlation is not the number that matters.
Quick check
Mention growth correlates 0.91 with sales growth. Why is that not enough to trade on?
Worked solution
Try it on paper, then open one step at a time.
30-second answerThe answer to give first
Trade the part of mention growth that consensus does not already expect, and test whether it predicts the sales surprise, not sales. Raw mentions track sales at 0.91 but the surprise at only 0.21, because analysts see the same trends. The unexpected part correlates 0.75 with the surprise, but on twelve quarters, fitted in-sample. It is worth trading only after it holds out of sample, ideally across many brands.
Step 1What exactly should the signal predict?
The surprise, because that is what moves the price. If everyone in a housing society already knows the monsoon will be heavy, the umbrella seller's good season is no news to his landlord. Fizzora's price already reflects consensus sales growth, which itself correlates 0.97 with actual growth, so a signal that predicts sales growth mostly repeats what the market knows. Raw mention growth correlates 0.91 with sales growth but only 0.21 with the sales surprise to consensus.
Step 2How do you build the signal?
Strip out what consensus already implies. Regress mention growth on consensus growth across quarters; the leftover, the residualThe part of a variable that a regression does not explain: here, the mention growth left over after allowing for what consensus already expects., is the part of the buzz analysts are not counting. Here mentions rise about 1.6 points for each point of consensus growth, and the residual correlates 0.75 with the surprise. Add sentiment as a second input only if it adds to that, and fix every choice, windows, weights and filters, before looking at the results.
Step 3How would you know it is not noise?
Count your data points honestly first. With twelve quarters, a correlation needs to exceed about 0.58 to pass a 5% test, and that test assumes you did not choose the signal after looking at those same twelve points. The residual here was fitted on the same data, so 0.75 is an optimistic number. The fix is breadth: build the identical signal for forty consumer brands with the same data, 480 brand-quarters, where a correlation above about 0.09 is significant, and test it on quarters and brands that played no part in designing it. Check it with a placebo too: shuffle the dates and confirm the relationship disappears.
Step 4What is the trade?
Across many brands, go long those whose unexpected mention growth is highest and short those where it is lowest, a few weeks before each results date, and close after results. That makes the trade about relative surprises and removes the market's direction. Two practical checks close the answer: whether the vendor's history was recorded at the time or reconstructed later, since backfilled data can quietly include information from the future, and whether the edge survives trading costs on the names that are liquid enough to trade.
Where candidates lose it
The common loss is celebrating the 0.91 correlation with sales. The market pays for surprises, and analysts see the same trends, so the tradable relationship is with the surprise.
The second is drawing conclusions from twelve quarters on one company. The interviewer wants breadth, out-of-sample testing and a plan to guard against looking into the future.
What the interviewer asks next
- The vendor changed its method for counting mentions in year two. How would you detect it and adjust?
- How would you use sentiment alongside mention volume?
- The signal works in large brands but not small ones. What might explain it, and would you still trade it?
Asked at Two Sigma, Generalist, New York, 2024 (Wall Street Oasis): How would you make money with social media data?
Asked at Two Sigma, Generalist, New York, 2024 (Wall Street Oasis): I was asked a very open ended question about how I would make money using social media data
Company names and figures are illustrative.
