# Predicting Stock Prices from Twitter Data Suppose you have access to a large stream of Twitter data — tweets with their text, timestamps, and author metadata — along with historical market data (prices and trading volumes) for the stocks you care about. How would you use the Twitter data to predict stock prices? Walk the interviewer through your end-to-end approach: how you would frame the prediction problem, what signal you would extract from tweets and how, what model you would build and train, and — critically — how you would convince yourself the model has real predictive power rather than just fitting noise. ```hint How to frame it Don't jump straight to a model. First pin down the *target*: predicting a raw price level is much harder and less useful than predicting **returns** or the **direction of movement** over a fixed horizon (next hour, next day). Pick a horizon and a stock universe, and define the label as the forward return computed only from information timestamped *after* the prediction time. ``` ```hint Turning tweets into features The raw text stream is not a feature. Think in layers: (1) map each tweet to tickers/companies (cashtags like `$AAPL`, entity linking), (2) score each tweet (sentiment, embeddings), and (3) aggregate per stock per time bucket — mean sentiment, tweet **volume** and its deviation from normal, author-credibility weighting. Abnormal tweet volume is often as informative as sentiment polarity. ``` ```hint The evaluation trap Financial time series make it very easy to fool yourself. Watch for **look-ahead bias** (using any data timestamped after prediction time), evaluate with **walk-forward, time-ordered splits** (never random shuffles), and always compare against a market-data-only baseline. Expect a tiny edge — directional accuracy barely above 50% can already be meaningful, and claims of 80% accuracy are a red flag. ``` ### Constraints & Assumptions - Assume a high-volume tweet stream (historically on the order of hundreds of millions of tweets per day) with text, timestamp, and author metadata (followers, account age). - Assume you can obtain aligned historical market data (prices, volumes) for your stock universe. - The signal-to-noise ratio is extremely low: markets are close to efficient, so any real predictive edge from public tweets will be small. - You may scope the problem down (e.g., liquid US large-cap stocks, a daily prediction horizon) as long as you state the assumption. - This was asked as an open-ended discussion question: the interviewer cares about how you structure the problem and reason about validity, not about naming one specific model. ### Clarifying Questions to Ask - What is the prediction horizon — minutes, hours, or days? Is the output meant to drive trading, or is it a research/analysis signal? - Which universe of stocks — a handful of heavily tweeted large caps, or thousands of tickers including illiquid names? - Besides the tweet stream, what data is available — historical prices, any labeled sentiment data, existing NLP tooling? - Are there latency constraints (real-time scoring of the stream), or is offline/daily prediction acceptable? - How will the prediction be consumed — as a standalone forecast, or as one signal among many in a larger model? ### What a Strong Answer Covers ```premium-lock What a Strong Answer Covers ``` ### Follow-up Questions - Your backtest shows a small but consistent edge. What checks would you run before believing the edge is real? - Tweet activity often spikes *after* a price move. How would you disentangle tweets that predict moves from tweets that merely react to them? - How would you defend the signal against adversarial content, such as coordinated pump-and-dump campaigns or bot networks? - If the signal decays after deployment, how would you detect that, and what would you do about it?