Evaluate Alternative Data for an Investment Pitch
Company: Citadel
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: easy
Interview Round: Technical Screen
You are given an alternative dataset and asked to develop an investment pitch. Describe how you would determine whether the data contains a real, usable signal and how you would communicate the thesis without overstating what the analysis proves.
### Constraints & Assumptions
- The dataset is not a traditional company filing or market-price series.
- Historical outcomes are available for evaluation, but the vendor's collection process may have changed.
- The exercise tests data-science reasoning and communication, not confidential investment knowledge.
### Clarifying Questions to Ask
- What population, behavior, and time period does the dataset cover?
- How is each record collected, revised, and mapped to the entity of interest?
- What outcome and forecast horizon should the signal predict?
- When would the data have been available to an analyst in real time?
- What usage rights, privacy constraints, and vendor dependencies apply?
### What a Strong Answer Covers
- Source and coverage diligence, including selection bias, survivorship, revisions, missingness, and changing methodology.
- A precise hypothesis linking the observed behavior to an outcome through a plausible mechanism.
- Leakage-safe, point-in-time validation against simple baselines with transaction costs or operational constraints when relevant.
- Robustness checks across periods, entities, sectors, and reasonable modeling choices.
- Separation of predictive association from causal interpretation and a candid discussion of uncertainty.
- A concise pitch that states the thesis, evidence, risks, invalidation criteria, and next data needed.
### Follow-up Questions
1. How would you detect that a vendor backfilled historical records using information unavailable at the time?
2. What if performance is concentrated in one unusual period?
3. How would you test whether the signal duplicates information already in public data?
4. Which result would cause you to abandon the thesis?
Quick Answer: Evaluate an alternative dataset for an investment pitch through provenance checks, a falsifiable signal hypothesis, and point-in-time validation. Test incremental value, selection bias, revisions, robustness, and costs before presenting the thesis, risks, and invalidation criteria.