Apply sequential testing without p-hacking
Company: Meta
Role: Data Scientist
Category: Statistics & Math
Difficulty: hard
Interview Round: Onsite
You want continuous monitoring with early stopping for efficacy or futility. a) Contrast Pocock vs. O’Brien–Fleming alpha‑spending for two interim looks and report the corresponding critical z‑values. b) Explain how mixture‑SPRT or e‑values yield always‑valid inference under arbitrary peeking. c) Describe how you would integrate sequential monitoring with multiple metrics without inflating error rates.
Quick Answer: This question evaluates a candidate's competency in sequential hypothesis testing, alpha‑spending rules (Pocock vs O’Brien–Fleming), always‑valid inference methods such as mixture‑SPRT and e‑values, and strategies for controlling error rates across multiple metrics.