Compute p-values, power, and adjust errors
Company: Meta
Role: Data Scientist
Category: Statistics & Math
Difficulty: hard
Interview Round: Onsite
Answer the following statistics questions precisely, showing calculations and interpretation:
(a) After 5 interim looks, you observe Control: 12,000 users, 1,380 conversions; Treatment: 12,200 users, 1,512 conversions. Compute the two-sided p-value and a 95% CI for the difference in proportions. Then adjust for sequential monitoring using an O’Brien–Fleming alpha-spending approach (describe whether the result would still be significant without fully recomputing spending functions if you lack the exact schedule).
(b) You track 5 metrics with sorted p-values {0.001, 0.012, 0.019, 0.070, 0.300}. Apply Benjamini–Hochberg at FDR q=0.05 and state which metrics are discoveries. Compare to Bonferroni at familywise α=0.05.
(c) Bot detection: prevalence is 5%. Your model flags a user as a bot with FPR=2% and FNR=10%. If a user is flagged, what is the posterior probability they are truly a bot? Show Bayes’ theorem numerically.
(d) Average Order Value (AOV) is right-skewed (mean=32, sd=15, heavy tail). Propose a robust approach for outlier handling and inference: compare IQR rule, z-scores, and MAD/Huber M-estimators; recommend a log-transform with delta and explain how to report effects back on the original scale with a smearing estimator.
Quick Answer: This question evaluates statistical reasoning and applied inference skills including hypothesis testing for proportions with sequential monitoring, multiple-testing control (Benjamini–Hochberg vs Bonferroni), Bayesian posterior estimation for classification flags, and robust AOV estimation and transformation techniques.