Derive variance and CTR confidence intervals
Company: Roblox
Role: Data Scientist
Category: Statistics & Math
Difficulty: medium
Interview Round: Technical Screen
CTR is a ratio metric C/I where C~Binomial(I, p). Answer:
1) Given I=200,000 and C=4,200, estimate p̂ and its standard error. Compute 95% CIs using (a) normal/Wald with continuity correction, (b) Wilson, and (c) exact Clopper–Pearson. Compare widths and coverage properties and state which you would use in production and why.
2) Daily pooled CTR across three days with impressions [50,000, 120,000, 30,000] and CTRs [2.0%, 2.6%, 1.8%]: compute (a) the pooled CTR, (b) the standard error of the pooled CTR, and (c) the day-to-day standard deviation of CTR treating days as the unit (unweighted STDDEV_SAMP) vs impression-weighted day-level SD. When is each appropriate?
3) Explain sample vs population variance (why divide by n−1 vs n). In SQL, when would you prefer STDDEV_SAMP vs STDDEV_POP for daily CTR and CPC aggregates? Provide concrete examples tied to experiment analysis.
Quick Answer: This question evaluates understanding of estimation and inference for proportions, including CTR point estimation, standard errors, multiple confidence-interval methods (Wald, Wilson, Clopper–Pearson), pooled estimation across days, and the distinction between sample and population variance for day-level variability.