A/B Test Dashboard Interpretation and Core Statistics Concepts
Scenario
You are reviewing an A/B test dashboard for an experiment (e.g., Variant B vs Control A on conversion rate) and must explain statistical outputs to non-technical stakeholders during a technical phone screen.
Questions
-
What is a confidence interval, and how would you interpret it in an experiment report?
-
List two shortcomings of relying solely on p-values.
-
Compare bar chart, box plot, and violin plot for displaying a distribution; give pros and cons for each.
-
Explain the bias–variance trade-off in model evaluation.