Analytics & Experimentation Interview Questions
Practice the exact questions companies are asking right now.
How to evaluate similar-listing notifications feature
Case study (Marketplace product analytics) Context: Circle is a US marketplace app for buying and selling second‑hand products. On a product listing p...
Evaluate new shop-ads ranking algorithm
You work on a marketplace with shop ads. A new ranking/recommendation algorithm is proposed to promote shop ads more aggressively, but stakeholders ar...
How to estimate feature impact on usage time
Problem A product team believes a new feature (or a variable you can influence, e.g., enabling notifications, new feed ranking, new UI) changes user t...
Design and evaluate a fraud detection strategy
Context You are interviewing for a Fraud Data Scientist role at a payments company. The company has a fraud model and some operational constraints. Pa...
Estimate ATE, ITT, and TOT from experiment
You are given a single dataset (CSV) from an A/B experiment on a streaming product. The goal is to estimate the causal effect of a personalization fea...
Measure feature impact with switchback, PSM, and CACE
You work at a ridesharing company and want to measure the impact of a new membership feature on rides-per-user (RPU). Part A — Switchback experimentat...
How validate a driving simulation is realistic?
You work on evaluating Waymo’s driving simulation. You have: - Real-world (logged) driving data collected on-road. - Simulated driving data generated ...
Design metrics and experiment for stolen-post detection
You work on Stolen Post Detection for a social platform (detecting content that is copied/reposted without permission). A new detection algorithm is p...
Identify User Interest in Group Video Calls Using Data
Group Video-Calling Feature Analysis Context You are asked to design, launch, and analyze a new group video-calling feature for a large social/messagi...
Use Bayes to interpret a broken radar alarm
A “radar” system (or anomaly alarm) is suspected to be unreliable. You are asked to interpret its alerts and recommend how to operate it. Given Define...
Design metrics and A/B test for maps and ETA
Context You work on Uber’s driver app. Drivers can navigate using either Google Maps or Uber Maps. Separately, Uber shows riders an estimated time of ...
Decide and experiment on Group Call feature
Assume today is 2025-09-01. You have only one table, calls_daily_agg(date, user_id, country, device_tier, one_to_one_calls_started, one_to_one_call_du...
Design and evaluate a new group call feature
Product / DS Case: Group Calls for Messenger Groups Messenger has Groups but does not currently support group calls. You are evaluating whether to bui...
Measure causal impact of YouTube ads
Estimate the incremental effect of a 6‑week YouTube campaign on weekly online sales. - Explain why naive OLS of sales on ad spend is biased; list at l...
Estimate free-trial conversion probability
Context A product offers a free trial. You are asked: “What is the probability of conversion for a group of people who started a free trial?” Data (as...
Design a switchback and choose block length
Switchback Experiment Design: Airport Pickup Pricing with Spillovers Context You are designing a switchback (time-based A/B) experiment for airport pi...
Investigate Causes of Cold Meal Deliveries
Investigate and Reduce Cold Food Deliveries Context You are a Data Scientist at a large food-delivery marketplace. Customer complaints about meals arr...
Measure App Store success and debug funnel anomaly
Part A — Product case: measuring success for a new App Store Shopify is launching a Shopify App Store where merchants can browse/install apps built by...
Decide on vegan-burger R&D investment
Investment Decision: Vegan Burger R&D and Launch Business Case You are evaluating whether to invest now in R&D to develop and launch a plant-based (ve...
Define engagement metrics and analyze comment distribution
You are a Data Scientist for a video platform. A PM asks you to: 1) Define metrics for “engagement” (they want a clear metric framework they can use i...