Evaluate Optimal Jogging Routes Feature with A/B Testing
Quick Overview
Evaluates A/B test design for Google Maps jogging route recommendations. Strong answers define jogging intent and exposure, measure completed runs and satisfaction, include safety and privacy guardrails, and specify randomization, MDE, power, logging, and rollout criteria.
Evaluate Optimal Jogging Routes Feature with A/B Testing
Company: Google
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
##### Scenario
Google Maps plans a feature that recommends optimal jogging routes.
##### Question
How would you evaluate whether this idea is valuable for users and the business? Design an experiment for launch: success metrics, triggering logic, unit of randomization, guardrails, and minimum detectable effect (MDE).
##### Hints
Define primary KPI (e.g., completed jogs), secondary metrics, power assumptions, and how to log route-recommendation events.
Quick Answer: Evaluates A/B test design for Google Maps jogging route recommendations. Strong answers define jogging intent and exposure, measure completed runs and satisfaction, include safety and privacy guardrails, and specify randomization, MDE, power, logging, and rollout criteria.
Evaluate an Optimal Jogging Routes Feature with A/B Testing
Google Maps is considering a feature that recommends optimal jogging routes, such as safe, scenic, distance-appropriate routes when a user shows running intent.
Constraints & Assumptions
Treat this as a product value and experiment design question.
Define the trigger for exposure carefully so users are compared only when eligible.
Include user value, business value, safety, and privacy guardrails.
Specify logging needed to measure recommendation exposure and outcomes.
Clarifying Questions to Ask Guidance
How does the product infer jogging intent?
What does "optimal" mean: distance, safety, scenery, elevation, lighting, or personalization?
Is the feature available everywhere or only where map and safety data are reliable?
What business goal does Maps have for this feature?
Part 1 - Assess Product Value
How would you assess whether the idea is valuable for users and the business?
What This Part Should Cover Guidance
User outcomes such as completed jogs, route starts, satisfaction, safety, repeat use, and reduced route abandonment.
Business outcomes such as Maps engagement, retention, ecosystem value, and strategic differentiation.
Pre-launch research or logs that identify jogging intent.
Part 2 - Design the Experiment
How would you launch an experiment to validate impact?
What This Part Should Cover Guidance
Triggering and eligibility logic, unit of randomization, treatment and control, sample size, MDE, power, duration, and ramp plan.
Exposure logging for recommendations shown, accepted, modified, started, completed, and abandoned.
Part 3 - Metrics and Guardrails
What primary, secondary, and guardrail metrics would you use?
What This Part Should Cover Guidance
Primary metric such as completed jogs per eligible user or route starts completed.
Secondary metrics for adoption, route quality, engagement, satisfaction, retention, and repeat usage.
Guardrails for unsafe routes, privacy, battery, navigation failures, complaints, and user trust.
What a Strong Answer Covers Guidance
A strong answer defines eligibility and exposure precisely, measures completed user value rather than taps alone, and designs an experiment with safety and privacy guardrails.
Follow-up Questions Guidance
How would you avoid exposing unsafe or low-quality routes?