Design A/B Test for Google Maps UI Change

Quick Overview

This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Design A/B Test for Google Maps UI Change states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Design A/B Test for Google Maps UI Change

Company: Google

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Onsite

##### Scenario The product team wants to move the Google Maps search bar from the top to the bottom of the screen. ##### Question Design an A/B test to evaluate this UI change. Define hypothesis, primary and guardrail metrics, experiment unit, randomization, sample-size calculation, runtime, and how you would interpret the results. ##### Hints Explain success metrics (e.g., searches per session), guardrails (retention, errors), power analysis, novelty effects, and rollout decisions.

Quick Answer: This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Design A/B Test for Google Maps UI Change states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

|Home/Analytics & Experimentation/Google
Google logo
Google
Aug 4, 2025, 10:55 AM
mediumData ScientistOnsiteAnalytics & Experimentation
9
0

Design A/B Test for Google Maps UI Change

A/B Test Design: Moving the Google Maps Search Bar to the Bottom

Context

Google Maps is considering a UI change: moving the search bar from the top to the bottom of the screen on mobile. We want to evaluate whether this increases search usage without harming user experience or system stability.

Task

Design an experiment that covers:

  1. Hypotheses
  2. Primary success metric(s) and secondary metrics
  3. Guardrail metrics (safety/quality)
  4. Experiment unit and exposure rules
  5. Randomization and bucketing
  6. Sample-size calculation and runtime
  7. Analysis plan and validity checks
  8. Result interpretation and rollout decision

Clarifying Questions to Ask Guidance

  • Clarify the business objective, unit of analysis, time window, exposure definition, and primary metric.
  • State assumptions about instrumentation, randomization, sample size, and data quality.
  • Separate descriptive analysis from causal claims.

What a Strong Answer Covers Guidance

  • A metric framework with primary, guardrail, and diagnostic metrics.
  • A credible analysis or experiment design with clear assumptions and bias checks.
  • SQL/statistical logic for segmentation, variance, confidence, and data validation where relevant.
  • An actionable recommendation that explains trade-offs and next steps.

Follow-up Questions Guidance

  • What sanity checks would you run before trusting the result?
  • How would you handle novelty effects, seasonality, or selection bias?
  • What decision would you make if metrics disagree?
Loading comments...

Browse More Questions

More Analytics & Experimentation•More Google•More Data Scientist•Google Data Scientist•Google Analytics & Experimentation•Data Scientist Analytics & Experimentation