Measure outage impact; choose fix vs build

Quick Overview

This question evaluates instrumentation design, telemetry analysis, causal impact estimation, failure deduplication, and decision-making under uncertainty for a Data Scientist, testing competencies in analytics, experimentation, and product-metric prioritization.

Measure outage impact; choose fix vs build

Company: Google

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

A major enterprise customer reports frequent Google Meet call drops. Build an end-to-end analysis plan: precisely define a “drop” (e.g., unexpected disconnect requiring rejoin within 60s), list instrumentation (session start/stop, reconnect attempts, ICE state changes, RTT/jitter/packet loss, device/OS/app version, network type/ASN, region), and deduplicate correlated failures; quantify business impact in three layers—meeting-minutes lost, user productivity loss, and account-level retention risk—and estimate marginal impact per 1pp increase in drop rate using matched controls or hierarchical models; determine whether to prioritize fixing the bug or building a new feature by framing expected value = impact × likelihood × duration ÷ effort, including opportunity cost and guardrail metrics (crash rate, CPU, startup latency); check if this is a regression by analyzing by-version/by-region canaries or holdbacks; produce a decision memo with assumptions, sensitivity analysis, and the exact observations that would invalidate your recommendation.

Quick Answer: This question evaluates instrumentation design, telemetry analysis, causal impact estimation, failure deduplication, and decision-making under uncertainty for a Data Scientist, testing competencies in analytics, experimentation, and product-metric prioritization.

|Home/Analytics & Experimentation/Google
Google logo
Google
Oct 13, 2025, 9:49 PM
hardData ScientistTechnical ScreenAnalytics & Experimentation
8
0

End-to-End Analysis Plan: Investigating Frequent Google Meet Call Drops

Context

A major enterprise customer reports frequent Google Meet call drops. As a Data Scientist in a technical screen, outline a rigorous plan to diagnose, quantify impact, prioritize action, and recommend a decision under uncertainty.

Tasks

  1. Define a drop precisely (e.g., unexpected disconnect requiring a rejoin within 60s). Clarify exclusions (user hang-up, host ends meeting) and edge cases.
  2. Specify client- and server-side instrumentation needed, including (but not limited to):
    • Session start/stop, reconnect attempts, ICE state changes
    • WebRTC metrics (RTT, jitter, packet loss, bitrate, frame drops)
    • Device/OS/app version, CPU/memory, background/foreground
    • Network type (Wi‑Fi/cellular), carrier/ASN, NAT type/VPN, region
    • Error codes/crash logs; server capacity/errors
  3. Propose a deduplication strategy to avoid overcounting correlated failures across layers (client/network/server) and multiple reconnects.
  4. Quantify business impact in three layers:
    • Meeting-minutes lost
    • User productivity loss
    • Account-level retention risk
  5. Estimate marginal impact per 1 percentage point (pp) increase in drop rate using matched controls and/or hierarchical models.
  6. Prioritize fixing the bug vs. building a new feature via expected value: expected value = impact × likelihood × duration ÷ effort. Include opportunity cost and guardrail metrics (e.g., crash rate, CPU, startup latency).
  7. Check for regression: analyze by-version and by-region using canaries/holdbacks.
  8. Produce a decision memo that states assumptions, sensitivity analysis, and specific observations that would invalidate your recommendation.
Loading comments...