Diagnose Causes of Low Retention for FB Light evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
##### Scenario
Facebook has launched an Android-only lightweight app called FB Light in emerging markets to grow MAU, but after one year the retention rate is low.
##### Question
How would you diagnose and determine the root causes of the low retention for FB Light?
##### Hints
Use a structured framework: cohort retention analysis, segmentation (device, network, country), funnel drop-offs, qualitative feedback, competing apps, and propose experiments or product changes.
Quick Answer: Diagnose Causes of Low Retention for FB Light evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
Diagnose Low Retention for FB Light (Android-only, Emerging Markets)
Context
You are a data scientist on the FB Light team. The app launched a year ago in several emerging markets to grow monthly active users (MAU). Despite initial adoption, retention is low.
Assume you have access to product analytics (event logs, device/network metadata), crash/ANR logs, Play Store data, basic market intel, and the ability to run A/B tests.
Task
Outline how you would diagnose and determine the root causes of low retention and propose next steps.
Please cover:
Metrics and Definitions
Define retention clearly (e.g., D1, D7, D30, rolling monthly retention), activation, and any guardrail metrics.
Cohort Retention Analysis
How you’d construct cohorts (e.g., by install month) and compute retention. Include how you’d handle seasonality and app version rollouts.
Segmentation Strategy
Segment by country, acquisition channel, device (RAM/CPU/Android version/OEM), network quality (2G/3G/4G/Wi‑Fi, latency), app version, language, and user type (new vs returning). Explain why each matters.
Funnels and Drop-offs
Map the core funnel from install → open → sign-up/login → activation → week 1 engagement → return. Identify where to look for major drop-offs.
Hypotheses and Diagnostics
List top hypotheses (e.g., performance on low-end devices, data cost, sign-up friction/SMS OTP failures, push notification reliability, content/network effects, crash/ANR issues, feature gaps vs full app, acquisition quality). For each, describe the specific analysis you’d run to validate or refute it.
Qualitative and Market Inputs
How you’d use app reviews, surveys/interviews, support tickets, and competitive benchmarking.
Modeling and Causality Aids
Any predictive modeling (e.g., survival analysis or logistic regression for D7) to quantify drivers; how you’d avoid common biases.
Experiments and Product Changes
Prioritized experiments or changes to address likely root causes, with success metrics and guardrails.
Validation Plan
How you’d confirm fixes generalize across segments and avoid regressions.
Constraints & Assumptions
Preserve the scope, facts, inputs, and requested outputs from the prompt above.
If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.
Clarifying Questions to Ask Guidance
Clarify the business objective, unit of analysis, time window, exposure definition, and primary metric.
State assumptions about instrumentation, randomization, sample size, and data quality.
Separate descriptive analysis from causal claims.
What a Strong Answer Covers Guidance
A metric framework with primary, guardrail, and diagnostic metrics.
A credible analysis or experiment design with clear assumptions and bias checks.
SQL/statistical logic for segmentation, variance, confidence, and data validation where relevant.
An actionable recommendation that explains trade-offs and next steps.
Follow-up Questions Guidance
What sanity checks would you run before trusting the result?
How would you handle novelty effects, seasonality, or selection bias?