Diagnose sustained drop in executed trades

Quick Overview

This question evaluates competency in product analytics and data science skills including KPI decomposition, causal troubleshooting, experiment design, and instrumentation for diagnosing a sustained decline in executed trades per active user.

Diagnose sustained drop in executed trades

Company: Robinhood

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

Context: You’re the analyst for a brokerage product covering onboarding → trading. Over the last 5 weeks, executed_trades per active user fell 22% and stayed low. No major outages are reported. How would you pinpoint the root cause and validate it? 1) Build a KPI tree from acquisition → KYC approval → first funding → first order attempt → execution quality → short-term retention. For each node, specify an unambiguous metric definition, the expected range, and guardrail thresholds (e.g., new-user day-7 retention, order rejection rate, deposit success rate, quote latency P95). Explain how you’d detect which node(s) are responsible for most of the variance in executed_trades. 2) Propose a drilldown plan and justify the exact cut order: acquisition channel, signup cohort, account tenure, funding method, platform (iOS/Android/Web), asset class (equity/option/crypto), order type (market/limit), market-hours vs after-hours, geo, and order reject_code family. For each cut, state the diagnostic you’d run (e.g., waterfall attribution, interaction effects) and the decision you’d make based on each possible outcome. 3) Disambiguate product vs external drivers. List at least five sanity checks with data sources: market volatility/indices, holiday calendar, competitor promos/fee changes, symbol halts, exchange connectivity, release calendar, instrumentation drift. Describe exactly how each check could confirm or refute a hypothesis. 4) Suppose an onboarding flow change shipped on 2025-07-10 across all platforms. Design a confirmation plan that doesn’t expose 100% of traffic: choose experiment unit (user, geo, or time-based switchback), outline rollout/rollback, define primary/guardrail metrics, and detail power analysis/monitoring windows to avoid novelty bias. 5) If instrumentation is insufficient, specify the minimum additional events/dimensions you’d add (e.g., KYC step_id, funding error_code, order entry latency, quote staleness, partial fill counts) and how you’d backfill or triangulate with logs to avoid a blind spot while the fix is in progress.

Quick Answer: This question evaluates competency in product analytics and data science skills including KPI decomposition, causal troubleshooting, experiment design, and instrumentation for diagnosing a sustained decline in executed trades per active user.

|Home/Analytics & Experimentation/Robinhood
Robinhood logo
Robinhood
Oct 13, 2025, 9:49 PM
hardData ScientistTechnical ScreenAnalytics & Experimentation
11
0

Brokerage Analytics Troubleshooting: Trades Per Active User Down 22%

Context

You are the analyst for a brokerage product that spans onboarding through trading. Over the last 5 weeks, executed_trades per active user fell by 22% and has remained low. No major outages have been reported. Describe how you would pinpoint the root cause and validate it.

Assume executed_trades per active user is measured weekly as: total executed trades in week / total active trading users in week.

Tasks

  1. Build a KPI tree from acquisition → KYC approval → first funding → first order attempt → execution quality → short-term retention. For each node, specify:
    • An unambiguous metric definition
    • Expected range and guardrail thresholds (examples: new-user day-7 retention, order rejection rate, deposit success rate, quote latency P95)
    • How you would detect which node(s) are responsible for most of the variance in executed_trades
  2. Propose a drilldown plan and justify the exact cut order across: acquisition channel, signup cohort, account tenure, funding method, platform (iOS/Android/Web), asset class (equity/option/crypto), order type (market/limit), market-hours vs after-hours, geo, and order reject_code family. For each cut, state the diagnostic you’d run (e.g., waterfall attribution, interaction effects) and the decision you’d make based on each possible outcome.
  3. Disambiguate product vs external drivers. List at least five sanity checks with data sources (e.g., market volatility/indices, holiday calendar, competitor promos/fee changes, symbol halts, exchange connectivity, release calendar, instrumentation drift). Describe exactly how each check could confirm or refute a hypothesis.
  4. Suppose an onboarding flow change shipped on 2025-07-10 across all platforms. Design a confirmation plan that doesn’t expose 100% of traffic: choose experiment unit (user, geo, or time-based switchback), outline rollout/rollback, define primary and guardrail metrics, and detail power analysis and monitoring windows to avoid novelty bias.
  5. If instrumentation is insufficient, specify the minimum additional events/dimensions you’d add (e.g., KYC step_id, funding error_code, order entry latency, quote staleness, partial fill counts) and how you’d backfill or triangulate with logs to avoid a blind spot while the fix is in progress.
Loading comments...