PracHub
QuestionsPremiumCoachesLearningGuidesInterview Prep
|Home/Analytics & Experimentation/Dropbox

Run org-safe online experiment for recommender

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's competency in online experimentation design, causal inference, clustered randomization, metric definition (primary, secondary, guardrails), sample size/MDE computation, variance reduction and sequential monitoring, instrumentation and privacy-aware logging.

  • Medium
  • Dropbox
  • Analytics & Experimentation
  • Data Scientist

Run org-safe online experiment for recommender

Company: Dropbox

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: Medium

Interview Round: Technical Screen

Propose an online experimentation plan to evaluate the file recommender in production across multiple organizations where collaborators can influence one another. Specify: primary metrics (e.g., file open-through-rate, time-to-open), secondary/business metrics (productivity proxies), and guardrails (latency, error rate, privacy incidents, access denials). Choose the unit of randomization (org-, team-, or user-level) and justify to minimize spillover; describe bucketing, stickiness, and holdouts. Compute required sample size and MDE with clustering/ICC assumptions; select variance reduction (CUPED/stratification) and sequential monitoring approach with alpha spending. Detail ramp schedule, novelty and carryover controls, and interference detection. Define logging needed to reconstruct exposure and attribution, plus a difference-in-differences fallback if only partial randomization is possible. Explain stop/ship criteria and how to guard against Simpson’s paradox across tenants and roles.

Quick Answer: This question evaluates a data scientist's competency in online experimentation design, causal inference, clustered randomization, metric definition (primary, secondary, guardrails), sample size/MDE computation, variance reduction and sequential monitoring, instrumentation and privacy-aware logging.

Dropbox logo
Dropbox
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Analytics & Experimentation
2
0

Propose an online experimentation plan to evaluate the file recommender in production across multiple organizations where collaborators can influence one another. Specify: primary metrics (e.g., file open-through-rate, time-to-open), secondary/business metrics (productivity proxies), and guardrails (latency, error rate, privacy incidents, access denials). Choose the unit of randomization (org-, team-, or user-level) and justify to minimize spillover; describe bucketing, stickiness, and holdouts. Compute required sample size and MDE with clustering/ICC assumptions; select variance reduction (CUPED/stratification) and sequential monitoring approach with alpha spending. Detail ramp schedule, novelty and carryover controls, and interference detection. Define logging needed to reconstruct exposure and attribution, plus a difference-in-differences fallback if only partial randomization is possible. Explain stop/ship criteria and how to guard against Simpson’s paradox across tenants and roles.

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Dropbox•More Data Scientist•Dropbox Data Scientist•Dropbox Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 8,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.