PracHub
QuestionsLearningGuidesInterview Prep
|Home/Analytics & Experimentation/Dropbox

Run org-safe online experiment for recommender

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's competency in online experimentation design, causal inference, clustered randomization, metric definition (primary, secondary, guardrails), sample size/MDE computation, variance reduction and sequential monitoring, instrumentation and privacy-aware logging.

  • medium
  • Dropbox
  • Analytics & Experimentation
  • Data Scientist

Run org-safe online experiment for recommender

Company: Dropbox

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Technical Screen

Propose an online experimentation plan to evaluate the file recommender in production across multiple organizations where collaborators can influence one another. Specify: primary metrics (e.g., file open-through-rate, time-to-open), secondary/business metrics (productivity proxies), and guardrails (latency, error rate, privacy incidents, access denials). Choose the unit of randomization (org-, team-, or user-level) and justify to minimize spillover; describe bucketing, stickiness, and holdouts. Compute required sample size and MDE with clustering/ICC assumptions; select variance reduction (CUPED/stratification) and sequential monitoring approach with alpha spending. Detail ramp schedule, novelty and carryover controls, and interference detection. Define logging needed to reconstruct exposure and attribution, plus a difference-in-differences fallback if only partial randomization is possible. Explain stop/ship criteria and how to guard against Simpson’s paradox across tenants and roles.

Quick Answer: This question evaluates a data scientist's competency in online experimentation design, causal inference, clustered randomization, metric definition (primary, secondary, guardrails), sample size/MDE computation, variance reduction and sequential monitoring, instrumentation and privacy-aware logging.

|Home/Analytics & Experimentation/Dropbox

Run org-safe online experiment for recommender

Dropbox logo
Dropbox
Oct 13, 2025, 9:49 PM
mediumData ScientistTechnical ScreenAnalytics & Experimentation
5
0

Propose an online experimentation plan to evaluate the file recommender in production across multiple organizations where collaborators can influence one another. Specify: primary metrics (e.g., file open-through-rate, time-to-open), secondary/business metrics (productivity proxies), and guardrails (latency, error rate, privacy incidents, access denials). Choose the unit of randomization (org-, team-, or user-level) and justify to minimize spillover; describe bucketing, stickiness, and holdouts. Compute required sample size and MDE with clustering/ICC assumptions; select variance reduction (CUPED/stratification) and sequential monitoring approach with alpha spending. Detail ramp schedule, novelty and carryover controls, and interference detection. Define logging needed to reconstruct exposure and attribution, plus a difference-in-differences fallback if only partial randomization is possible. Explain stop/ship criteria and how to guard against Simpson’s paradox across tenants and roles.

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Dropbox•More Data Scientist•Dropbox Data Scientist•Dropbox Analytics & Experimentation•Data Scientist Analytics & Experimentation

Write your answer

Your first approved answer each day earns 20 XP.

Sign in to write your answer.
PracHub

Master your tech interviews with 9,000+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • AI Coding Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.