Evaluate channels and allocate budget
Company: Upstart
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Technical Screen
You are given a marketing-by-channel dataset with daily aggregates: channel, spend, visits, form_starts, form_completes, loan_funded, revenue. Tasks: (1) build a funnel with stage-to-stage conversion rates and CAC at each meaningful milestone; (2) compute ROAS and payback under both last-click and 7-day first-touch attribution (describe how you would re-attribute if you only had raw click/impression logs); (3) recommend next-month budget shifts by channel using a simple response curve (e.g., log or Hill function) and estimate marginal ROAS; (4) design an incrementality test (geo or time-based holdout) to validate your recommendations, including unit of randomization, sample size, contamination risks, and success criteria; (5) flag channels that look effective but likely non-incremental due to retargeting or brand spillovers, and propose diagnostics to detect this. Provide concrete formulas, any assumptions you must make, and how you would communicate the recommendation with risk bounds.
Quick Answer: This question evaluates competency in marketing analytics and experimentation for a Data Scientist role, covering funnel analysis, conversion and CAC metrics, attribution modeling, response-curve-based budget allocation, and incremental test design using channel-level spend and outcome data.