Select interest thresholds under skewness and cost

Read the full interview experience this question came from →

Quick Overview

This question evaluates a data scientist's competency in profit-oriented threshold selection using calibrated interest scores, uplift estimation, and decision-theoretic optimization under cost and weekly capacity constraints.

Select interest thresholds under skewness and cost

Company: Meta

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Onsite

You have a per-user interest_score in [0,1] for a new feature; the distribution appears right-skewed. You can afford to target at most K users per week. Propose a principled way to choose a threshold (e.g., 75th or 90th percentile) that maximizes incremental value under a per-contact cost c and expected benefit function b(score). Explain how you would: (a) estimate b(score) from historical data (e.g., isotonic or spline calibration), (b) compute the profit-optimal cutoff using expected lift × margin − c, (c) handle the case where the distribution is actually left-skewed, (d) prevent instability from sampling noise (e.g., Bayesian shrinkage or percentile CIs), and (e) set up an ongoing backtest to validate the threshold against alternatives. Provide formulas and a step-by-step selection algorithm.

Overview: This question evaluates a data scientist's competency in profit-oriented threshold selection using calibrated interest scores, uplift estimation, and decision-theoretic optimization under cost and weekly capacity constraints.

Read the full Meta Data Scientist interview experience this question came from

|Home/Analytics & Experimentation/Meta
Meta logo
Meta
Oct 13, 2025
hardData ScientistOnsiteAnalytics & Experimentation
6
0

Profit-Optimal Threshold Selection from an Interest Score

You have a per-user interest_score s ∈ [0, 1] for a new feature. The score distribution appears right-skewed. You can contact at most K users per week. Each contact has a known cost c. Let b(s) denote the expected incremental benefit (e.g., monetized uplift) if a user with score s is contacted.

Propose a principled method to choose a score threshold (e.g., 75th or 90th percentile) that maximizes incremental value under the cost constraint. Specifically:

  1. Define notation and the profit objective.
  2. Explain how to estimate b(s) from historical data, e.g., via isotonic calibration or splines.
  3. Show how to compute the profit-optimal cutoff using expected lift × margin − c, under a capacity constraint K.
  4. Explain how the approach adapts if the score distribution is left-skewed.
  5. Describe how to prevent instability due to sampling noise (e.g., Bayesian shrinkage or percentile CIs).
  6. Describe how to set up an ongoing backtest to validate the threshold against alternatives.

Provide formulas and a step-by-step selection algorithm.

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Meta•More Data Scientist•Meta Data Scientist•Meta Analytics & Experimentation•Data Scientist Analytics & Experimentation