Select interest thresholds under skewness and cost

Q: Select interest thresholds under skewness and cost

This question evaluates a data scientist's competency in profit-oriented threshold selection using calibrated interest scores, uplift estimation, and decision-theoretic optimization under cost and weekly capacity constraints.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Q: What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Meta.

Q: What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Question

Profit-Optimal Threshold Selection from an Interest Score

You have a per-user interest_score s ∈ [0, 1] for a new feature. The score distribution appears right-skewed. You can contact at most K users per week. Each contact has a known cost c. Let b(s) denote the expected incremental benefit (e.g., monetized uplift) if a user with score s is contacted.

Propose a principled method to choose a score threshold (e.g., 75th or 90th percentile) that maximizes incremental value under the cost constraint. Specifically:

Define notation and the profit objective.
Explain how to estimate b(s) from historical data, e.g., via isotonic calibration or splines.
Show how to compute the profit-optimal cutoff using expected lift × margin − c, under a capacity constraint K.
Explain how the approach adapts if the score distribution is left-skewed.
Describe how to prevent instability due to sampling noise (e.g., Bayesian shrinkage or percentile CIs).
Describe how to set up an ongoing backtest to validate the threshold against alternatives.

Provide formulas and a step-by-step selection algorithm.

Select interest thresholds under skewness and cost

Quick Overview

Select interest thresholds under skewness and cost

Profit-Optimal Threshold Selection from an Interest Score

Write your answer