You have a per-user interest_score in [0,1] for a new feature; the distribution appears right-skewed. You can afford to target at most K users per week. Propose a principled way to choose a threshold (e.g., 75th or 90th percentile) that maximizes incremental value under a per-contact cost c and expected benefit function b(score). Explain how you would: (a) estimate b(score) from historical data (e.g., isotonic or spline calibration), (b) compute the profit-optimal cutoff using expected lift × margin − c, (c) handle the case where the distribution is actually left-skewed, (d) prevent instability from sampling noise (e.g., Bayesian shrinkage or percentile CIs), and (e) set up an ongoing backtest to validate the threshold against alternatives. Provide formulas and a step-by-step selection algorithm.

This question evaluates a data scientist's competency in profit-oriented threshold selection using calibrated interest scores, uplift estimation, and decision-theoretic optimization under cost and weekly capacity constraints.

How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Select interest thresholds under skewness and cost

Profit-Optimal Threshold Selection from an Interest Score

You have a per-user interest_score s ∈ [0, 1] for a new feature. The score distribution appears right-skewed. You can contact at most K users per week. Each contact has a known cost c. Let b(s) denote the expected incremental benefit (e.g., monetized uplift) if a user with score s is contacted.

Propose a principled method to choose a score threshold (e.g., 75th or 90th percentile) that maximizes incremental value under the cost constraint. Specifically:

Define notation and the profit objective.
Explain how to estimate b(s) from historical data, e.g., via isotonic calibration or splines.
Show how to compute the profit-optimal cutoff using expected lift × margin − c, under a capacity constraint K.
Explain how the approach adapts if the score distribution is left-skewed.
Describe how to prevent instability due to sampling noise (e.g., Bayesian shrinkage or percentile CIs).
Describe how to set up an ongoing backtest to validate the threshold against alternatives.

Provide formulas and a step-by-step selection algorithm.

Profit-Optimal Threshold Selection from an Interest Score

Propose a principled method to choose a score threshold (e.g., 75th or 90th percentile) that maximizes incremental value under the cost constraint. Specifically:

Define notation and the profit objective.

Explain how to estimate b(s) from historical data, e.g., via isotonic calibration or splines.

Show how to compute the profit-optimal cutoff using expected lift × margin − c, under a capacity constraint K.

Explain how the approach adapts if the score distribution is left-skewed.

Describe how to prevent instability due to sampling noise (e.g., Bayesian shrinkage or percentile CIs).

Describe how to set up an ongoing backtest to validate the threshold against alternatives.

Provide formulas and a step-by-step selection algorithm.

Select interest thresholds under skewness and cost

Quick Overview

Profit-Optimal Threshold Selection from an Interest Score

Solution

Submit Your Answer to Earn 20XP

Select interest thresholds under skewness and cost

Quick Overview

Profit-Optimal Threshold Selection from an Interest Score

Solution

Submit Your Answer to Earn 20XP