Test a New Feature at Its Natural Product Placement
Company: LinkedIn
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Onsite
# Test a New Feature at Its Natural Product Placement
Design an experiment for a new feature. Explain why initially evaluating it in its natural product placement can be useful before increasing its exposure. Define what effect the first experiment estimates and how you would separately evaluate later changes in prominence or promotion.
### What a Strong Answer Covers
- A primary product outcome and a stable eligible-user denominator.
- Separation of the feature-availability effect from the effect of extra promotion.
- Exposure diagnostics and a distinction between low discoverability and weak value after use.
- A second experiment or factorial design for feature and placement effects.
```hint Define the policy being tested
A feature with a large promotional placement is a different treatment from the same feature in its ordinary location.
```
### Follow-up Questions
- Can low exposure make a natural-placement experiment inconclusive?
- Why is comparing users who discovered the feature with users who did not a biased causal comparison?
Overview: Design a feature launch experiment that separates natural-placement availability, discoverability, promotional exposure, and downstream product value.
Test a New Feature at Its Natural Product Placement
Design an experiment for a new feature. Explain why initially evaluating it in its natural product placement can be useful before increasing its exposure. Define what effect the first experiment estimates and how you would separately evaluate later changes in prominence or promotion.
What a Strong Answer Covers Guidance
A primary product outcome and a stable eligible-user denominator.
Separation of the feature-availability effect from the effect of extra promotion.
Exposure diagnostics and a distinction between low discoverability and weak value after use.
A second experiment or factorial design for feature and placement effects.
Follow-up Questions Guidance
Can low exposure make a natural-placement experiment inconclusive?
Why is comparing users who discovered the feature with users who did not a biased causal comparison?