How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Design Experiment to Measure Shopping Feature Impact

Last updated: Mar 29, 2026

Quick Overview

This question evaluates experimental design and causal inference skills for product analytics, focusing on metric selection, handling interference and non-compliance, and analysis of heavy-tailed and delayed conversion outcomes within the Analytics & Experimentation domain.

Experiment Design: Measuring Instagram Shopping's Impact

Scenario

Instagram is launching an in-app Shopping feature (e.g., product tags, shop surfaces, in-app checkout). Leadership wants to quantify its incremental impact on user engagement and revenue.

Task

Design an experiment to measure the impact of the Shopping feature on engagement and revenue.

Address:

What experimental design(s) would you use and why (e.g., A/B vs. geo hold-out; rollout strategy)?
Which metrics (north-star and guardrails) would you monitor, with success criteria?
How would you mitigate and measure selection bias and network effects (spillovers across the social graph)?

Considerations

Users can influence each other via follows/likes/shares (potential interference).
Some users may be eligible but never use the feature (non-compliance).
Rare, heavy-tailed outcomes (purchases/GMV) and delayed conversions.
Hints: A/B vs. hold-out, north-star metrics, guard-rails, diff-in-diff.

Solution

Show

Submit Your Answer

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Meta•More Data Scientist•Meta Data Scientist•Meta Analytics & Experimentation•Data Scientist Analytics & Experimentation