How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Justify building a new feature with evidence

Quick Overview

This question evaluates a data scientist's product analytics, experimental design, and strategic decision-making skills, covering market sizing, competitive analysis, privacy-aware data and infrastructure assessment, risk identification, and A/B testing planning.

Case Prompt: 10-Minute Go/No-Go Recommendation for a New Feature

You are the data science lead supporting a large-scale consumer messaging product. Your VP asks you to justify building a new product feature from scratch and present a go/no-go recommendation in 10 minutes.

To make the discussion concrete, assume the candidate feature is "Smart Reply Suggestions" in 1:1 chats (the app shows 1–3 short, tap-to-send replies inferred from the last message). You may limit scope to English and mobile clients.

Address the following:

1) External Analysis

Market sizing: estimate TAM, SAM, and SOM, and outline assumptions and formulas.
Competitive landscape: identify at least 3 comparable offerings, what they do well/poorly, and likely defensibility.
Market signals: cite at least three signals you would gather and how you would quantify each (data sources, metrics, thresholds).

2) Internal Assessment

Capability fit: engineering/ML feasibility (e.g., on-device vs server inference), latency, cost, and quality.
Data availability and policies: what data is needed; constraints due to encryption, privacy, and user consent.
Risks: integrity, misuse/abuse, safety, localization, and regulatory risks.
MVP proposal: minimal viable launch surface (languages, platforms, entry points), and key dependencies (teams/systems).

3) Evidence & Decision Plan

Pre-launch work: offline evals, prototypes, research; how you would estimate impact before an A/B test.
Experiment plan: primary success metrics, guardrails, segments, and ramp strategy.
Milestones and thresholds: define what success looks like at 6 weeks and 6 months, and the exact decision thresholds to proceed, pivot, or kill.

Deliver a crisp recommendation (go/pilot/no-go) supported by the above.

Quick Overview

Case Prompt: 10-Minute Go/No-Go Recommendation for a New Feature

Address the following:

1) External Analysis

Market sizing: estimate TAM, SAM, and SOM, and outline assumptions and formulas.

Competitive landscape: identify at least 3 comparable offerings, what they do well/poorly, and likely defensibility.

Market signals: cite at least three signals you would gather and how you would quantify each (data sources, metrics, thresholds).

2) Internal Assessment

Capability fit: engineering/ML feasibility (e.g., on-device vs server inference), latency, cost, and quality.

Data availability and policies: what data is needed; constraints due to encryption, privacy, and user consent.

Risks: integrity, misuse/abuse, safety, localization, and regulatory risks.

MVP proposal: minimal viable launch surface (languages, platforms, entry points), and key dependencies (teams/systems).

3) Evidence & Decision Plan

Pre-launch work: offline evals, prototypes, research; how you would estimate impact before an A/B test.

Experiment plan: primary success metrics, guardrails, segments, and ramp strategy.

Milestones and thresholds: define what success looks like at 6 weeks and 6 months, and the exact decision thresholds to proceed, pivot, or kill.

Deliver a crisp recommendation (go/pilot/no-go) supported by the above.

Justify building a new feature with evidence

Quick Overview

Case Prompt: 10-Minute Go/No-Go Recommendation for a New Feature

1) External Analysis

2) Internal Assessment

3) Evidence & Decision Plan

Solution

Submit Your Answer to Earn 20XP

Justify building a new feature with evidence

Quick Overview

Case Prompt: 10-Minute Go/No-Go Recommendation for a New Feature

1) External Analysis

2) Internal Assessment

3) Evidence & Decision Plan

Solution

Submit Your Answer to Earn 20XP