Identify non-table data for feature demand

Quick Overview

The question evaluates a data scientist's competency in triangulating product demand from non-table sources, defining and de-biasing signals, specifying metrics and guardrails, and designing evidence-driven experiments within the Analytics & Experimentation domain for a Data Scientist role.

Identify non-table data for feature demand

Company: Meta

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

You must decide whether to build a "group call" feature without relying solely on existing product tables. What specific non-table data sources would you use, what signal would you extract from each, and how would you quantify and de-bias them? Discuss at least: (a) qualitative channels (customer interviews, support tickets, sales call transcripts, app store reviews, community/social listening), (b) market/competitive intel (pricing pages, public docs, analyst reports), (c) operational data (infra costs, incident logs, capacity constraints), and (d) user research/UX telemetry (task success, time-on-task, heatmaps). Then design an evidence-driven experiment plan to validate demand: propose a fake-door test (exposed to X% of eligible users), a waitlist, and a limited beta. For each, specify the hypothesis, eligibility, primary success metric (e.g., new weekly active group-call hosts), guardrails (e.g., crash rate, setup failure rate, CSAT), and minimum detectable effect with rough sample-size math and timeline using "today" = 2025-09-01. Explain how you will segment results (e.g., team size, geography, plan tier), control for novelty effects, and ensure ethics (avoid dark patterns).

Quick Answer: The question evaluates a data scientist's competency in triangulating product demand from non-table sources, defining and de-biasing signals, specifying metrics and guardrails, and designing evidence-driven experiments within the Analytics & Experimentation domain for a Data Scientist role.

|Home/Analytics & Experimentation/Meta
Meta logo
Meta
Oct 13, 2025, 9:49 PM
hardData ScientistTechnical ScreenAnalytics & Experimentation
3
0

Evaluate Demand for a New "Group Call" Feature Using Non-Table Data and Experiments

Context

You are a data scientist evaluating whether to invest in a new "group call" feature for a large consumer communications product. Assume you cannot rely primarily on existing product usage tables (e.g., call logs, feature flags). Instead, you must triangulate demand and feasibility from non-table sources and then design experiments to validate demand.

Define "eligible users" as users who have access to calling in the app and belong to at least one group/thread with ≥3 members.

Task

  1. Identify specific non-table data sources you would use and for each, describe:
    • The signal you would extract.
    • How you would quantify it (metrics/models).
    • How you would de-bias it.
    Discuss at least the following categories: (a) Qualitative channels: customer interviews, support tickets, sales call transcripts, app store reviews, community/social listening. (b) Market/competitive intel: pricing pages, public docs, analyst reports. (c) Operational data: infra costs, incident logs, capacity constraints. (d) User research/UX telemetry: task success, time-on-task, heatmaps.
  2. Design an evidence-driven experiment plan to validate demand, including:
    • A fake-door test (exposed to X% of eligible users), a waitlist, and a limited beta.
    • For each, specify: hypothesis, eligibility, primary success metric (e.g., new weekly active group-call hosts), guardrails (e.g., crash rate, setup failure rate, CSAT), and minimum detectable effect (MDE) with rough sample size math.
    • Provide a realistic timeline using "today" = 2025-09-01.
    • Explain how you will segment results (e.g., team size, geography, plan tier), control for novelty effects, and ensure ethics (avoid dark patterns).
Loading comments...