Define and validate product metrics

Q: Define and validate product metrics

This is a Analytics & Experimentation interview question from Meta for Data Engineer roles. View the full question and solution on PracHub.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

End-to-End Analytics Design for a New Product Feature

Context: You are the data engineer partnering with product, engineering, and data science to launch a new user-facing feature in a large-scale consumer app. You must define success, design instrumentation, produce reliable KPIs, and operate the metrics in both streaming and batch.

Tasks:

Define the core business metrics:
- North-star metric(s) that represent durable user/business value.
- Guardrail metrics to protect reliability, performance, quality, and adjacent business outcomes.
Instrumentation and tracking plan:
- Specify events, schemas, identifiers, and when each event fires.
- Include experiment fields, versions, deduplication, and sampling choices.
From raw events to KPIs:
- Describe transformations/modeling steps to derive daily/weekly KPIs from raw logs.
Metric quality and operations:
- How to sanity-check correctness pre/post launch.
- How to set thresholds and detect/investigate anomalies.
Batch vs. streaming views:
- How definitions and numbers can differ; watermarks, late data, deduplication, approximations.
Communication:
- How to document caveats, data latency, and metric maturity to stakeholders.

Define and validate product metrics

End-to-End Analytics Design for a New Product Feature

Solution (Locked)

Comments (0)