North Star Metrics & Experiment Design

Q: North Star Metrics & Experiment Design

This question evaluates a product manager's competency in product sense, metric definition, analytics instrumentation, and controlled experiment design by requiring a North Star metric, supporting leading indicators with SQL-level data considerations, and a randomized experiment to improve the metric.

Q: How do I approach Product / Decision Making interview questions?

Product / Decision Making questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master product / decision making interviews.

Question

Product Sense: Define a North Star Metric and Experiment (Microsoft)

Context

Choose any Microsoft product you know (e.g., Teams, Outlook, Azure AI).
Assume you have access to anonymized, event-level telemetry (users, teams/tenants, events, subscriptions) and can run controlled experiments.

Tasks

Pick one Microsoft product and briefly state the customer job-to-be-done.
Define a single north star metric (NSM):
- Name it and give a precise, measurable definition (window, qualifying actions/thresholds).
- Explain why it reflects long-term customer value.
List two leading indicators that support/drive the NSM:
- Define each metric clearly.
- Specify the SQL-level data you would track (tables/columns) and, if helpful, short SQL sketches.
Describe one experiment to improve the NSM:
- Hypothesis and treatment.
- Unit of randomization and eligibility.
- Success criteria: primary metric, supporting/guardrail metrics, sample size/MDE assumptions, and duration.

North Star Metrics & Experiment Design

Product Sense: Define a North Star Metric and Experiment (Microsoft)

Context

Tasks

Solution

Comments (0)