Design visualizations for streaming metrics
Design a Monitoring and Diagnosis Visualization for a Video-Streaming Metric
Context
You are building an observability dashboard for a global consumer video product. Choose one critical streaming quality metric and design how you would visualize it to monitor health and diagnose issues.
Assume you have standard playback events (play_attempt, start, first_frame, rebuffer, stop), client metadata (device, OS, app version, network), and infrastructure metadata (CDN/POP, ISP, region). The dashboard should support both real-time monitoring and retrospective diagnosis.
Task
Pick one critical metric (e.g., start-failure rate, rebuffering ratio, time-to-first-frame, hours watched per user) and specify:
-
Primary time-series view
-
Time granularity (near real-time vs. daily); zoom behavior
-
Smoothing and seasonality handling
-
Anomaly detection and bands (baseline and confidence/prediction intervals)
-
Key breakdowns for diagnosis
-
Device/OS/app version
-
Network type/ISP
-
Geography (region/country/city)
-
Cohorts (new vs. returning, app release, experiment group)
-
Diagnostic visuals
-
Percentile bands where applicable (e.g., p50/p90/p99)
-
Geographic heatmap
-
Funnel from play attempt → start → watch ≥ X minutes
-
Interaction patterns
-
How users drill down, pivot dimensions, and compare baselines
-
Guardrail metrics
-
Which additional metrics you would co-display to prevent blind spots
Constraints & Assumptions
-
Preserve the scope, facts, inputs, and requested outputs from the prompt above.
-
If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
-
Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.
Clarifying Questions to Ask
-
Clarify the business objective, unit of analysis, time window, exposure definition, and primary metric.
-
State assumptions about instrumentation, randomization, sample size, and data quality.
-
Separate descriptive analysis from causal claims.
What a Strong Answer Covers
-
A metric framework with primary, guardrail, and diagnostic metrics.
-
A credible analysis or experiment design with clear assumptions and bias checks.
-
SQL/statistical logic for segmentation, variance, confidence, and data validation where relevant.
-
An actionable recommendation that explains trade-offs and next steps.
Follow-up Questions
-
What sanity checks would you run before trusting the result?
-
How would you handle novelty effects, seasonality, or selection bias?
-
What decision would you make if metrics disagree?