Quality and frequency control for push notifications
Quick Overview
Evaluates push-notification quality and frequency control for a consumer app with fatigue risks. Strong answers define incremental metrics, fatigue guardrails, overload mitigation, and a per-user cap algorithm that balances opens with opt-outs and uninstalls.
Quality and frequency control for push notifications
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
Scenario: Pushes drive re‑engagement but risk fatigue. Define quality metrics, tackle overload complaints, and build a per‑user cap algorithm that maximises opens without driving uninstalls.
Question 1: What metrics gauge notification quality? (Hint: open‑rate, downstream sessions, unsubscribe rate)
Question 2: Users complain about too many pushes—how improve? (Hint: frequency throttling, personalised content)
Question 3: How to set a daily push limit per user? (Hint: multi‑armed bandit, satisfaction model)
Quick Answer: Evaluates push-notification quality and frequency control for a consumer app with fatigue risks. Strong answers define incremental metrics, fatigue guardrails, overload mitigation, and a per-user cap algorithm that balances opens with opt-outs and uninstalls.
Push Notifications: Quality, Overload Mitigation, and Per-user Caps
You are a data scientist working on push notifications for a consumer app. Pushes can increase re-engagement but can also cause fatigue, opt-outs, and uninstalls.
Assume you have notification logs, app session logs, opt-out and uninstall events, user features, and content metadata.
Constraints & Assumptions
Measure incremental impact, not only raw open rate.
Include negative outcomes such as opt-outs, uninstalls, complaints, and fatigue.
Design a daily per-user cap that can vary by user and context.
Include delivery and content-quality diagnostics.
Clarifying Questions to Ask Guidance
What actions are notifications trying to drive?
Are notifications transactional, social, promotional, or recommendation-based?
Are there legal, timezone, quiet-hour, or frequency constraints?
Is a randomized no-push holdout available?
Part 1 - Notification Quality Metrics
What metrics would you use to gauge notification quality?
What This Part Should Cover Guidance
Include delivery rate, open rate, unique open rate, downstream sessions, conversion, time to next session, and holdout-based incremental lift.
Include opt-out rate, uninstall rate, complaint rate, mute rate, and notification fatigue metrics.
Segment by notification type, user segment, timezone, device, and content.
Part 2 - Overload Mitigation
Users complain about too many pushes. How would you improve the system?
What This Part Should Cover Guidance
Diagnose volume, timing, relevance, redundancy, and low-quality content.
Use ranking, deduplication, batching, quiet hours, cooldowns, personalization, and preference controls.
Test reductions or smarter prioritization with user-level experiments.
Part 3 - Per-user Cap Algorithm
Design a daily per-user cap that maximizes opens while avoiding uninstalls or opt-outs.
What This Part Should Cover Guidance
Estimate expected incremental value and expected harm for each notification.
Set caps by user tolerance, recent engagement, history, channel, and content priority.
Use constrained optimization or bandits with safety guardrails.
Monitor long-term retention and recalibrate.
Follow-up Questions Guidance
How would you estimate notification fatigue?
What if open rate increases but opt-outs also increase?
How would you handle critical transactional notifications?