Evaluates analytics design for a LinkedIn online presence indicator in messaging. Strong answers estimate reach and lift, use historical analogs, monitor engagement, responsiveness, privacy, trust, and network effects.
Scenario: LinkedIn Messaging plans to show an online indicator next to connections.
Question 1: Which user cohorts and behaviors are likely to be affected by this feature?
Question 2: Propose a method to estimate the number of reachable users and expected lift pre‑launch.
Question 3: How would you leverage analogous historical features to calibrate lift assumptions?
Question 4: What key metrics and guardrails would you monitor during rollout?
Quick Answer: Evaluates analytics design for a LinkedIn online presence indicator in messaging. Strong answers estimate reach and lift, use historical analogs, monitor engagement, responsiveness, privacy, trust, and network effects.
LinkedIn plans to display an online presence indicator, such as a green dot, next to first-degree connections across key messaging surfaces. The goal is to increase messaging engagement and responsiveness without harming user trust or privacy.
Assume the feature can be toggled at the user level, and logs include session activity, presence state, and messaging events across web and mobile.
Constraints & Assumptions
Consider both senders who see presence and recipients whose presence is shown.
Include privacy, trust, and notification fatigue guardrails.
Estimate reach and lift before launch using historical behavior and analogous features.
Monitor network and pair-level effects.
Clarifying Questions to Ask Guidance
On which surfaces will the indicator appear?
Can users opt out or control visibility?
What counts as online, recently active, or reachable?
Is success more about message sends, replies, response time, or conversation quality?
Part 1 - Affected Cohorts and Behaviors
Which cohorts and behaviors are likely to be affected?
What This Part Should Cover Guidance
Include high-activity members, heavy messagers, recruiters, job seekers, close connection pairs, mobile users, and time-zone-aligned networks.
Identify behaviors such as message starts, reply rate, response latency, conversation depth, session length, and connection interactions.
Consider recipient-side pressure or privacy concerns.
Part 2 - Pre-launch Reach and Lift Estimate
How would you estimate reachable users and expected lift before launch?
What This Part Should Cover Guidance
Estimate eligible users, sessions with visible first-degree connections, overlap online windows, and presence impressions.
Use historical messaging propensity and exposure opportunities.
Apply funnel assumptions from presence impression to send, reply, and conversation.
Provide ranges and sensitivity analysis.
Part 3 - Historical Calibration and Rollout Metrics
How would you use analogous features and what metrics would you monitor during rollout?
What This Part Should Cover Guidance
Calibrate from features such as read receipts, active status, typing indicators, notification badges, or prior messaging UI nudges.
Monitor message sends, replies, response time, conversation depth, retention, blocks, reports, privacy setting changes, opt-outs, and support contacts.
Use A/B testing with sender/recipient interference considerations.
Follow-up Questions Guidance
What if messages increase but blocks and reports also rise?
How would you handle users who do not want their online status shown?
How would you estimate spillovers when treatment users message control users?