Netflix Engagement Diagnosis: A Million Paying Accounts With No Logins

Quick Overview

A metric-diagnosis question for a Netflix product team: one million subscribers with active subscriptions are not logging in. It tests how a candidate defines the problem, sizes it against a baseline, verifies the data, segments by device, region and billing channel, and orders hypotheses before proposing next steps.

Netflix Engagement Diagnosis: A Million Paying Accounts With No Logins

Company: Google

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

You are on the Netflix product team. One million subscribers whose subscriptions are active are not logging in to the service. How would you investigate what is going on? ```hint Size it before you chase it Before listing causes, decide what the number is measured against: how big a million is relative to the subscriber base, and what the same figure looked like in earlier periods. ``` ### Clarifying Questions - Does "not logging in" mean these members have no viewing sessions at all, or that they have been signed out and cannot get back in? - Over what period have they been absent, and is this a sudden jump or a slow build-up compared with before? - Do we already know whether it is concentrated in a device type, app version, region or plan? - Has anyone confirmed that the session and login data behind this number is complete and accurate? ### What a Strong Answer Covers - A precise definition of the problem, its time window and baseline, and a judgment of how serious the number is - Verification of the data and instrumentation before treating the signal as real - A structured set of possible causes, internal and external, technical and behavioral, checked in a sensible order with the evidence each would leave - The segmentation cuts that would isolate the cause and the data or tests that would confirm it - Next steps once a cause is found, including the effect on retention and how the issue would be monitored ### Follow-up Questions - Suppose the absent members are spread evenly across devices, regions and signup dates. Where would you look next? - If most of them have simply stopped watching but still pay, what would you do, and what are the risks of contacting them? - What alerting would you put in place so the team spots a pattern like this sooner?

Overview: A metric-diagnosis question for a Netflix product team: one million subscribers with active subscriptions are not logging in. It tests how a candidate defines the problem, sizes it against a baseline, verifies the data, segments by device, region and billing channel, and orders hypotheses before proposing next steps.

Google logo
Google
Apr 14, 2024
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

You are on the Netflix product team. One million subscribers whose subscriptions are active are not logging in to the service. How would you investigate what is going on?

Clarifying Questions Guidance

  • Does "not logging in" mean these members have no viewing sessions at all, or that they have been signed out and cannot get back in?
  • Over what period have they been absent, and is this a sudden jump or a slow build-up compared with before?
  • Do we already know whether it is concentrated in a device type, app version, region or plan?
  • Has anyone confirmed that the session and login data behind this number is complete and accurate?

What a Strong Answer Covers Guidance

  • A precise definition of the problem, its time window and baseline, and a judgment of how serious the number is
  • Verification of the data and instrumentation before treating the signal as real
  • A structured set of possible causes, internal and external, technical and behavioral, checked in a sensible order with the evidence each would leave
  • The segmentation cuts that would isolate the cause and the data or tests that would confirm it
  • Next steps once a cause is found, including the effect on retention and how the issue would be monitored

Follow-up Questions Guidance

  • Suppose the absent members are spread evenly across devices, regions and signup dates. Where would you look next?
  • If most of them have simply stopped watching but still pay, what would you do, and what are the risks of contacting them?
  • What alerting would you put in place so the team spots a pattern like this sooner?