Facebook Login for Third-Party Apps: Diagnosing a Sudden Drop in Successful Sign-Ins

You are the product manager for Facebook Login, the option that lets people sign in to third-party apps and websites with their Facebook account. On a Monday morning, your dashboard shows that successful logins through it have dropped. Walk through how you would find the cause and what you would do about it. ```hint Split the count before you guess A count of successful logins can fall because fewer people try or because fewer attempts succeed. Decide which of those moved before you list causes, then cut the data along the dimensions that point to an owner. ``` ### Clarifying Questions - Is the metric a count of successful logins or a success rate, and how large is the drop compared with the same hours on recent Mondays? - Did the drop start abruptly, over the weekend or this morning, and has anything like it happened on other Mondays? - Does a successful login mean a completed sign-in to a partner app, rather than people signing in to Facebook itself? - Are the logging pipeline and the dashboard known to be healthy, or could the data be late or wrong? ### What a Strong Answer Covers - Confirming the metric, the size and timing of the drop, and ruling out data or logging problems - Breaking successful logins into a funnel, from activity on partner apps to a completed sign-in - Ordered hypotheses, internal and external, tested with cuts by partner app, platform, OS and SDK version, region and user type - Weekend-specific suspects such as deploys, OS updates and partner releases - Mitigation, communication with partner developers, and alerting that would catch a repeat sooner ### Follow-up Questions - The whole drop comes from one large partner whose site was down this morning. Is there anything left for you to do? - The success rate fell only on one mobile OS, right after an OS update. Whom do you involve, and what do you tell developers? - What alerting would have flagged this before you opened the dashboard?