Launch metrics plan for a new online learning site offering programming courses
Company: Google
Role: Product Manager
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Other
You have just joined an education website as its product manager. The site offers online courses that anyone can take, such as an introduction to Python programming, and it is about to launch. Which metrics would you track, which would you get ready to monitor from launch day, and why does each one matter?
```hint Match each metric to a moment
Some numbers tell you within hours whether the launch works at all; others only mean something after learners have had weeks to progress. Organize your metrics by the learner's journey and by when each becomes meaningful.
```
### Clarifying Questions
- How does the site make money: paid courses, a subscription, free courses with paid certificates, or nothing yet?
- Are the learners individuals building skills for themselves, or employers and schools that enroll groups?
- Is this a full public launch or a limited beta, and is there any pre-launch data such as a waitlist or a pilot cohort?
- Are the courses produced in-house, or can outside instructors publish them?
### What a Strong Answer Covers
- A goal for the site stated in terms of learner outcomes and the business model, leading to one primary success metric.
- Metrics across the learner journey, from first visit to enrollment, progress, completion and return, with the reason for each.
- Launch readiness: event tracking, technical health signals such as video and coding-exercise failures, and alerts in place before day one.
- Guardrails and quality signals such as refunds, course ratings and lesson-level drop-off, and how targets are set without a historical baseline.
### Follow-up Questions
- Two weeks after launch, sign-ups beat the plan but most learners never finish the first module. What would you look at first?
- How would you show that learners actually learned Python rather than just watching the videos?
- If the launch dashboard could hold only one number, which would you choose and why?
Overview: A product metrics question about an online learning site, with courses such as Python programming, that is about to launch. It tests choosing a primary success metric, building metrics across the learner journey, preparing tracking and alerts before launch, and setting guardrails without a historical baseline.