##### Question
Pick a product you consider poorly designed; explain why and how you would improve it, including trade-offs.
Tell me about a time you went above and beyond to deliver value to users and what the result was.
Describe a disagreement you resolved by using metrics.
Quick Answer: This question evaluates a product manager's behavioral and leadership competencies, specifically product sense, user-impact orientation, metrics-driven decision-making, trade-off analysis, and the ability to resolve disagreements through data.
Solution
# How to Approach These Prompts
- Use STAR+L: Situation, Task, Actions, Result, Learning.
- Lead with the user and measurable impact. Quantify outcomes (absolute and relative changes).
- Show PM thinking: define success metrics, guardrails, trade-offs, and an experiment/validation plan.
---
## 1) Product Critique: Poorly Designed Product → Improvements and Trade-offs
Framework to use
- Users and JTBD: Who are the users? What job are they trying to get done?
- Evidence of pain: What proves it’s broken (errors, drop-offs, support tickets, time-on-task)?
- Root causes: UX, information architecture, tech constraints, incentives, standards.
- Options and trade-offs: Enumerate alternatives, costs, risks, and complexity.
- Recommendation: Prioritized plan with success metrics and a validation approach.
Model answer (example): Calendar time zone handling
- Users & JTBD: Knowledge workers scheduling cross-time-zone meetings. The job: coordinate time reliably without confusion.
- What’s broken (symptoms):
- Events shift unexpectedly when traveling (DST, “floating” events vs fixed time zones).
- Participants see times in different zones, leading to missed/late meetings.
- All-day events slip a day due to DST or time zone interpretation.
- Evidence (how I’d quantify):
- 7–10% of multi-time-zone events get edited within 24 hours with only time changes (proxy for “oops” corrections).
- 12% of calendar-related support tickets mention time zones/DST.
- Qual research: users report low confidence; time-to-schedule >2 minutes when >2 time zones are involved.
- Root causes:
- Ambiguous mental model: “floating” vs “locked to a zone.”
- Poor affordance for participant local times; confusing DST boundaries.
- ICS standards and OS time settings create constraints.
- Improvements (prioritized):
1) Clear “Time zone mode” chips in event editor: “Lock to Los Angeles (PDT)” vs “Stay at 9:00 local time wherever I am.” Defaults based on event type.
2) Multi-zone composer: show participant local times side-by-side and suggest best overlap windows.
3) Safer defaults for recurring events across DST: nudge owners with recommended shifts and one-click accept.
4) All-day events: explicit toggle “all-day in creator’s zone vs participant’s local day,” with preview.
5) Lightweight education: inline hints and first-time tooltips; undo support.
- Success metrics:
- Scheduling correction rate for multi-zone events: -30% within 6 weeks.
- Time-to-schedule across 3+ time zones: -25%.
- Support tickets on time zones: -40% per 10k events.
- Satisfaction (CSAT) on scheduling flow: +10 points.
- Validation plan:
- Wizard-of-Oz prototype with 15–20 target users; then A/B test the new composer on 10% traffic.
- Guardrails: no increase in event creation time for single-zone events; error rate (failed invites) unchanged.
- Trade-offs:
- Complexity vs clarity: adding chips/options risks UI clutter; mitigate via progressive disclosure and smart defaults.
- Standard compatibility: ICS/CalDAV may limit expressive models; must ensure backward-compatible serialization.
- Education cost: slight learning curve; offset with just-in-time tooltips.
Why this works
- Anchors on users and evidence, shows root-cause thinking, proposes specific changes, defines metrics and guardrails, and discusses trade-offs realistically.
---
## 2) Above and Beyond to Deliver Value to Users
Framework to use (STAR+L)
- Situation/Task: What was broken and why it mattered.
- Actions: What you personally did beyond the usual remit (initiative, scrappiness, cross-functional leadership).
- Result: Quantified impact (and how you measured it).
- Learning: What you’d repeat or change.
Model story (example): Accelerated developer activation for an API platform
- Situation/Task: Developer activation (first successful API call within 24 hours) was 55%. Docs were long; sample data was hard to use; new devs churned.
- Actions (above and beyond):
- Shadowed 12 developers live; created a journey map of failure points.
- Built a 5-minute interactive “Quickstart” with copy/paste examples in 3 languages, a mock API key, and sandbox sample data.
- Partnered with DevRel to add a “Run in Postman” button and a 90-second tutorial video; I personally handled forum/Slack triage for the launch week to capture and fix issues in near real-time.
- Wrote the instrumentation plan (activation funnel, time-to-first-success, doc search queries) and built a Looker dashboard so Eng/Docs could see impact daily.
- Result:
- Activation 24h: 55% → 74% (+19 points, +34% relative) in 3 weeks.
- Median time-to-first-success: 2.5 days → 6 hours.
- Support tickets per 100 new devs: -28%.
- NPS for onboarding: +14 points.
- Learning:
- Instrumentation before building sped iteration.
- A tiny sandbox plus copy/paste samples delivered outsized value vs expanding docs.
- Real-time support during launch closed the loop quickly and built goodwill.
Why this works
- Demonstrates user obsession, scrappy execution, and quantified impact. Shows you led across Docs/DevRel/Eng and did unusual work (live shadows, real-time triage) to unlock value fast.
---
## 3) Disagreement Resolved Using Metrics
Framework to use
- Frame the decision as a hypothesis with a success metric and guardrails.
- Get shared definitions and a decision rubric upfront (e.g., expected value).
- Collect data via experiment or rigorous analysis; decide based on pre-agreed criteria.
Model story (example): SMS verification in signup (growth vs. fraud)
- Situation: Growth team wanted to remove SMS verification to improve signup conversion; Risk team wanted to keep it to reduce fraud. We were deadlocked.
- Alignment on metrics and rubric:
- North Star: Net economic value per month (revenue from activated users minus fraud losses and variable costs).
- Guardrails: Complaint rate, time-to-signup, support tickets, and region-specific pass rates.
- Decision rule: Choose the variant with higher expected value (EV) while meeting guardrails.
- Experiment design:
- A/B/C test for 4 weeks across comparable geos/devices: A = current (SMS), B = no SMS, C = risk-adjusted SMS (only for high-risk signals).
- Instrumentation: conversions, activation, 60-day retention, fraud incidence, SMS cost, support tickets.
- Results (illustrative numbers):
- A (current): 100k signups → 40% activate = 40k; LTV = $10 → $400k; fraud loss = $80k; SMS cost = $2k → EV = $400k − $80k − $2k = $318k.
- B (no SMS): 100k → 42% = 42k; LTV = $10 → $420k; fraud loss = $140k; cost ≈ $0 → EV = $420k − $140k = $280k.
- C (risk-adjusted SMS): 100k → 38.5% = 38.5k; LTV = $10 → $385k; fraud loss = $32k; SMS cost = $1.1k → EV = $385k − $32k − $1.1k = $351.9k.
- Guardrails: Complaint rate and time-to-signup unchanged for C; worse for A; better but risky for B.
- Decision: Ship C (risk-adjusted verification). Net EV +$33.9k/month vs current; fraud down 60%; conversions only -1.5 points vs A.
- Follow-up: Rolled out per-geo risk thresholds; 8-week lookback confirmed retention unaffected and fraud savings persisted.
Why this works
- You aligned stakeholders on a decision rubric, measured the right outcomes, included guardrails, and chose the option with the highest expected value.
Note: Expected value framing
- EV = (Activated users × LTV) − Fraud losses − Variable costs.
- Pre-commit to this formula and the guardrails to avoid post-hoc debate.
---
## Common Pitfalls to Avoid
- Vague claims without metrics or user evidence.
- Ignoring trade-offs (cost, complexity, privacy, standards, localization).
- Shipping without guardrails or a rollback plan.
- Confusing outputs (features shipped) with outcomes (user/business impact).
## Quick Prep Checklist
- For product critiques: Know your users, JTBD, pain evidence, and 2–3 crisp improvements with trade-offs and a measurement plan.
- For behavioral stories: Prepare 2–3 STAR+L stories with numbers (activation, conversion, retention, NPS, revenue, cost).
- For conflict resolution: Practice one story where you established success metrics/guardrails upfront and made a metrics-backed decision.