On-call behavioral: judging severity, escalating, and tracing an issue to a recent change

These two prompts come from a senior engineering loop for a team that keeps a large live service healthy. Both are behavioral: answer from your own experience with a specific incident, but expect the interviewer to probe the technical reasoning behind each decision you describe. ### Clarifying Questions - Should the example come from a customer-facing production service, or is an internal platform acceptable? - How much technical detail would you like about the system, compared with the coordination? ### Part 1 — Owning an incident on a customer-facing service Describe an issue you investigated and resolved while on call for a large, customer-facing service. How did you judge how severe it was, and how did you bring in and coordinate the engineering teams you needed? Follow-up: did you involve the owning engineering team, or work through it alone, and why? ```hint Severity is a judgment with inputs Name the signals you used to decide how bad it was, and the moment you decided it needed more people. ``` #### What This Part Should Cover - How severity was assessed from customer impact, not only from alerts - The escalation decision and the reasoning for working alone or engaging others - Coordination: roles, communication cadence, decision making - Mitigation, resolution, and the follow-up that prevented a repeat ### Part 2 — Tying runtime signals to a recent change Describe a case where runtime data pointed to a problem in a system and you had to link it to a recent deployment or feature-flag change to triage it fast. What specifically did you do to shorten the time to resolution? ```hint Line up two timelines Think about how you aligned when the symptom started with when things changed, and what evidence turned a suspicion into a decision. ``` #### Clarifying Questions for this Part - Should the example involve a change made by your own team, or one coordinated with an external team? #### What This Part Should Cover - The specific signals and how they were correlated with deployment or flag history - The decision to mitigate (roll back, turn the flag off) before the full root cause was known - Concrete steps that reduced time to resolution, during and after the incident - Coordination with whoever owned the change ### What a Strong Answer Covers - Specific, first-person incident stories with a timeline and measurable impact - Mitigation first, diagnosis second - Clear escalation criteria and calm coordination - Systemic improvements afterward (alerts, tooling, process), not only the fix - Blameless framing of other teams and individuals ### Follow-up Questions - What would you do differently if the same incident happened tomorrow? - How do you decide between rolling back and rolling forward with a fix? - How did you keep customers or customer-facing teams informed during the incident? - What changed in your team's deployment or flag process as a result?