Describe taking initiative and acting first

# How to Answer Well (STAR(L) + Engineering Focus) Use STAR(L): - Situation: Brief, specific context and why it mattered. - Task: Your responsibility/goal and constraints (SLOs, timelines, risks). - Action: Options you considered; why you picked one; immediate, concrete steps. - Result: Quantified impact with before/after metrics. - Learning: What you took away and institutionalized. What to quantify (pick 2–3): latency (p95/p99), error rate, throughput/QPS, incidents/MTTR, cost, developer hours, conversion/revenue proxy. Simple impact formulas: - Latency improvement (%) = (old − new) / old × 100 - Error budget (minutes/month) for 99.9% SLO ≈ 43.2 minutes - Dev-hours saved = frequency × time per event # Option Evaluation Framework (keep it lightweight) Consider: speed to implement, risk/blast radius, reversibility, cost, and long-term maintainability. Prefer reversible, low-blast-radius steps first (instrumentation, flags, canaries) before structural changes. # Sample Answer (Engineering, Proactive Performance Fix) Situation - A week before a major marketing push, our high-throughput API’s order placement p99 latency crept from ~120 ms to ~300 ms during peak simulations. Our SLO target was p99 ≤ 150 ms. If unchanged, we risked timeouts and failed orders under the upcoming traffic spike. Task - As the on-call backend engineer, I aimed to bring p99 < 150 ms, keep error rate < 0.1%, and avoid risky, irreversible changes before the event. Options Considered 1) Database changes: add a composite index and reduce lock contention via a targeted migration. Pros: big win; Cons: operational risk. 2) Reduce DB load: cache hot reads (balances/positions) to free write capacity. Pros: low risk, reversible; Cons: staleness to manage. 3) App-level optimizations: swap slow JSON serializer, eliminate redundant validation. Pros: safe, fast; Cons: moderate gains. 4) Backpressure: rate limit low-priority endpoints to protect critical paths. Pros: high safety; Cons: UX trade-offs. 5) Larger redesign (async write-ahead queue): Pros: scalable; Cons: too risky pre-event. I chose a layered approach: start with observability, then safe app-level wins and caching, then a narrowly scoped index migration with a canary, plus backpressure as a guardrail. Immediate Actions (first 48 hours) - Instrumentation: Added tracing spans around DB writes, serialization, and RPCs; built a Grafana dashboard for p50/p95/p99, error rate, DB CPU, lock wait time. - Safe optimizations: Switched to a faster JSON library and removed duplicate schema validations in the hot path (two small PRs behind flags). - Cache: Implemented a read-through Redis cache for user balance reads with a 200 ms TTL and explicit invalidation on writes; added hit/miss metrics. - Guardrails: Enabled circuit breaker for non-critical endpoints and wrote a rate-limit rule to ensure order placement remained protected. - Scoped DB index: Created a shadow table and built the composite index off-peak; canaried to 5% of traffic, monitored lock waits and latency, then rolled to 50% and 100% with a rollback plan. - Validation: Ran load tests (2× projected traffic) in staging using production-like data, verified dashboards and alarm thresholds. Impact (measured) - p99 latency: 320 ms → 140 ms (56% improvement) - Error rate: 0.7% → 0.08% - DB CPU: 85% → 55%; lock waits down 70% - Throughput: sustained 2.1× previous peak in load tests - Cost/operational: avoided an incident during the campaign; reduced cloud DB cost by ~18% month-over-month Pushback/Uncertainty - DBA concern about the index migration risk: Mitigated via shadow index, canary rollout, off-peak scheduling, and explicit rollback. - Product concern about cache staleness: Kept TTL short (200 ms), added write-through invalidation, and added a per-user "bypass cache" header for critical admin flows. Learning - Measure before optimizing: Observability first reduced risk and focused effort. - Prefer reversible changes: Flags, canaries, and small PRs let us move fast with guardrails. - Institutionalize: We added a pre-event readiness checklist, standard tracing on hot paths, and a runbook for index rollouts. # Why This Works in an Interview - Shows initiative: You acted before an incident. - Demonstrates engineering judgment: You weighed speed, risk, and reversibility. - Is data-driven: Clear before/after metrics and SLO framing. - Handles pushback: You de-risked with flags/canaries and stakeholder alignment. # Pitfalls to Avoid - Vague actions ("we improved it"): Name concrete steps and tools. - No metrics: Provide at least directional numbers; if exacts are unavailable, estimate and label as such. - Over-claiming ownership: Clarify your role and collaborators. - Risky big-bang changes under time pressure: Prefer reversible, low-blast-radius steps first. # Alternate Example (Opportunity, not Incident) - Situation: CI times grew to 45 minutes, slowing releases. - Options: Parallelize tests, selective test runs via change impact analysis, container caching, or splintering monorepo workflows. - Actions: Added test selection based on changed modules, cached dependencies, split workflow into parallel shards; rolled out behind a flag and compared DORA metrics. - Impact: CI 45 → 18 minutes (60% faster), weekly deploys 3 → 6, saved ~40 engineer-hours/week. - Pushback: Flakiness concerns handled via a quarantine list and nightly full runs. - Learning: Invest early in platform productivity; small platform improvements compound. # Quick Checklist Before You Answer - 1–2 sentences of context tied to impact/SLO. - 3–5 bullets of actions with options/trade-offs. - 2–3 measurable outcomes. - One pushback you handled and how. - One learning you applied later. Use this template to keep your story crisp, technical, and outcome-oriented within 2–3 minutes.