Design resilient auth with flaky third-party tokens
Company: Snowflake
Role: Software Engineer
Category: System Design
Difficulty: hard
Interview Round: Onsite
A client must obtain a token from a third-party authorization service before calling your main API. The third-party service is unreliable and may return malformed responses or fail intermittently. Design a resilient, secure end-to-end request flow. Cover token acquisition, validation, caching and refresh, retries with exponential backoff and jitter, circuit breakers, request hedging, idempotency keys, fallback paths (e.g., degraded mode or cached permissions), quarantining/poison-pill detection for bad responses, observability (metrics, tracing, alerts), rate limiting, SLA/SLOs, and blast-radius containment across regions. Describe APIs, data models, and failure-mode playbooks.
Quick Answer: This question evaluates a candidate's ability to design resilient, secure authentication and authorization flows that tolerate unreliable third-party token services, testing competencies in distributed systems architecture, security engineering, reliability, API design, and operational engineering.