## Consolidate Three Partner APIs into a Query Service Design a service that retrieves data from three partner APIs, consolidates their different representations into one model, and serves user queries over the combined data. The design must continue to give useful, honest results when partners are slow, unavailable, stale, or inconsistent. ### Constraints & Assumptions - Each partner has its own schema, identifiers, pagination, rate limits, and update behavior. - The same real-world entity may appear in more than one partner response. - Partner data can be delayed or corrected, so every normalized record needs provenance and freshness metadata. - Required scale, query patterns, acceptable staleness, and conflict policy must be clarified. - Credentials and raw partner payloads are not exposed directly to end users. ### Clarifying Questions to Ask - Are partners polled, queried on demand, or able to push changes? - What entities and filters must the user-facing API support? - How are records matched across partners, and is there an authoritative source for each field? - May a query return partial or stale data, and how must that status be shown? - What are the partner quotas, dataset sizes, update rates, and user-query targets? ### Part 1 — Normalize and Resolve Identity Define adapter boundaries, the canonical data model, source provenance, and the process for matching records that may represent the same entity. #### What This Part Should Cover - A versioned adapter per partner rather than partner fields leaking into every consumer. - Raw payload retention or references for replay and audit. - Deterministic identity resolution with confidence and manual-review paths where needed. - Field-level source, observed time, and normalization version. ```hint Preserve evidence before choosing a winner Consolidation is easier to repair when the normalized value still points to the partner record and rule that produced it. ``` ### Part 2 — Fetch, Consolidate, and Serve Queries Design ingestion, retry, deduplication, storage, and the user query API. Explain which work happens asynchronously and what happens when one partner is unavailable. #### What This Part Should Cover - Checkpoints, bounded retries, rate-limit handling, and idempotent writes. - A canonical store and indexes matched to user filters. - Explicit freshness and partial-result semantics. - Cache keys and invalidation tied to canonical versions. ```hint Decouple partner latency from user latency If users do not require a live partner read, a durable normalized snapshot can keep queries predictable during a partner outage. ``` ### Part 3 — Scale and Operate the System Address increasing partner volume, query traffic, hot entities, backfills, schema changes, and production debugging. #### What This Part Should Cover - Partitioning that isolates partner ingestion from user-query load. - Backpressure and fair use of partner quotas. - Safe adapter-version rollout and replay from raw data. - Metrics for lag, match rates, conflicts, partial results, and query latency. ```hint Measure freshness per source One combined "last updated" timestamp can hide that two partners are current while the third is hours behind. ``` ### What a Strong Answer Covers - Clear requirements and a canonical model that retains source lineage. - Reliable partner-specific ingestion with replayable normalization and deterministic reconciliation. - Query behavior that exposes stale, conflicting, or partial data instead of silently masking it. - A scalable partitioning, backfill, rollout, and observability plan. ### Follow-up Questions 1. How would you correct a faulty normalization rule without refetching every partner record? 2. What should a query return when two partners disagree on a field with no declared authority? 3. How would you stop one large backfill from exhausting a partner's quota for fresh updates? 4. Which key would you use to partition if one entity accumulates unusually many records?