# Implement Stateful Fetch-N over a Paginated Upstream ## Scenario An existing upstream method `fetch(pageToken)` returns `{items, nextPage}`. Build a stateful class initialized with the first page token and expose `fetchN(n)`, which returns the next items in source order. It should return exactly `n` items unless the source reaches the end, preserve unreturned items for later calls, and return an empty list for `n = 0`. A missing `nextPage` marks the end of the source. Assume calls are sequential unless you explicitly add concurrency control. ### Part 1: Define the state and contract Specify the state that survives between calls and define behavior for invalid input, end of source, empty pages, and repeated calls after exhaustion. #### What This Part Should Cover - The current page token, a buffer of fetched but unreturned items, and an end-of-source marker. - Whether `n` may be negative and how that case is reported. - The difference between an empty page with another token and an empty terminal page. - Detection of malformed pagination such as a repeated-token loop. ```hint Name the persistent invariant Write down the invariant relating emitted items, buffered items, and the next page before designing the loop. ``` ### Part 2: Implement `fetchN` Describe or write the method that drains buffered items, fetches as many pages as needed, and retains any surplus for the next call. #### What This Part Should Cover - Returning buffered items before making a new upstream request. - Preserving source order across page boundaries. - Avoiding loss or duplication when a page contains more items than the current call needs. - Terminating correctly at end of source and across consecutive empty pages. ```hint Separate two state transitions Separate consuming an item from advancing the page token; they do not always happen at the same time. ``` ### Part 3: Handle an unreliable upstream Extend the design for timeouts, retryable errors, duplicate responses, and an upstream whose pages can change while the client is reading them. #### What This Part Should Cover - Bounded retry, backoff, and which failures are retryable. - A declared all-or-error or partial-return contract. - How state remains consistent if a call fails after some pages were fetched. - Requirements for stable page tokens, snapshots, or response deduplication. - Metrics for retries, latency, empty pages, token loops, and exhaustion. ```hint Choose a commit point Consider building the result against temporary state and deciding exactly when persistent state is committed. ``` ### What a Strong Answer Covers A strong answer defines persistent invariants before presenting code. It returns items in order, avoids unnecessary fetches, preserves leftovers, and handles terminal and nonterminal empty pages. Its failure contract makes clear whether a retry can lose, duplicate, or skip data, and it states what guarantees must come from the upstream service. ### Follow-up Questions - How would you make `fetchN` safe for concurrent callers? - What would change if page tokens expire? - How can the client distinguish a legitimate repeated item from a replayed page? - What state would need to be persisted to resume after a process restart?