Implement A Concurrent Download De-Duplication Layer
Quick Overview
Study a concurrent downloader design where duplicate requests for the same URL must share one in-flight download result. The prompt covers futures, atomic map updates, failure semantics, cleanup, and an extension into accurate delayed-event scheduling.
Implement A Concurrent Download De-Duplication Layer
Company: Uber
Role: Software Engineer
Category: Software Engineering Fundamentals
Difficulty: medium
Interview Round: Onsite
Design and implement a downloader that accepts many URL download requests concurrently. Requests for different URLs should run independently. If several threads request the same URL while the first download is still in progress, only one actual download should happen, and all waiting callers should receive the same result when it finishes. The interviewer also asks how you would extend the design to fire a large number of delayed events accurately.
<details>
<summary>Hint 1</summary>
Start by naming the core entities, constraints, and success criteria.
</details>
<details>
<summary>Hint 2</summary>
Make the trade-offs explicit before going deep on implementation details.
</details>
### Constraints & Assumptions
- Downloads may succeed or fail.
- Multiple callers can request the same URL at nearly the same time.
- The implementation should be thread-safe.
- For delayed events, accuracy and scale matter more than business-specific payloads.
### Clarifying Questions to Ask
- Should failed downloads be shared with waiters or retried independently?
- Should completed results be cached after the first completion?
- What timeout and cancellation behavior is required?
- How many delayed events can be scheduled at once?
- What timing accuracy is acceptable?
### What a Strong Answer Covers
```premium-lock What a Strong Answer Covers
```
### Follow-up Questions
- How would you implement this in Java with `CompletableFuture`?
- What changes if downloads run across multiple processes?
- How would you prevent one hot URL from exhausting memory?
- How would you persist delayed events across restarts?
Quick Answer: Study a concurrent downloader design where duplicate requests for the same URL must share one in-flight download result. The prompt covers futures, atomic map updates, failure semantics, cleanup, and an extension into accurate delayed-event scheduling.
Design and implement a downloader that accepts many URL download requests concurrently. Requests for different URLs should run independently. If several threads request the same URL while the first download is still in progress, only one actual download should happen, and all waiting callers should receive the same result when it finishes. The interviewer also asks how you would extend the design to fire a large number of delayed events accurately.
<details>
<summary>Hint 1</summary>
Start by naming the core entities, constraints, and success criteria.
</details>
<details>
<summary>Hint 2</summary>
Make the trade-offs explicit before going deep on implementation details.
</details>
Constraints & Assumptions
Downloads may succeed or fail.
Multiple callers can request the same URL at nearly the same time.
The implementation should be thread-safe.
For delayed events, accuracy and scale matter more than business-specific payloads.
Clarifying Questions to Ask
Should failed downloads be shared with waiters or retried independently?
Should completed results be cached after the first completion?
What timeout and cancellation behavior is required?
How many delayed events can be scheduled at once?
What timing accuracy is acceptable?
What a Strong Answer Covers Premium
Follow-up Questions
How would you implement this in Java with
CompletableFuture
?
What changes if downloads run across multiple processes?
How would you prevent one hot URL from exhausting memory?
How would you persist delayed events across restarts?