Optimize HTTP requests for speed and limits
Overview
This question evaluates understanding of concurrency, throughput optimization, rate limiting, and trade-offs in designing scalable networked services. It is commonly asked to assess architectural decision-making for handling large-scale HTTP request workloads, falls under the System Design domain, and tests practical application–level design thinking rather than low-level coding details.
You are building a service that needs to send HTTP requests to a large number of URLs (for example, a simple web crawler or data fetcher). The naive implementation processes URLs sequentially in a single thread, so total runtime is long.
The interviewer asks two follow-up questions:
- If you want to speed up the overall processing, how would you change the design?
- If you want to avoid sending too many requests at the same time (e.g., to respect rate limits or avoid overloading remote servers), how would you handle that?
Describe a high-level design that:
- Increases throughput and reduces total completion time.
- Controls how many requests are in flight simultaneously (globally and/or per host).
You do not need to provide code, but clearly explain the main components (threading model, queues, rate limiting, etc.) and the trade-offs of your approach.
Solution
Login required