Optimize HTTP requests for speed and limits

You are building a service that needs to send HTTP requests to a large number of URLs (for example, a simple web crawler or data fetcher). The naive implementation processes URLs sequentially in a single thread, so total runtime is long.

The interviewer asks two follow-up questions:

1. If you want to speed up the overall processing, how would you change the design?
2. If you want to avoid sending too many requests at the same time (e.g., to respect rate limits or avoid overloading remote servers), how would you handle that?

Describe a high-level design that:

• Increases throughput and reduces total completion time.
• Controls how many requests are in flight simultaneously (globally and/or per host).

You do not need to provide code, but clearly explain the main components (threading model, queues, rate limiting, etc.) and the trade-offs of your approach.