Resolving IO, Memory and CPU Bottlenecks, and Choosing Threads vs Processes
Company: Hudson River Trading
Role: Software Engineer
Category: Software Engineering Fundamentals
Difficulty: medium
Interview Round: Technical Screen
A phone-screen round opens with a conversation about your past projects and then moves to concurrency fundamentals. Answer both parts as you would to the interviewer, with concrete detail.
### Clarifying Questions
- Should the story in Part 1 come from production work, and how much measured detail does the interviewer expect?
- For Part 2, is there a particular language or runtime in mind, such as standard Python with its global interpreter lock, or C++ or Java?
- Is the concurrency question limited to one machine, or should scaling across machines be included?
### Part 1 — Constraints you have hit
Describe a time a system you worked on was limited by IO, memory or CPU. How did you establish which resource was the bottleneck, what did you change, and how did you confirm that the change worked?
```hint Measure before naming the bottleneck
Think about which measurements separate a process that is waiting on disk or network from one that is busy computing, and from one that is running out of memory.
```
#### What This Part Should Cover
- The evidence used to identify the limiting resource
- A fix that matches that resource, and the trade-off it introduced
- Before and after measurements that confirm the result
### Part 2 — Threads versus processes
Explain the difference between multithreading and multiprocessing, and when you would use each.
```hint Follow the shared memory
Ask what threads share that processes do not, and what that sharing costs or buys for work that mostly waits compared with work that mostly computes.
```
#### What This Part Should Cover
- Address space, isolation, and the cost of creating, switching and communicating
- How the workload type (waiting on IO versus computing) drives the choice
- Runtime-specific limits, such as the global interpreter lock in standard Python
### What a Strong Answer Covers
- A real constraint tied to the right resource with measurements, not intuition
- Fixes chosen for that resource, with their trade-offs stated
- Accurate mechanics of threads and processes and their failure modes
- A clear decision rule for choosing between them, including runtime nuances
- A link between the two parts: how the type of bottleneck determines the concurrency model
### Follow-up Questions
- In a CPU-bound Python service, why can adding threads make throughput worse, and what would you do instead?
- Several processes need the same large read-only dataset. How do you avoid holding one copy per process?
- How would you choose between threads and an asynchronous event loop for a service making thousands of concurrent network calls?
- What failure modes does a multithreaded design introduce that a multiprocess one avoids, and the reverse?
Overview: A two-part systems fundamentals discussion: describe a real project where IO, memory or CPU was the bottleneck and how you identified and fixed it, then explain multithreading versus multiprocessing and when to use each, including the effect of Python's global interpreter lock.
Read the full Hudson River Trading Software Engineer interview experience this question came from