Classic interview question: web crawler. You can also find this one on the usual practice-problem site.
First single-threaded, then the follow-up was multi-threaded. Only need to consider pages under a single hostname.
The extra piece compared to the practice-site version is that you need to handle the fragment — the #... part of the URL needs to be stripped. No need to worry about URL normalization.
Two pitfalls I ran into:
One was that you need to strip the fragment from the URL before deduplicating. I actually knew this, but when I was explaining it out loud I didn't get the order across clearly, so the interviewer kept pushing me on exactly which step the sanitizing happens at.
The other was the concurrency part. I'm actually very comfortable with thread pools and task synchronization, and I'd done similar practice problems before, but in the interview my brain just blanked for a second and I started rattling off a few low-level primitive approaches first. The interviewer's face right then... I did recover and get back to the thread pool eventually, but he clearly already felt like I'd gone in the wrong direction.
We also expanded into some system-design-adjacent questions during the interview, but didn't spend much time on it. My guess is that once I stumbled, I just got quietly rejected.
Discussion
Loading comments…