Implement a Finite Concurrent Web Crawler
Company: MongoDB
Role: Software Engineer
Category: Software Engineering Fundamentals
Difficulty: hard
Interview Round: Onsite
Overview: Implement a finite concurrent web crawler with transactional URL deduplication, a bounded disk-backed frontier, polite fetching, lossless discovery, classified retries, correct termination detection, and crash recovery.