Design file deduplication across nested directories
Company: Anthropic
Role: Software Engineer
Category: Coding & Algorithms
Difficulty: Medium
Interview Round: Technical Screen
Design and implement a file deduplication tool that, given a root directory, identifies groups of duplicate files. Requirements: traverse nested directories safely; handle symbolic links and potential cycles; minimize I/O with a multi-stage strategy (e.g., compare file sizes, then partial hashes, then full hashes); support very large files; and output duplicate groups with file paths. Explain your data structures, step-by-step algorithm, time and space complexity, and key test cases.
Quick Answer: This question evaluates a candidate's competency in file system traversal, robust I/O handling including symbolic links and cycles, content-based deduplication, and algorithmic efficiency within the Coding & Algorithms domain.