Find Duplicate Files While Handling Symbolic-Link Cycles

Design `find_dups` to discover duplicate file contents under a directory root. Discuss depth-first traversal, symbolic links that can form cycles, and content hashing used to identify candidates. ### Constraints & Assumptions - This is a filesystem engineering discussion, because traversal, file reads, permissions, and symbolic links are part of the task. - Define duplicates by identical byte contents, not merely equal names, sizes, or hashes. - State whether symlinks are followed and whether hard-link aliases count as separate duplicates before giving the algorithm. - No particular operating-system API or immutable-filesystem guarantee is supplied. Explain the assumptions behind any claim of a complete result. ### Clarifying Questions to Ask - Are symlinks followed only inside the requested root, and should external targets be skipped? - Should two paths to the same underlying file appear twice in the result? - Can files change during the scan, and is a filesystem snapshot available? - Should unreadable files fail the whole scan or be reported separately? ```hint A path string is not always an object identity Different paths can refer to the same file or directory. Choose the identity used for traversal before relying on a visited set. ``` ### What a Strong Answer Covers - A traversal policy that prevents symlink cycles and avoids repeatedly scanning the same directory. - File identity, hard-link handling, and separation of aliases from equal contents in distinct files. - Size grouping, streamed content hashes, and exact comparison when correctness requires it. - Bounded-memory reads and explicit handling of permissions or changes during scanning. - Costs in visited filesystem objects and bytes read. ### Follow-up Questions - Why is using only a content hash insufficient for a mathematical guarantee of identical contents? - How would you scale the scan when candidate files are larger than memory?