How do I approach System Design interview questions?

System Design questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master system design interviews.

What difficulty level is this interview question?

This is a medium difficulty System Design question, commonly asked during Technical Screen rounds at Dropbox.

What role is this question designed for?

This question is commonly asked for Software Engineer candidates at Dropbox during technical interviews.

Scale a file-crawling API using async jobs | Dropbox Interview Question

Quick Overview

This question evaluates a candidate's ability to design scalable, fault-tolerant systems for long-running file-crawl operations, including async job orchestration, pagination/streaming, concurrency control, caching/deduplication, failure handling, and storage-abstraction considerations.

You implemented a synchronous API that lists all files under a directory (recursive crawl). Now you need to scale it.

Part A — Scaling discussion (no code required)

How would you scale the “list all files under a path” API when:

The directory tree can be huge (millions/billions of files)
Crawls can take minutes+ and may time out
Many users may request crawls concurrently
The underlying storage could be local disks or network storage (e.g., NFS / object-store-like abstraction)

Discuss:

API shape (sync vs async)
Pagination/streaming of results
Concurrency limits and backpressure
Caching and deduplication
Failure handling and partial results

Part B — Async job implementation (with provided templates)

Based on your design in Part A, implement (or describe at a detailed interface level) an async crawl job:

Client submits rootPath and gets back a jobId .
Clients can poll job status and fetch results.
The job should be robust to failures, support retries, and avoid repeating work excessively.

Specify key components/APIs, e.g.:

POST /crawl-jobs → { jobId }
GET /crawl-jobs/{jobId} → status/progress
GET /crawl-jobs/{jobId}/results?cursor=... → paged results

State any assumptions (e.g., maximum result size, retention window, storage type).

Quick Overview

Part A — Scaling discussion (no code required)

How would you scale the “list all files under a path” API when:

The directory tree can be huge (millions/billions of files)

Crawls can take minutes+ and may time out

Many users may request crawls concurrently

The underlying storage could be local disks or network storage (e.g., NFS / object-store-like abstraction)

Discuss:

API shape (sync vs async)

Pagination/streaming of results

Concurrency limits and backpressure

Caching and deduplication

Failure handling and partial results

Part B — Async job implementation (with provided templates)

Based on your design in Part A, implement (or describe at a detailed interface level) an async crawl job:

Client submits rootPath and gets back a jobId .

Clients can poll job status and fetch results.

The job should be robust to failures, support retries, and avoid repeating work excessively.

Specify key components/APIs, e.g.:

POST /crawl-jobs → { jobId }

GET /crawl-jobs/{jobId} → status/progress

GET /crawl-jobs/{jobId}/results?cursor=... → paged results

State any assumptions (e.g., maximum result size, retention window, storage type).

Scale a file-crawling API using async jobs

Quick Overview

Part A — Scaling discussion (no code required)

Part B — Async job implementation (with provided templates)

Solution

Submit Your Answer to Earn 20XP

Scale a file-crawling API using async jobs

Quick Overview

Part A — Scaling discussion (no code required)

Part B — Async job implementation (with provided templates)

Solution

Submit Your Answer to Earn 20XP