Batch-Apply Pillow Image Transforms, Then Parallelize: Processes vs Threads

You are given a set of cat images and must apply a series of image transforms to each one. The interviewer fixes the image library, Pillow, and the starter material shows how to call each required transform in Pillow. A provided test harness runs your code over all the images, checks the results, and reports how long it took to process all of them. For practice, treat the transforms as a fixed sequence of standard Pillow operations (for example a resize, a rotation and a grayscale conversion) applied in order to every image. In the interview, the exact list and the exact calls come from the starter material. ### Constraints and Clarifications - Use Pillow for all image work; the call for each transform is given. - A faster version must produce exactly the same output images as the simple version, because the tests check the results. - The number of images, their sizes and the number of CPU cores on the test machine are not given. Ask, or state your assumptions. ### Clarifying Questions - Is the same transform sequence applied to every image, or does each image have its own list? - Where should outputs be written, and how should they be named? May an output overwrite its input? - If one image is corrupt or cannot be opened, should the whole run fail, or should it continue and report the failure? - Roughly how many images are there, how large are they, and how many CPU cores does the test machine have? ### Part 1 — Apply the transforms Implement the processing: for every input image, open it, apply the transforms in order using the given Pillow calls, and save the result. Make the provided tests pass. ```hint Keep one image's work self-contained Write the work for a single image, from opening the file to saving the result, as one unit that shares no state with the others. Testing, error handling and Part 2 all become easier. ``` #### What This Part Should Cover - Correct use of the given Pillow calls in the required order, including image modes and formats on save - Resource handling: closing files and keeping only one image's data in memory at a time - A clean per-image unit of work and a driver over all inputs - Handling of unreadable or unusual images ### Part 2 — Make it faster The harness reports the total time. Reduce it. If you parallelize the work, be ready for two questions from the interviewer: - Why did you choose multiple processes rather than multiple threads, or the reverse? - If you use processes, how many workers do you start, and why that number? ```hint Find where the time goes Before choosing a tool, decide whether the per-image work is limited by the CPU or by waiting on the disk, and what that means for Python threads. ``` ```hint Sizing the pool Think about how many workers the machine can actually run at the same moment, and what each extra worker costs in start-up time and memory. ``` #### Clarifying Questions for this Part - Are all the cores the operating system reports available to this process, or is it limited, for example by a container CPU quota? - Does the reported time include starting the workers? #### What This Part Should Cover - Whether the workload is CPU-bound or I/O-bound, and how to check - The global interpreter lock and what it means for threads versus processes, including where Pillow does and does not release it - Reasoning for the worker count, confirmed by measurement - Overheads: process start-up, data sent between processes, batching of tasks, and memory per worker ### What a Strong Answer Covers - A correct, tested Part 1 before any optimization - A measured baseline and measured improvements - A parallel structure that sends paths rather than image data and produces identical outputs - An accurate explanation of the trade-off between processes and threads for this workload - Failure handling that still works in the parallel version ### Follow-up Questions - The image set grows to millions of files spread across many machines. What changes? - Profiling shows most of the time is spent decoding and encoding JPEG files. What would you try? - On an 8-core machine, 8 workers turn out slower than 6. What could explain it? - How would you make the run resumable if it crashes halfway through?