# Analyze a Regression Dataset and Build a Safe File-Processing Utility You receive a feature matrix X and a continuous target y. After the analysis, implement a Python utility that discovers files, renames selected files under a deterministic rule, filters log lines by keywords, and writes outputs without corrupting the source directory. ### Constraints & Assumptions - The number of rows may be smaller than the number of features. - Features may be missing, duplicated, highly correlated, or measured on different scales. - The rename operation must preserve extensions and avoid collisions. - Log processing must stream line by line and leave the input files unchanged. ### Clarifying Questions to Ask - Is prediction or coefficient interpretation the main modeling objective? - Is there a temporal or grouped split requirement? - Should rename and log writes be dry-run capable and atomic? ### Part 1 — Regression exploration Identify which columns relate to y, which add incremental signal, and which carry redundant information. Propose models and an honest evaluation plan. #### What This Part Should Cover - Univariate and conditional analysis - Collinearity diagnostics and regularization - Leakage-safe baselines and validation ### Part 2 — File utility Design functions for path discovery, collision-safe renaming, keyword-based log filtering, and output creation. Explain error handling and test cases. #### What This Part Should Cover - Pathlib-based traversal and explicit selection - Two-phase rename or collision detection - Streaming, encoding, atomic replacement, and rollback behavior ### What a Strong Answer Covers - A model-evaluation plan tied to the dataset shape - File-system invariants and failure handling - Concrete edge-case tests ```hint Make destructive actions reversible Produce a rename plan before changing paths. Validate that destinations are unique and outside the input set, then apply the plan with a recovery record. ``` ### Follow-up Questions - How would you detect nonlinear signal missed by correlation? - How would the file utility resume after a process crash?