A pandas DataFrame contains \(N\) daily temperature observations for `NYC` and \(P\ge5\) town columns. Implement: ```python q1_q5(df) -> list q6(df) -> list[str] ``` `q1_q5(df)` must return exactly this five-element list: ```text [ largest_std_column, rounded_conditional_median, rounded_absolute_slope_sum, best_single_town, [best_pair_town_1, best_pair_town_2] ] ``` `q6(df)` must return exactly five town names in greedy selection order. ### Constraints & Assumptions - The DataFrame has at least one row, unique string column names, a column named `NYC`, a column named `Town2`, and at least five non-`NYC` columns. - All cells are finite, numeric, and nonmissing. `NYC` is the response and every other column is a candidate town. - Use population standard deviation (`ddof=0`). If columns tie for the largest standard deviation, return the lexicographically smallest column name. This comparison includes `NYC`. - Every regression includes an intercept and is scored by in-sample mean squared error on all supplied rows. - Break exact MSE ties lexicographically by town name or by the ordered two-town tuple. - “Round” means nearest integer, with exact half values rounded away from zero. - If no row has `Town2` between 90 and 100 inclusive, raise `ValueError`. - A constant predictor has slope zero. For rank-deficient multivariate designs, use the minimum-norm least-squares solution; scoring is based on its predictions. ### Part 1 — Variability and conditional median Set `largest_std_column` to the column with the largest population standard deviation. Set `rounded_conditional_median` to the median `NYC` value among rows where `90 <= Town2 <= 100`, rounded once. #### What This Part Should Cover - Population standard deviation across rows for every column, including `NYC`, with the stated lexicographic tie rule. - Inclusive `Town2` filtering, the required empty-subset error, and one half-away-from-zero rounding step. - Placement of these two outputs in positions 0 and 1 of the returned list. ### Part 2 — Single-town regressions Fit one separate simple linear regression per candidate town. Set `rounded_absolute_slope_sum` to the sum of the absolute slope coefficients, rounded once after summing. Set `best_single_town` to the town whose single-predictor model has the lowest MSE. #### What This Part Should Cover - A distinct intercept-bearing fit for each non-`NYC` column and a zero slope for a constant predictor. - Absolute slopes summed before the single required rounding operation. - In-sample MSE comparison with the stated town-name tie rule and outputs in positions 2 and 3. ### Part 3 — Best pair Fit every unordered two-town linear model. Return the lowest-MSE pair as two lexicographically ordered names. #### What This Part Should Cover - Enumeration of each unordered town pair exactly once and a joint two-predictor fit with an intercept. - Minimum-norm least squares for a rank-deficient pair and MSE scoring from its predictions. - Lexicographic ordering within the pair, tuple-based score ties, and the nested list in position 4. ### Part 4 — Approximate best five-town subset For `q6`, begin with no predictors. At each of five steps, refit each model formed by the already selected towns plus one remaining town. Append the town producing the lowest MSE, using its name to break an exact tie. Greedy selection is an approximation and need not equal the globally best five-town subset. #### What This Part Should Cover - Five forward-selection rounds, with every remaining candidate refit alongside the already selected towns at each round. - In-sample MSE and the exact town-name tie rule at every selection step. - Exactly five distinct names returned in selection order, plus a clear statement that the greedy result is not guaranteed globally optimal. ```hint Reuse one scorer Build a deterministic least-squares scoring helper, a half-away-from-zero rounding helper, and one sorted candidate-town list, then use them for every output. ``` ### What a Strong Answer Covers - Executable pandas and NumPy code for both functions. - The exact five-element `q1_q5` layout and five-name `q6` layout. - One intercept-bearing fit per requested predictor set. - Constant columns, the empty conditional subset, rank deficiency, and deterministic ties. - \(O(P^2)\) pair enumeration and approximately \(5P\) greedy candidate fits. ### Follow-up Questions - How would nested cross-validation change feature selection and final scoring? - Why can a town with a weak univariate model become useful in a multivariate model? - How could sufficient statistics reduce repeated fitting when \(P\) is large?