What Cross-Validation Reduces: Bias, Variance, Type I or Type II Error

A multiple-choice statistics question asks: what is cross-validation used to reduce? - A. Bias - B. Variance - C. Type I error - D. Type II error Choose an answer and justify it, including why each of the other options does not fit. ```hint Ask what is being estimated Think about what quantity cross-validation produces and how reliable that quantity is compared with a single train/validation split. ``` ### Constraints and Clarifications - The question is reported without further context. Be explicit about which "variance" or "bias" you mean: that of the fitted model, or that of the estimated generalization error. ### Clarifying Questions - Is cross-validation used only to evaluate one fixed model, or also to choose hyperparameters or between models? - How many folds are used, and is the data independent across rows or ordered in time? ### What a Strong Answer Covers - The chosen option, supported by the mechanism rather than just a letter. - The difference between what cross-validation does to a performance estimate and what cross-validation-based model selection does to the chosen model. - Why the hypothesis-testing options do not describe cross-validation. - How the number of folds trades bias against variance of the error estimate. ### Follow-up Questions 1. Why can leave-one-out cross-validation give a higher-variance error estimate than 5-fold or 10-fold cross-validation? 2. What goes wrong if you tune hyperparameters with cross-validation and then report that same cross-validation score as the final performance? 3. How would you cross-validate a model trained on time-ordered data such as daily returns?