Can Test Correlation Exceed Training Correlation?

A linear model is fitted on 70% of a dataset containing 100,000 observations. Its predictions have Pearson correlation 0.20 with the training targets. On the remaining 30%, its predictions have correlation 0.25 with the test targets. ### Clarifying Questions - Is the fit ordinary least squares with an intercept, using the same feature columns in both subsets? - Are the correlations empirical correlations between predictions and targets, with nonzero variances? - Does refitting mean finding the exact unconstrained least-squares solution, without regularization? For the main argument, use these ordinary least-squares assumptions. Explain which conclusions depend on them. ### Part 1 — Compare Training and Test Correlations Can the reported 0.20 training correlation and 0.25 test correlation occur? Explain why the larger training sample does or does not determine the ordering. #### What This Part Should Cover Separate a typical generalization pattern from a mathematical guarantee about two different samples. ### Part 2 — Refit on the Former Test Set Fit a new linear model directly on the 30% subset and evaluate its predictions on that same subset. Can its correlation with the targets be below 0.25? #### What This Part Should Cover Use the geometry of least squares, distinguish correlation from squared error, and identify assumptions under which the conclusion can fail. ### What a Strong Answer Covers A precise yes/no answer to each part, a projection argument for the refit, and a clear distinction between evaluation on held-out data and evaluation on the data used for fitting. ### Follow-up Questions - What changes if the refit uses regularization or a different feature set? - What if the reported metric is an uncentered similarity instead of Pearson correlation?