Slopes of y-on-x vs x-on-y Regression: Relationship and When They Are Equal

You fit two simple ordinary least squares (OLS) regressions, each with an intercept, on the same paired dataset $(x_i, y_i)$ for $i = 1, \dots, n$: 1. Regress $y$ on $x$: $\;y = \alpha_1 + \beta_{y|x}\, x + \varepsilon$ 2. Regress $x$ on $y$: $\;x = \alpha_2 + \beta_{x|y}\, y + \eta$ The interviewer wants you to reason about how the two fitted slopes relate to each other, and what it takes for them to be equal. ### Constraints & Assumptions - Both fits are plain OLS with an intercept and a single predictor. - The data are not degenerate: $n \ge 3$, and the sample variances of $x$ and $y$ are strictly positive. - "Coefficients" refers to the slope estimates $\hat\beta_{y|x}$ and $\hat\beta_{x|y}$. - You may transform the data (e.g., rescale variables) when asked how to make the slopes equal. ### Clarifying Questions to Ask - Are both fits ordinary least squares, or is a symmetric method (e.g., orthogonal / total least squares) intended? - Should I treat the data as raw, or can I assume it has already been centered or standardized? - Do you care about the intercepts as well, or only the slopes? - When you ask how to make the coefficients equal, may I transform the variables, or must a condition hold on the raw data? ### Part 1 Derive both slope estimates in terms of sample moments. What is the relationship between $\hat\beta_{y|x}$ and $\hat\beta_{x|y}$? In particular, what is their product, and why is $\hat\beta_{x|y}$ generally *not* equal to $1/\hat\beta_{y|x}$, even though the second regression looks like the first one "solved for $x$"? ```hint Write both slopes with sample moments For simple OLS, the slope is the covariance over the variance of the predictor: $\hat\beta_{y|x} = \widehat{\mathrm{Cov}}(x,y)/\widehat{\mathrm{Var}}(x)$. Write the analogous expression for the reverse regression and compare the two. ``` ```hint Bring in the correlation Express each slope as the sample correlation $r$ times a ratio of standard deviations. Multiplying the two expressions makes the relationship pop out — and connects it to a familiar goodness-of-fit quantity. ``` #### What This Part Should Cover ```premium-lock What This Part Should Cover ``` ### Part 2 Under what condition on the data are the two slopes exactly equal? How would you transform the data so that the two regressions are guaranteed to produce the same slope, and what is the value of that common slope? ```hint Set the two expressions equal Using the moment forms from Part 1, equate them and see what must be true of the sample variances of $x$ and $y$ (watch for the degenerate zero-covariance case). ``` ```hint A standard preprocessing step does it Think about what happens to both slopes if you z-score each variable. What do the standard-deviation ratios become, and what quantity is each slope then equal to? ``` #### What This Part Should Cover ```premium-lock What This Part Should Cover ``` ### What a Strong Answer Covers ```premium-lock What a Strong Answer Covers ``` ### Follow-up Questions - Suppose both $x$ and $y$ are noisy measurements of the same underlying quantity. Is either OLS slope an unbiased estimate of the true relationship? What symmetric alternatives exist, and what do they assume? - In the $(x, y)$ plane, how do the two fitted *lines* (not just slopes) relate geometrically? When do they coincide? - What happens to each of the two slopes if you add independent noise to $x$ only, leaving $y$ unchanged?