You are a data scientist building and interpreting regression models on a product dataset at a marketplace company. The outcome variable is a continuous user-level metric such as **spend**, **session duration**, or **order value**. The dataset includes user attributes, prior engagement, device type, geography, and treatment indicators. This is a "mini-case" round: each part probes a specific piece of regression knowledge. Answer each part clearly, stating your reasoning, the relevant formula, and the assumptions you rely on. ### Constraints & Assumptions - The outcome $y$ is continuous and user-level; several candidate metrics (spend, order value, session duration) are heavily right-skewed. - Covariates include user attributes, prior engagement, device type, geography, and treatment indicators. - Sample size is large (millions of user rows), so even tiny effects can be statistically significant — keep this in mind for the p-value discussion. - For the log-transform part, assume $y$ is strictly positive unless you explicitly handle zeros. - You can hold out data for validation. The interviewer cares about correct interpretation and sound decision-making, not coding syntax. ### Clarifying Questions to Ask - What is the decision the model supports — causal inference on a treatment effect, prediction/targeting, or descriptive interpretation? This changes everything downstream. - Which specific outcome are we modeling (spend, order value, duration), and does it contain zeros or negative values? - Is the treatment indicator randomized, or do we need to worry about confounding, and which covariates are pre- vs. post-treatment? - How much data do we have (number of rows vs. number of candidate covariates)? - Do stakeholders need absolute-scale effects (dollars/minutes) or relative effects (percent lift)? ### Part 1 — Interpreting a coefficient and its p-value In a linear regression, how do you interpret a single coefficient $\beta_j$ and its p-value? Be precise about what is held constant, what null hypothesis the p-value tests, and the common ways these quantities are misread. Handle both a continuous predictor and a $0/1$ indicator. ```hint What the coefficient conditions on A coefficient is a *partial* (ceteris paribus) effect: ask what is held constant when $x_j$ moves, and why $\beta_j$ can shift as you add or drop other covariates. The interpretation differs for a $0/1$ dummy versus a continuous predictor. ``` ```hint What the p-value actually tests The p-value is tied to a specific null, usually $H_0: \beta_j = 0$, under the model's standard-error assumptions. Think about what changes as $n$ grows, and keep three ideas distinct: **statistical** significance, **practical** (effect-size) significance, and **causality**. ``` #### What This Part Should Cover - A precise, *conditional* reading of $\beta_j$ that names what is held fixed, and the distinct interpretations for a continuous predictor (slope) versus a $0/1$ dummy (group-mean difference). - An accurate statement of the p-value's null ($H_0:\beta_j=0$), what the t-statistic is, and what the p-value is **not** (not $\Pr(H_0)$, not the probability the effect is real). - The large-$n$ point: $\text{SE}(\hat\beta_j)\propto 1/\sqrt{n}$, so trivial effects become "significant" — separate statistical, practical, and causal significance. - Awareness of the inference assumptions (functional form, error independence, homoscedasticity/robust SEs) and how multicollinearity destabilizes coefficients. ### Part 2 — Raw outcome vs. log-transformed outcome If the outcome distribution is heavily right-skewed, how would you choose between ordinary linear regression on the raw outcome $y$ and linear regression on $\log(y)$? What are the downsides of using a log transform? ```hint Frame it as additive vs. multiplicative Ask whether the business question is about an **absolute** change (dollars, minutes) or a **percentage / multiplicative** change — that, plus residual diagnostics, drives the choice. Recall what $\beta_1$ means in $\log(y)=\beta_0+\beta_1 x+\epsilon$. ``` ```hint Pitfalls to name Consider what happens at $y \le 0$, why the log compresses the heavy upper tail, and why $E[\log Y] \ne \log E[Y]$ (Jensen's inequality) makes back-transformed predictions tricky. ``` #### What This Part Should Cover - A decision rule grounded in the **estimand** (absolute / arithmetic-mean effect vs. multiplicative / percentage effect), not merely "the data is skewed," reinforced by residual diagnostics and variance behavior. - The log-model interpretation: a unit change in $x$ multiplies $y$ by $e^{\beta_1}$, i.e. a $100(e^{\beta_1}-1)\%$ change, $\approx 100\beta_1\%$ for small $\beta_1$. - The downsides: undefined at $y\le 0$ (and why $\log(1+y)$ is a compromise), the estimand shift to the geometric mean, retransformation / Jensen bias, and compression of high-value users. - Recognition that skew in $y$ alone does not invalidate OLS for the conditional mean — it mainly affects efficiency and naive SEs. ### Part 3 — R-squared near zero: just add covariates? If a regression model has an $R^2$ close to 0, can you simply add more covariates to improve performance? What are the risks, and what would you do instead? ```hint In-sample vs. generalization Adding any covariate weakly *increases* in-sample $R^2$ mechanically — so reach for adjusted $R^2$ / cross-validation, and think about overfitting and multicollinearity rather than raw covariate count. ``` ```hint Not every control is a good control Distinguish a noisy-but-correct model from a misspecified one, and beware variables downstream of the outcome or treatment — leakage, mediators, and colliders can hurt rather than help. ``` #### What This Part Should Cover - The mechanical fact that in-sample $R^2$ weakly increases with *any* added covariate (even noise), so training $R^2$ is the wrong target — adjusted $R^2$ / cross-validation / held-out error are. - The risks of piling on covariates: overfitting, multicollinearity, bad controls (mediators / colliders), data leakage, and spurious significance. - The insight that a low $R^2$ can be perfectly correct — individual outcomes are noisy, and an unbiased treatment-effect estimate does not require high $R^2$. - Better moves: improve feature quality over quantity, model nonlinearity / interactions, and in causal work include pre-treatment confounders and exclude post-treatment variables. ### Part 4 — Lasso vs. Ridge when R-squared is already promising If a model already has a promising $R^2$, should you reach for Lasso or Ridge regression? Why or why not? How would you evaluate whether regularization actually helps? ```hint What each penalty buys you Contrast the $\ell_1$ penalty (sparsity / feature selection) with the $\ell_2$ penalty (shrinkage, stability under multicollinearity). A good $R^2$ does not rule out high variance or correlated predictors. ``` ```hint How to decide The deciding evidence is **out-of-sample**: tune $\lambda$ by cross-validation and compare held-out error (RMSE / MAE / out-of-sample $R^2$), not training fit. ``` #### What This Part Should Cover - Why a promising $R^2$ does **not** rule out regularization (many predictors, multicollinearity, or high variance can still hurt generalization). - A correct contrast of the $\ell_1$ penalty (sparsity / feature selection, arbitrary pick within correlated groups) versus the $\ell_2$ penalty (shrinkage, stability under multicollinearity, rarely exact zeros), plus Elastic Net as the middle ground. - Out-of-sample evaluation: CV-tuned $\lambda$ (and $\alpha$), train/validation/test discipline, appropriate metrics (RMSE / MAE / out-of-sample $R^2$), and coefficient stability. - Operational hygiene: standardize features before penalizing, do not penalize the intercept, and tie the final choice to the downstream business metric. ### What a Strong Answer Covers These dimensions span all four parts: - A consistent through-line that **out-of-sample evidence**, not in-sample fit, decides model quality — applied to covariate selection (Part 3) and regularization (Part 4) alike. - Clean separation of **statistical** vs. **practical** vs. **causal** significance, carried from the coefficient / p-value reading (Part 1) into the bad-control discussion (Part 3). - Always tying the modeling choice back to the **estimand and the decision** (absolute vs. relative effect; prediction vs. causal inference), rather than reaching for a technique because it sounds advanced. ### Follow-up Questions - For the log model, you fit on $\log(y)$ but stakeholders need predictions in dollars. How would you correct the retransformation bias and produce an unbiased prediction back on the original dollar scale? - Many product outcomes are zero-inflated (most users spend $\$0$). Would a two-part / hurdle model or a GLM (e.g., gamma or Tweedie) be more appropriate than log-OLS, and why? - If the treatment is randomized, why might adding pre-treatment covariates still be worthwhile even when $R^2$ stays low? - How do you choose between Lasso and Ridge when predictors are both numerous *and* highly correlated?