Evaluate College Impact on Income: Address Bias and Validity
Quick Overview
Evaluates whether simple regression can estimate college impact on income from observational data. Strong answers separate descriptive and causal goals, address sampling bias, model validity, confounding, and alternatives.
Evaluate College Impact on Income: Address Bias and Validity
Company: Google
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
##### Scenario
Studying whether attending college affects income using data from 1,000 Mountain View residents.
##### Question
Is fitting a simple linear regression with a binary college variable appropriate? Discuss sampling bias, model validity, and propose better analytic alternatives.
##### Hints
Consider external validity and two-sample t-tests or multivariate models.
Quick Answer: Evaluates whether simple regression can estimate college impact on income from observational data. Strong answers separate descriptive and causal goals, address sampling bias, model validity, confounding, and alternatives.
Evaluating College Impact on Income with Observational Data
You have an observational, cross-sectional dataset of 1,000 adult Mountain View residents. The outcome is individual annual pre-tax income. The exposure is a binary indicator for completing a 4-year college degree.
Is fitting a simple linear regression of income on the college indicator appropriate? Discuss sampling bias, model validity, and better alternatives for descriptive and causal goals.
Constraints & Assumptions
Distinguish descriptive association from causal impact.
Treat the dataset as observational, not randomized.
Consider external validity because the sample is limited to Mountain View residents.
Income may be skewed, heavy-tailed, or censored.
Clarifying Questions to Ask Guidance
How was the sample of Mountain View residents collected?
Does income include wages, bonuses, capital gains, and self-employment income?
What confounders are available, such as age, occupation, experience, industry, immigration status, or family background?
Is the goal description, prediction, or causal inference?
Part 1 - Sampling Bias and External Validity
Discuss sampling bias and whether results generalize.
What This Part Should Cover Guidance
Explain that Mountain View residents may not represent the broader population.
Consider selection into living in Mountain View, labor-market composition, tech concentration, and cost of living.
Discuss nonresponse, survivorship, and sample-frame bias.
Limit conclusions to the sampled population unless assumptions support generalization.
Part 2 - Model Validity and Assumptions
Assess the simple linear regression model.
What This Part Should Cover Guidance
Explain that the simple coefficient estimates a mean income difference in the sample, not necessarily a causal effect.
Discuss skewness, outliers, heteroskedasticity, robust standard errors, log income, and diagnostics.
Identify confounding from ability, family background, occupation, age, experience, and selection.
Consider functional form and subgroup heterogeneity.
Part 3 - Better Alternatives
Suggest alternatives for descriptive and causal goals.
What This Part Should Cover Guidance
For description, use robust comparisons, stratified summaries, adjusted regressions, and visualizations.
For causal inference, consider matching, weighting, regression adjustment, doubly robust methods, IV, regression discontinuity, or natural experiments if assumptions are credible.
Include sensitivity analysis for unobserved confounding.
State limitations clearly.
Follow-up Questions Guidance
Why might log income be more appropriate than raw income?
What variable would you most want to control for?
What would make an instrumental variable credible in this setting?