Explain core probability and ML statistics concepts
Quick Overview
This question evaluates core probability and machine learning statistics competencies including probability basics, descriptive statistics, correlation versus independence, linear regression and regularization, and dimensionality reduction (PCA), falling under the Machine Learning domain for Data Scientist roles and testing both conceptual understanding and practical application. Such multi-part theory questions are commonly asked to probe understanding of fundamental statistical concepts and modeling assumptions, ensuring the candidate can reason about uncertainty, estimator properties, multicollinearity, and eigenstructure without relying solely on implementation details.
Answer the following short theory questions (you may use equations and brief examples):
Probability
-
You roll two fair six-sided dice.
- What is the probability that one die shows a strictly larger value than the other (i.e., the two values are different and one is greater)?
- What is the probability that a specific die (e.g., the first die) is strictly larger than the other?
Basic statistics
- Define mean and variance of a random variable.
- Is the usual sample variance estimator “biased”? If yes, what correction makes it unbiased?
Correlation vs. independence
- Let be (marginally) normally distributed with . Are and necessarily independent? State the condition under which zero correlation does imply independence.
Linear regression / OLS
- Explain linear regression and list common assumptions behind Ordinary Least Squares (OLS).
- Write the closed-form OLS estimator.
- Why is (multi)collinearity a problem in regression? How can you detect it, and how can you mitigate it?
- Briefly explain Ridge and Lasso regression and how they relate to collinearity.
PCA
- Explain Principal Component Analysis (PCA).
- What are eigenvalues and eigenvectors in this context, and what do they represent?
- List key limitations of PCA.