Compare Gradient Methods, Linear Regression, and Feature Encoding
Company: Molocoads
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: medium
Interview Round: Onsite
# Compare Gradient Methods, Linear Regression, and Feature Encoding
Answer the following machine-learning fundamentals questions as one coherent discussion. State any assumptions about dataset size, feature types, and optimization objective.
### Clarifying Questions to Ask
- Is “SGD” being used for one-example updates or for small mini-batches?
- Does the linear-regression discussion assume ordinary least squares and independent observations?
- Are the features numerical, categorical, ordinal, cyclic, or a mixture?
### Part 1 — Batch Gradient Descent and Stochastic Updates
Compare batch gradient descent with stochastic or mini-batch gradient descent. Explain update frequency, computational cost, gradient variance, convergence behavior, and when each is practical.
#### What This Part Should Cover
- The amount of data used for one gradient estimate.
- The trade-off between stable directions and cheap, noisy updates.
- Learning-rate schedules, batching, and hardware utilization.
### Part 2 — Linear Regression
Explain the objective of linear regression, the meaning of its coefficients, common assumptions behind statistical inference, and ways to diagnose a poor fit.
#### What This Part Should Cover
- Squared-error minimization and the prediction equation.
- Residual checks, multicollinearity, outliers, and regularization.
- The distinction between predictive usefulness and causal interpretation.
### Part 3 — Feature Encoding
Describe how you would encode numerical, nominal categorical, ordinal, and cyclic features for a linear model. Explain how leakage and unseen categories affect the preprocessing design.
#### What This Part Should Cover
- Scaling numerical features where optimization or regularization makes it useful.
- One-hot, ordinal, and sine/cosine encodings tied to feature semantics.
- Training-only fitting, stable schemas, and unknown-category handling.
### What a Strong Answer Covers
- Precise comparisons rather than treating one optimizer or encoding as universally best.
- Connections between optimization behavior, model assumptions, and preprocessing.
- Concrete failure modes such as leakage, unstable gradients, collinearity, and arbitrary category order.
- Evaluation on held-out data with preprocessing fitted only on the training partition.
### Follow-up Questions
- Why can a larger mini-batch require a different learning rate?
- How does L2 regularization change a linear-regression solution?
- When is target encoding appropriate, and how would you prevent leakage?
Overview: Compare batch, stochastic, and mini-batch gradient methods alongside linear-regression assumptions and feature encoding choices. The answer connects optimizer variance, residual diagnostics, regularization, categorical and cyclic representations, unknown categories, and training-only preprocessing to practical model evaluation.
Read the full Molocoads Machine Learning Engineer interview experience this question came from