Explain SHAP, Word2vec, Random Forests, Normalization, and Transformers

Explain the following machine learning fundamentals. Use definitions, mechanisms, and small conceptual examples; no dataset or company-specific application is assumed. ### Part 1 — SHAP explanations What does a SHAP value explain for one prediction? Describe the baseline, how feature contributions relate to the model output, and why correlated features complicate interpretation. #### What This Part Should Cover - Contributions relative to a stated background distribution and output scale. - The difference between explaining the model and establishing a causal effect. ### Part 2 — Word2vec Compare the continuous bag-of-words and skip-gram training objectives. Explain why their learned vectors can encode useful similarities and identify a limitation of one vector per word. #### What This Part Should Cover - Which words are inputs and targets in each objective, and how context supplies a training signal. - The difference between a static word vector and a representation that changes with the sentence. ### Part 3 — Random forests How does a random forest differ from a single decision tree? Explain the roles of training-sample randomness and feature randomness, and how predictions are combined. #### What This Part Should Cover - Bootstrap sampling and random feature subsets at candidate splits. - Why averaging helps most when the constituent trees' errors are not perfectly correlated. ### Part 4 — L1 and L2 normalization Define L1 and L2 normalization of a nonzero feature vector and explain their effect on its magnitude and direction. What should happen for a zero vector? Here, normalization means rescaling a vector to unit norm. It does not mean adding L1 or L2 penalties to a training loss. #### What This Part Should Cover - The denominator for each norm and an explicit zero-vector policy. - The distinction between normalizing each sample, standardizing features, and regularizing model parameters. ### Part 5 — Bias and variance Explain the bias-variance tradeoff using expected squared prediction error. Describe how underfitting and overfitting appear in training and validation behavior, and what interventions can address each. #### What This Part Should Cover - Bias, variation over training samples, and irreducible noise under the squared-error assumptions. - Why more data, more model capacity, and stronger regularization affect different sources of error. ### Part 6 — Transformer models Explain scaled dot-product self-attention and the roles of multiple heads, positional information, and attention masking. Contrast token representations in a transformer with Word2vec embeddings. #### What This Part Should Cover - Queries, keys, values, and normalized attention weights. - Position sensitivity, causal visibility, and the cost of dense attention as sequence length grows. ### What a Strong Answer Covers - Correct distinctions between an explanation method, a representation-learning objective, an ensemble, and preprocessing. - Consistent notation for norms, model outputs, and prediction error, with assumptions stated where a formula depends on them. - Connections grounded in these topics, such as correlated trees limiting variance reduction and contextual attention resolving the limitations of static word vectors. ### Follow-up Questions - If two input features are highly correlated, how could changing a SHAP background or dependence assumption affect the explanation? - Why can adding more trees to a random forest stop helping, while adding more training examples may still help?