Encode Categorical Variables Across Model Families

# Encode Categorical Variables Across Model Families A nominal feature has `K` categories. Explain how many dummy variables you would use in a linear regression, a random forest, and a basic feed-forward neural network. Discuss intercepts, multicollinearity, unseen categories, high cardinality, and model-native alternatives. ### Constraints & Assumptions - Categories have no natural numeric order. - The transformation must be identical in training and serving. - Some production rows may contain missing or previously unseen categories. - The question asks about ordinary dummy encoding, while alternatives may be discussed. ### Clarifying Questions to Ask - Does the linear model include an intercept? - Does the tree library support native categorical splits? - Is the neural network allowed to learn embeddings? - How large and stable is `K`? ### What a Strong Answer Covers - `K - 1` indicators with an intercept for an identifiable linear model - Why trees do not require a dropped reference column - Full one-hot input or embeddings for neural networks - Unknown-category policy, leakage, sparsity, and high-cardinality trade-offs ### Follow-up Questions 1. What changes in linear regression if the intercept is removed? 2. Why is integer label encoding risky for nominal categories? 3. When would target encoding be useful, and how would you prevent leakage? ```hint Ask whether linear dependence matters to the learner The reference-category rule addresses rank in a linear design matrix; split-based and nonlinear models have different reasons for retaining or replacing all indicators. ```