Implement Gaussian Naive Bayes Training and Prediction from Scratch

Implement a Gaussian Naive Bayes classifier from scratch, in plain Python or NumPy and without a machine learning library: training estimates per-class statistics, and prediction picks the most probable class for each example. The assessment supplied the formulas; the version below states them. For training examples with feature vectors $\mathbf{x}$ of length $d$ and class labels: - Class prior: $P(c) = \frac{n_c}{n}$, where $n_c$ is the number of training examples in class $c$ and $n$ the total. - Per class $c$ and feature $j$: the mean $\mu_{c,j}$ and the variance $\sigma_{c,j}^2$ of feature $j$ over the examples of class $c$, with the variance dividing by $n_c$. - Likelihood of one feature value: $p(x_j \mid c) = \frac{1}{\sqrt{2\pi\sigma_{c,j}^2}}\exp\left(-\frac{(x_j - \mu_{c,j})^2}{2\sigma_{c,j}^2}\right)$. - Naive assumption, features independent given the class: $P(c \mid \mathbf{x}) \propto P(c)\prod_{j=1}^{d} p(x_j \mid c)$. - Prediction: the class with the largest posterior. ```python class GaussianNaiveBayes: def fit(self, X: list[list[float]], y: list[int]) -> "GaussianNaiveBayes": ... def predict(self, X: list[list[float]]) -> list[int]: ... def predict_proba(self, X: list[list[float]]) -> list[list[float]]: ... # columns in sorted label order ``` A check for your implementation: ```text training data class 0: (1, 3), (2, 2), (3, 1) class 1: (6, 6), (7, 8), (8, 7) queries: (3, 4) and (5, 4.5) expected predictions: class 0, then class 1 ``` ```hint Products of small numbers Multiplying many densities together quickly underflows to zero. Find a form of the comparison that stays numerically safe. ``` ```hint A feature that never varies If a feature has the same value for every training example of a class, its variance is zero. Decide what the density formula does then, and how to keep it finite. ``` ### Clarifying Questions - How should a zero variance be handled, for example a feature that is constant within one class? - How should ties between classes be broken? - Are class labels arbitrary integers, and should `predict_proba` be implemented or only `predict`? ### What a Strong Answer Covers - A correct fit: priors, and per-class means and variances - Prediction in log space, with an argmax over classes and a deterministic tie rule - Zero-variance handling with a small smoothing term, and validation of the input - Vectorized prediction and its complexity - Correct results on the check, and a clear statement of the model's assumptions ### Follow-up Questions - Turn the log scores into class probabilities without overflow or underflow. - Where does the independence assumption fail, and why does the classifier often still choose the right class? - How would you update the model incrementally as new labeled batches arrive? - How does Gaussian Naive Bayes relate to linear and quadratic discriminant analysis?