Implement Gradient Descent and Explain Adam Updates

Write the core gradient-descent training loop, then explain the update equations and state needed for Adam. ### Constraints and Clarifications No particular objective, model architecture, dataset shape, or stopping threshold is specified. Use a differentiable objective f(theta) with an available gradient as the interface for the exercise. State the learning rate and iteration budget explicitly. Any scalar example is an illustration rather than a supplied interview dataset. ### Part 1 — Gradient Descent Give pseudocode that updates parameters using the current gradient. Explain the learning rate, termination, and how to diagnose a divergent or stationary run. #### What This Part Should Cover Use the gradient at the current parameters, distinguish an update budget from convergence, and provide a small illustrative calculation. ### Part 2 — Adam Specify the first and second moment updates, initialization, bias correction, and parameter update. Explain how the algorithm differs from plain gradient descent. #### What This Part Should Cover Keep optimizer state across steps, use elementwise squares and square roots, and describe the roles of beta1, beta2, and epsilon. ### What a Strong Answer Covers Executable-level pseudocode for the basic loop, correct Adam formulas with step indexing, and practical reasoning about step size, gradients, numerical problems, and stopping. ### Follow-up Questions - How do full-batch and stochastic gradients change the behavior of these loops? - Why does zero initialization of Adam's moments motivate bias correction?