How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

What difficulty level is this interview question?

This is a medium difficulty Machine Learning question, commonly asked during Onsite rounds at OpenAI.

What role is this question designed for?

This question is commonly asked for Machine Learning Engineer candidates at OpenAI during technical interviews.

Compute entropy and implement 1-NN | OpenAI Interview Question

Quick Overview

This question evaluates skills in probabilistic modeling and numerical methods (entropy computation from logits emphasizing numerical stability and streaming aggregation) as well as vectorized linear-algebra implementation and model interpretation (1-nearest-neighbor using L2/L1 distances and its expression as a neural-network forward pass) within the Machine Learning domain. It is commonly asked to assess a candidate's ability to implement efficient, numerically robust algorithms and to connect practical implementations to neural-network abstractions, testing both practical application and conceptual understanding.

You are given two machine-learning coding tasks.

Entropy from logits Given a vector of real-valued logits z = [z1, z2, ..., zn], define p = softmax(z). Compute the categorical entropy H(p) = -sum_i p_i log(p_i). Your implementation should be numerically stable and should use the log-sum-exp trick. Then extend the method to the case where the logits arrive as a stream of blocks, so the full vector may not fit in memory. Describe how to compute the same entropy online.
Vectorized 1-nearest neighbor and neural-network interpretation Given training examples X_train with shape (n, d), labels y_train with shape (n,), and query examples X_query with shape (m, d), implement 1-nearest-neighbor prediction using L2 distance. Do not use Python loops over examples; use NumPy-style vectorization.

Then express L2 1-nearest-neighbor inference as a neural-network forward pass using a linear layer followed by softmax. Finally, explain how the construction changes if the distance metric is L1 instead of L2.

Quick Overview

You are given two machine-learning coding tasks.

Entropy from logits Given a vector of real-valued logits z = [z1, z2, ..., zn], define p = softmax(z). Compute the categorical entropy H(p) = -sum_i p_i log(p_i). Your implementation should be numerically stable and should use the log-sum-exp trick. Then extend the method to the case where the logits arrive as a stream of blocks, so the full vector may not fit in memory. Describe how to compute the same entropy online.
Vectorized 1-nearest neighbor and neural-network interpretation Given training examples X_train with shape (n, d), labels y_train with shape (n,), and query examples X_query with shape (m, d), implement 1-nearest-neighbor prediction using L2 distance. Do not use Python loops over examples; use NumPy-style vectorization.

Compute entropy and implement 1-NN

Quick Overview

Solution

Comments (0)

Compute entropy and implement 1-NN

Quick Overview

Solution

Comments (0)