Build a bigram next-word predictor with weighted sampling

Q: Build a bigram next-word predictor with weighted sampling

This question evaluates understanding of basic probabilistic language modeling—specifically bigram/first-order Markov models and weighted sampling for next-word prediction—within the Machine Learning (natural language processing) domain, emphasizing practical implementation skills alongside conceptual scalability trade-offs.

Q: How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

Question

You are given a training set of token sequences (sentences), for example:

[["a","b","c"],
 ["a","s","d"]]

Train a simple next-word prediction model that, for each word w , counts which words most frequently appear immediately after w (a bigram / 1st-order Markov model).
At inference time, given a current word w , output a random next word sampled proportionally to the observed counts after w (i.e., weighted by frequency).
Discuss what you would do if the vocabulary and/or number of distinct next-words per token is very large (memory and latency constraints).

Build a bigram next-word predictor with weighted sampling

Quick Overview

Solution

Comments (0)