Quick Overview

Tokenize a string against a fixed vocabulary by repeatedly taking the longest vocabulary entry that matches at the current position, and return the token IDs in order. It tests exact greedy semantics, efficient prefix matching over up to 100,000 characters, and careful handling of overlapping vocabulary entries.

Greedy Longest-Match Tokenization of a String Against a Fixed Vocabulary

Company: Anthropic

Role: Software Engineer

Category: Coding & Algorithms

Difficulty: medium

Interview Round: Onsite

Overview: Tokenize a string against a fixed vocabulary by repeatedly taking the longest vocabulary entry that matches at the current position, and return the token IDs in order. It tests exact greedy semantics, efficient prefix matching over up to 100,000 characters, and careful handling of overlapping vocabulary entries.

Loading…