Quick Overview

Implement a case-sensitive longest-match tokenizer that emits token IDs with the exact consumed text. Support per-character or coalesced unknown runs, validate the vocabulary, preserve Unicode code points, and compare direct matching with a trie.

Implement a Longest-Match Tokenizer

Company: Anthropic

Role: Software Engineer

Category: Coding & Algorithms

Difficulty: hard

Interview Round: Onsite

Overview: Implement a case-sensitive longest-match tokenizer that emits token IDs with the exact consumed text. Support per-character or coalesced unknown runs, validate the vocabulary, preserve Unicode code points, and compare direct matching with a trie.

Read the full Anthropic Software Engineer interview experience this question came from

Loading…