Contrast LSTM and Transformer for long sequences

Contrast LSTM and Transformer for long sequences

Company: TikTok

Role: Data Scientist

Category: Machine Learning

Difficulty: hard

Interview Round: Technical Screen

Overview: This question evaluates understanding of sequence-model architectures and system-level trade-offs for long-context autoregressive language models, covering computational and activation memory complexity, positional encoding and extrapolation behavior, and streaming inference and KV-cache management within GPU constraints.