Client-side Load Balancer for Twitter

Introduction

The previous diagram shows load balancing as a logical component. It need not be one physical machine. This lesson examines client-side balancing for internal RPCs, using Twitter’s 2019 deterministic-aperture article as a historical example. The problem is how to distribute both requests and connections across a large backend fleet.

Twitter’s design history

Twitter’s move from a Ruby on Rails monolith toward independently deployed services created a need for internal service discovery and routing. The source highlights several motivations for separating services:

• Codebase contention: A large number of developers working on the same codebase made it difficult to update individual services.

• Cascading failures: An upgrade in one service could break others.

• Resource coupling: Components with different CPU, memory, and scaling needs can compete when deployed together. Separation helps only if the added network and operating cost is justified.

• Complex recovery: Recovering from failures was slow and difficult.

Once services have many instances, each caller needs a way to select a healthy destination without maintaining an excessive number of connections.

Client-side load balancing