Design a Software Load Balancer

# Design a Software Load Balancer Design a software load balancer at either layer 4 or layer 7, and explain why that layer fits the chosen requirements. Cover backend selection, health checks, connection lifecycle, high availability, and safe configuration changes. ### Constraints & Assumptions - Backends can fail partially or become slow without closing every connection. - The load balancer itself must not be a single point of failure. - Existing long-lived connections may outlive a deployment or backend drain. ### Clarifying Questions to Ask - What protocols, connection duration, traffic volume, and latency targets apply? - Is routing based only on network endpoints or on HTTP attributes? - Are session affinity, TLS termination, or geographic routing required? ```hint Distinguish readiness from liveness A backend can be alive yet unsafe to receive new requests, and health signals can be stale or noisy. ``` ### What a Strong Answer Covers - A justified L4 or L7 boundary and data path. - Selection algorithms, capacity weighting, and overload behavior. - Active and passive health, hysteresis, and connection draining. - Redundant balancers, configuration rollout, observability, and failure modes. ### Follow-up Questions - How would you prevent a retry storm during a regional failure? - When would consistent hashing be preferable to least-connections routing?