System Design: The Distributed Messaging Queue

A message queue separates accepting work from completing it. This chapter designs the buffer, ownership, acknowledgement, and retry rules that let producers and consumers run at different rates.

What is a messaging queue?

A messaging queue acts as an intermediary buffer between producers and consumers.

Producers create messages and place them in the queue. Consumers retrieve and process these messages. Multiple producers and consumers can interact with the queue simultaneously.

The following illustration shows two applications interacting via a single messaging queue:

Key components of a messaging queue

A messaging system consists of four main parts:

Producers