Design of a Distributed Messaging Queue: Part 2

The broker layer stores messages and coordinates their ownership. Compare a designated-primary arrangement with one where a contacted host coordinates a request using a placement map. In both cases, routing a request is only the first step toward durable acceptance or safe delivery.

Previously, we discussed frontend servers and metadata services. This lesson focuses on the backend service that stores queues and messages.

Backend service

The backend service receives a request routed from the frontend, identifies the queue’s storage replicas, and executes the queue’s replication and delivery protocol. The two arrangements below describe request coordination and placement. They are not complete replication algorithms on their own:

1. Primary-secondary model

2. A cluster of independent hosts

This lesson calls within-cluster placement “internal” management and across-cluster placement “external” management. These are roles in the proposed architecture, not universal broker product names:

Internal vs. External Cluster Managers