Pub-Sub System Design

Design a publication service for a hypothetical application that sends the same event stream to three independent uses:

The publisher should not have to send three copies itself or wait for all three consumers to finish. Use the preceding lessons to define how the broker stores publications and tracks each subscription.

Work through one publication

Draw its path from producer acknowledgement to the three subscriptions. Mark the durable state at each step. Explain how a consumer resumes after a crash and why its progress must not advance another subscription’s position.

Choose an ordering domain. If events about one entity must remain ordered, show how that entity maps to a partition. Explain what parallelism remains for unrelated entities and what the design does not promise across partitions.

Change the conditions

First, let analytics stop consuming while notifications continue. Calculate the storage growth using your assumed event rate and average message size, then state when retention expires. Explain what analytics sees when it returns with an offset older than the retained log.