Data Serialization in Distributed Systems

Compare how JSON, XML, Protocol Buffers, Thrift, and Avro represent data and manage schema changes. You will identify what a reader needs to decode a message, then reason about old and new producers and consumers running at the same time.

In a distributed system, services must constantly exchange information to function.

This communication breaks down if a service written in Python cannot understand data sent from a service written in Java. This is where data serialization comes in. It translates data structures into a format that can be stored or transmitted and then reconstructed later.

A shared encoding is only part of the contract. Both sides must also agree on field meaning, units, missing values, and compatibility rules.

In real-world System Design, choosing the right format directly impacts performance, data size, and how easily our system can evolve. In interviews, showing awareness of these trade-offs is just as important. To understand these trade-offs, we’ll divide the most common formats into two broad categories.

First compare textual representations with schema-based binary encodings. Measure message size and encoding cost for your workload instead of assuming that every binary format is smaller or faster.

Textual data formats