[BLOG POST] Streaming compression for RDB and replication in Valkey 9.2

[sarthakaggarwal97]

New Blog Post

What is your proposed topic?

I’d like to write about the streaming compression work added across:

• Streaming Compression support for RDB valkey#3531
• Streaming Compression support for fullsync valkey#4075
• Streaming Compression support for Replication valkey#3853
• ZSTD Support for Streaming Compression  valkey#3798

Together, these changes add LZ4 and Zstd Streaming Compression for RDB generation, full synchronization, and steady-state replication.

The post would explain:

• why replication bandwidth can become expensive for large datasets and cross-zone or cross-region deployments;
• how rdbcompression and repl-compression control the different paths;
• how codec negotiation and plaintext fallback preserve compatibility with older or opted-out replicas;
• how compression works with partial writes, replication offsets, partial resynchronization, diskless sync, and multiple replicas;
• how operators can enable the feature and observe its effectiveness;
• benchmark results, including the workload and methodology used to produce them.

Who is writing this blog post?

Sarthak Aggarwal (@sarthakaggarwal97). I’m also happy to incorporate feedback from the contributors and reviewers of the three pull requests.

What is your ideal publishing date?

At or shortly after the Valkey 9.2 release.

Is this blog post dependent on something else?

The implementation has been merged. Publication should be coordinated with the Valkey 9.2 release so readers can use the feature in an available release.

The related documentation work is tracked in:

• Document RDB and replication compression behavior valkey-doc#487

[hpatro]

Personal opinion: We shouldn't make a design document out of this. Rather take some use case and explain how this feature is much better/faster/compact. Might be more effective for a regular reader.