V1 [OPS] Data checksums on the write path

[austin-barrington]

Context

WAL entries have no payload checksum. If RocksDBs internal block checksum is the only integrity protection, silent bit-rot or storage-level corruption could go undetected until a query returns wrong results. The sha2 and crc32fast crates are already in Cargo.toml but are not used for data integrity.

Deliverables

• Add a CRC32C or xxHash64 checksum to every WAL entry payload.
• The checksum should cover the serialized point batch (not the RocksDB key, which RocksDB already protects).
• On WAL read (flush, replication, backup), verify the checksum and log + increment a metric on mismatch.
• Expose hyperbytedb_wal_checksum_mismatches_total counter.
• Include checksum verification in the hyperbytedb check subcommand (see ticket for WAL integrity tool).
• The format change should be backward-compatible: old entries without checksums are read without verification.

References

• hyperbytedb/Cargo.toml:89-90 — sha2 and crc32fast already present
• WAL reading code in flush and replication paths
• RocksDB internal checksum is per-block, not per-logical-entry

Priority

P1 — Data integrity