Skip to content

V1 [OPS] WAL integrity check + repair tool #70

Description

@austin-barrington

Context

When the WAL becomes corrupted (unclean shutdown, disk error), the only recovery options are "restore from backup or delete the wal_dir" (docs/user-guide/troubleshooting.md:67). There is no hyperbytedb check or hyperbytedb repair subcommand to:

  • Scan the WAL for corruption
  • Verify RocksDB consistency
  • Recover as much data as possible before discarding

Deliverables

hyperbytedb check subcommand

  • Verify RocksDB WAL integrity (open, iterate all entries, check for read errors)
  • Verify metadata store integrity
  • Verify chDB data directory accessibility (list tables, check for obvious corruption)
  • Report summary: entry count, last sequence, any errors found
  • Exit code 0 if clean, non-zero if issues found

hyperbytedb repair subcommand

  • Attempt to salvage readable entries from a corrupted WAL
  • Rebuild metadata from surviving data where possible
  • Create a backup of the corrupted directory before modifying
  • Report what was recovered and what was lost

Checksums

Optionally (if WAL checksums are implemented per #XX [data checksums ticket]), the check tool should also verify payload checksums and report mismatches.

References

  • docs/user-guide/troubleshooting.md:67 — "restore from backup or delete"
  • src/main.rs:32-46 — current subcommands: serve, backup, restore
  • RocksDB DB::Open() failure modes

Priority

P1 — Data integrity, production requirement

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    Status
    Ready

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions