Skip to content

Fix audit findings: append-log value cap, multipart abort, stream-limit detection - #29

Merged
jaredLunde merged 1 commit into
mainfrom
jared/audit-fixes
Oct 3, 2026
Merged

jaredLunde merged 1 commit into
mainfrom
jared/audit-fixes

Conversation

@jaredLunde

Copy link
Copy Markdown
Contributor

Summary

  • Append-log value cap (data-loss class). The writer accepted values up to u32::MAX; replay treated value_len > 16 MiB as a CRC mismatch, so after one 17 MiB value every apply/load returned Corrupted (reproduced: the compaction triggered by the write itself fails). Writer and reader now share MAX_VALUE_LEN = 64 MiB (NATS's payload ceiling); the writer refuses with InvalidFormat before writing anything. Regression test round-trips a value at the limit and checks a larger one leaves the fold loadable.
  • Abort failed multipart uploads. Any error after put_multipart dropped WriteMultipart without abort(); S3/GCS keep those parts (billed, unlisted) until aborted. The upload now drains parts itself and aborts on any failure; finish() then only sends the completion, which object_store aborts on failure.
  • Stream-limit detection. Match err_code == 10027 (JSMaximumStreamsLimitErr) or the text case-insensitively, instead of an exact substring. A false positive is harmless: StreamLimit re-checks the bucket.
  • Minor: RocksDB import verify-open uses sync: true like fjall and its own export; resume_window_ok docs name resume_start_sequence as the only source of the start sequence; import docs cover a failed open after the rename; ARCHITECTURE.md documents the value cap, the length-corruption limitation, and both new failure modes.

Test plan

  • CI: fmt, clippy -D warnings, docs, full feature matrix
  • New largest_value_round_trips_and_larger_is_refused_before_writing
  • New raw_create_stream_limit_by_code_or_reworded_text

🤖 Generated with Claude Code

https://claude.ai/code/session_0152kQKDdP8XRhoeqisJYpWr

…it detection

The append-log writer accepted values up to u32::MAX while replay called
anything over 16 MiB corruption, so one large value made the fold
unloadable. Both sides now share a 64 MiB cap (NATS's payload ceiling) and
the writer refuses before writing. Failed multipart uploads are aborted so
S3/GCS don't keep billed orphan parts. The Synadia stream-limit reply is
recognized by err_code 10027 or case-insensitive text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0152kQKDdP8XRhoeqisJYpWr
@jaredLunde jaredLunde mentioned this pull request Oct 3, 2026
@jaredLunde
jaredLunde merged commit 6fceb50 into main Oct 3, 2026
1 check passed
@jaredLunde
jaredLunde deleted the jared/audit-fixes branch October 3, 2026 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant