Skip to content

perf(l1): enable unordered writes for trie node column families#6531

Open
dicethedev wants to merge 1 commit intolambdaclass:mainfrom
dicethedev:perf/rocksdb-unordered-writes-trie
Open

perf(l1): enable unordered writes for trie node column families#6531
dicethedev wants to merge 1 commit intolambdaclass:mainfrom
dicethedev:perf/rocksdb-unordered-writes-trie

Conversation

@dicethedev
Copy link
Copy Markdown

Motivation
Trie node writes during block execution do not require strict ordering guarantees. RocksDB's default ordered-write path adds unnecessary synchronization overhead for these CFs.

Description
Adds cf_opts.set_unordered_write(true) to ACCOUNT_TRIE_NODES and STORAGE_TRIE_NODES. Unordered writes allow RocksDB to skip WAL sequencing constraints, potentially increasing write throughput for state-heavy workloads.

Not applied to other CFs where ordering or consistency guarantees may matter (e.g. HEADERS, BODIES, RECEIPTS).

Checklist

  • Updated STORE_SCHEMA_VERSION (crates/storage/lib.rs) if the PR
    includes breaking changes to the Store requiring a re-sync.

Closes #5937

@dicethedev dicethedev requested a review from a team as a code owner April 25, 2026 10:34
@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented Apr 25, 2026

Greptile Summary

This PR adds cf_opts.set_unordered_write(true) to the ACCOUNT_TRIE_NODES and STORAGE_TRIE_NODES column family configurations, aiming to skip WAL sequencing overhead for trie writes and improve write throughput.

  • The change is a no-op: unordered_write is a DBOptions field in RocksDB, not a ColumnFamilyOptions field. When set on options passed to a ColumnFamilyDescriptor, RocksDB silently ignores it — the value is never read during CF initialization. The intended performance gain will not be realized.
  • To actually enable this, it must be set on the top-level opts (line 32), which would apply it DB-wide (per-CF granularity is not supported). That path would also need reconciliation with enable_pipelined_write(true) and the PointInTime WAL recovery mode already configured.

Confidence Score: 3/5

Safe to merge in terms of stability (the option is ignored), but the claimed performance optimization is not achieved — the PR does not do what it intends.

There is one P1 finding: set_unordered_write(true) is silently a no-op when applied to CF-level options, so the PR's stated goal is not met. No crash or data-corruption risk from the current change, but shipping it as-is gives a false impression that the optimization is active.

crates/storage/backend/rocksdb.rs — specifically line 133 and the relationship between the CF-level cf_opts and the DB-level opts at line 32.

Important Files Changed

Filename Overview
crates/storage/backend/rocksdb.rs Adds set_unordered_write(true) to trie node CF options, but this is a no-op because unordered_write is a DB-level (DBOptions) setting in RocksDB that is silently ignored when applied to CF-level options in a ColumnFamilyDescriptor.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[RocksDB open] --> B[DB-level opts\nenable_pipelined_write=true\nwal_recovery_mode=PointInTime\nunordered_write=NOT SET]
    A --> C[CF loop]
    C --> D{CF name?}
    D --> |ACCOUNT_TRIE_NODES\nSTORAGE_TRIE_NODES| E[cf_opts\nset_unordered_write=true\n⚠️ DB-level option, silently ignored]
    D --> |HEADERS / BODIES / etc.| F[cf_opts\nstandard settings]
    E --> G[ColumnFamilyDescriptor]
    F --> G
    G --> H[open_cf_descriptors\nunordered_write flag dropped,\nnever applied to DB write path]
    B --> H
    H --> I[RocksDB running\nno unordered_write effect]
Loading
Prompt To Fix All With AI
This is a comment left during a code review.
Path: crates/storage/backend/rocksdb.rs
Line: 133

Comment:
**`unordered_write` is a DB-level option, not a CF-level option — this is a no-op**

`unordered_write` belongs to `DBOptions` in RocksDB, not `ColumnFamilyOptions`. When it is set on the `Options` object passed to a `ColumnFamilyDescriptor`, the flag is silently ignored during CF-open because RocksDB only reads the DB-level portion of the options from the primary DB-open call, not from individual CF descriptors (confirmed by the [RocksDB blog](http://rocksdb.org/blog/2019/08/15/unordered-write.html), which shows it set exclusively on `DBOptions`).

To actually take effect, `set_unordered_write(true)` must be called on the top-level `opts` object (line 32). However, doing so makes it a DB-wide setting — RocksDB has no mechanism to enable unordered writes selectively per column family. Additionally, enabling it at DB level would need verification against `enable_pipelined_write(true)` (line 60) and the `PointInTime` WAL recovery mode (line 54), which may require configuration adjustments.

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "perf(l1): enable unordered writes for tr..." | Re-trigger Greptile

cf_opts.set_min_write_buffer_number_to_merge(2);
cf_opts.set_target_file_size_base(256 * 1024 * 1024); // 256MB
cf_opts.set_memtable_prefix_bloom_ratio(0.2); // Bloom filter
cf_opts.set_unordered_write(true);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 unordered_write is a DB-level option, not a CF-level option — this is a no-op

unordered_write belongs to DBOptions in RocksDB, not ColumnFamilyOptions. When it is set on the Options object passed to a ColumnFamilyDescriptor, the flag is silently ignored during CF-open because RocksDB only reads the DB-level portion of the options from the primary DB-open call, not from individual CF descriptors (confirmed by the RocksDB blog, which shows it set exclusively on DBOptions).

To actually take effect, set_unordered_write(true) must be called on the top-level opts object (line 32). However, doing so makes it a DB-wide setting — RocksDB has no mechanism to enable unordered writes selectively per column family. Additionally, enabling it at DB level would need verification against enable_pipelined_write(true) (line 60) and the PointInTime WAL recovery mode (line 54), which may require configuration adjustments.

Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/storage/backend/rocksdb.rs
Line: 133

Comment:
**`unordered_write` is a DB-level option, not a CF-level option — this is a no-op**

`unordered_write` belongs to `DBOptions` in RocksDB, not `ColumnFamilyOptions`. When it is set on the `Options` object passed to a `ColumnFamilyDescriptor`, the flag is silently ignored during CF-open because RocksDB only reads the DB-level portion of the options from the primary DB-open call, not from individual CF descriptors (confirmed by the [RocksDB blog](http://rocksdb.org/blog/2019/08/15/unordered-write.html), which shows it set exclusively on `DBOptions`).

To actually take effect, `set_unordered_write(true)` must be called on the top-level `opts` object (line 32). However, doing so makes it a DB-wide setting — RocksDB has no mechanism to enable unordered writes selectively per column family. Additionally, enabling it at DB level would need verification against `enable_pipelined_write(true)` (line 60) and the `PointInTime` WAL recovery mode (line 54), which may require configuration adjustments.

How can I resolve this? If you propose a fix, please make it concise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enable unordered writes for State (RocksDB)

1 participant