SORT and SORT_RO should be read-only: stop converting listpack sorted sets to btree - #4698
rainsupreme wants to merge 7 commits into
Conversation
SORT and SORT_RO destructively converted a listpack-encoded sorted set to the full encoding (skiplist, now btree) before sorting. This dates back to the original Redis "SORT by nosort" change and was an implementation convenience, not a feature. With forkless save (valkey-io#4460) a background iterator may be reading an object's value memory while the main thread executes commands. Writes are gated by bgIteration_blockClientIfRequired(), which only blocks commands classified as writes. SORT_RO is READONLY, so the conversion freed the listpack from under a background reader. Rather than special-casing SORT_RO as a write (the PFCOUNT approach), make SORT genuinely read-only with respect to the key being sorted: handle the listpack encoding directly in the two places that consumed the btree (the BY nosort ordered walk, and the collect-all path), using the same lpSeek/zzlNext/zzlPrev idiom as ZRANGE. User-visible change: SORT / SORT_RO no longer changes the encoding of a small sorted set, as observable via OBJECT ENCODING. Tests: SORT and SORT_RO leave a listpack zset as listpack across ALPHA, BY nosort, LIMIT, BY pattern and GET; integer members (the listpack integer path) in both encodings; and a 50-member parity test asserting identical results between listpack and btree for the nosort/LIMIT edge cases. Signed-off-by: Rain Valentine <rsg000@gmail.com>
Reword the sort.c comment to describe the invariant (SORT does not modify the object being sorted, because a background iterator may be reading it) rather than prior behavior. Run the encoding-stability and parity tests over both listpack and btree, and over both SORT and SORT_RO, so the two encodings have the same coverage for ALPHA, BY nosort, LIMIT, BY pattern and GET. Signed-off-by: Rain Valentine <rsg000@gmail.com>
The three tests that force zset-max-ziplist-entries to 0 assert the btree encoding; title them accordingly. The third of these runs an ALPHA DESC sort with no BY pattern, so its title now mirrors the listpack-encoded "SORT sorted set" test it parallels. Signed-off-by: Rain Valentine <rsg000@gmail.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review. 📝 WalkthroughWalkthrough
ChangesSorted set sorting
Priority: ➖ Normal Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant SortCommand as SORT/SORT_RO
participant SortGeneric as sortCommandGeneric
participant Zset as Sorted set
SortCommand->>SortGeneric: Sort sorted set
SortGeneric->>Zset: Read length and encoding
SortGeneric->>Zset: Iterate LISTPACK or BTREE in place
Zset-->>SortGeneric: Return sorted members
Merge Risk: ⚪ Minimal · up to The change preserves sorted-set behavior and encoding across the reviewed paths, with no material merge-blocking risk identified. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## unstable #4698 +/- ##
============================================
+ Coverage 80.41% 80.56% +0.15%
============================================
Files 191 192 +1
Lines 99454 100839 +1385
============================================
+ Hits 79975 81242 +1267
- Misses 19479 19597 +118
🚀 New features to boost your workflow:
|
|
Madelyn approves of this message. I didn't look at the PR, but this seems fine. I am OK if only write commands change the encoding. |
The weight keys used by the BY and GET patterns hash to different slots, so MSET fails against an external cluster. Move those assertions into their own cluster:skip test, matching the existing BY/GET tests in this file, so the encoding-stability checks that need only the sorted set itself keep running in cluster mode. Signed-off-by: Rain Valentine <rsg000@gmail.com>
…ter mode Keep the sorted set and the BY/GET weight keys in one slot with a hash tag. SORT accepts a BY or GET pattern in cluster mode when the pattern's slot matches the sorted key's, so the pattern assertions no longer need to be skipped there. Signed-off-by: Rain Valentine <rsg000@gmail.com>
Tcl reads an unquoted {s}zset as a brace group.
Signed-off-by: Rain Valentine <rsg000@gmail.com>
Both listpack walks decode a member the same way; a static helper keeps that in one place and lets the BY nosort branch read like its btree twin. The sorted set length is only needed by the btree branch, so it is computed there. Signed-off-by: Rain Valentine <rsg000@gmail.com>
SORT and SORT_RO destructively converted a listpack-encoded sorted set to btree encoding before sorting. This dates back to the original Redis "SORT by nosort" change and was an implementation convenience, not a feature.
With forkless save (#4460) a background iterator may be reading an object's value memory while the main thread executes commands. Writes are gated by
bgIteration_blockClientIfRequired(), which only blocks commands classified as writes. SORT_RO is READONLY, so the conversion freed the listpack from under a background reader.Rather than special-casing SORT_RO as a write (the PFCOUNT approach), make SORT genuinely read-only with respect to the key being sorted: handle the listpack encoding directly in the two places that consumed the btree (the BY nosort ordered walk, and the collect-all path), using the same lpSeek/zzlNext/zzlPrev idiom as ZRANGE.
User-visible change: SORT / SORT_RO no longer changes the encoding of a small sorted set, as observable via OBJECT ENCODING.
Tests: SORT and SORT_RO leave a listpack zset as listpack across ALPHA, BY nosort, LIMIT, BY pattern and GET; integer members (the listpack integer path) in both encodings; and a 50-member parity test asserting identical results between listpack and btree for the nosort/LIMIT edge cases.