db: rework merging iterator range deletion handling #3600
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
|
This change is |
jbowens
force-pushed
the
rangedel-span
branch
4 times, most recently
from
e9f5888 to
5c24383
Compare
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Rework the merging iterator's handling of range deletions to use the interleaved range deletion boundary keys to determine when the iterator is positioned within a level's range deletion. This removes the direct manipulation of a range deletion keyspan.FragmentIterator from the mergingIter, delegating that to the child iterator's keyspan.InterleavingIter. This factoring is a little cleaner and decouples the mergingIter from the details of range deletion iterators, and in particular, the levelIter's individual sstables. It also should reduce key comparisons, especially during scans, by avoiding unnecessary key comparisons with range deletions that are loaded but outside the keyspace being iterated over. Close cockroachdb#2863.
jbowens
added a commit
to jbowens/pebble
that referenced
this pull request
Jan 12, 2026
In 1a2a77b we began interleaving the boundaries of range deletions from level iterators. This interleaving avoids stepping away from a table until its range deletions are no longer useful. However, this interleaving required its own range deletion iterator, necessitating opening two separate range deletion iterators into the same block. This commit updates the levelIter implementation to support interleaving range deletions without using a rangeDelIterSetter callback and adds a new getTombstone method that may be used to retrieve the tombstone at the current iterator's position. In this commit we update the level checker to no longer request its own range deletion iterator through rangeDelIterSetter. Instead use the levelIter's getTombstone method to read the current tombstone within each level, avoiding opening the duplicate range deletion iterator. This commit is a subset of the changes in the unmerged PR cockroachdb#3600. Future work will apply the same technique to the merging iterator, allowing ordinary user iterators to avoid the duplicate range deletion iterator too.
jbowens
added a commit
to jbowens/pebble
that referenced
this pull request
Jan 12, 2026
In 1a2a77b we began interleaving the boundaries of range deletions from level iterators. This interleaving avoids stepping away from a table until its range deletions are no longer useful. However, this interleaving required its own range deletion iterator, necessitating opening two separate range deletion iterators into the same block. This commit updates the levelIter implementation to support interleaving range deletions without using a rangeDelIterSetter callback and adds a new getTombstone method that may be used to retrieve the tombstone at the current iterator's position. In this commit we update the level checker to no longer request its own range deletion iterator through rangeDelIterSetter. Instead use the levelIter's getTombstone method to read the current tombstone within each level, avoiding opening the duplicate range deletion iterator. This commit is a subset of the changes in the unmerged PR cockroachdb#3600. Future work will apply the same technique to the merging iterator, allowing ordinary user iterators to avoid the duplicate range deletion iterator too.
jbowens
added a commit
to jbowens/pebble
that referenced
this pull request
Jan 14, 2026
In 1a2a77b we began interleaving the boundaries of range deletions from level iterators. This interleaving avoids stepping away from a table until its range deletions are no longer useful. However, this interleaving required its own range deletion iterator, necessitating opening two separate range deletion iterators into the same block. This commit updates the levelIter implementation to support interleaving range deletions without using a rangeDelIterSetter callback and adds a new getTombstone method that may be used to retrieve the tombstone at the current iterator's position. In this commit we update the level checker to no longer request its own range deletion iterator through rangeDelIterSetter. Instead use the levelIter's getTombstone method to read the current tombstone within each level, avoiding opening the duplicate range deletion iterator. This commit is a subset of the changes in the unmerged PR cockroachdb#3600. Future work will apply the same technique to the merging iterator, allowing ordinary user iterators to avoid the duplicate range deletion iterator too.
jbowens
added a commit
that referenced
this pull request
Jan 14, 2026
In 1a2a77b we began interleaving the boundaries of range deletions from level iterators. This interleaving avoids stepping away from a table until its range deletions are no longer useful. However, this interleaving required its own range deletion iterator, necessitating opening two separate range deletion iterators into the same block. This commit updates the levelIter implementation to support interleaving range deletions without using a rangeDelIterSetter callback and adds a new getTombstone method that may be used to retrieve the tombstone at the current iterator's position. In this commit we update the level checker to no longer request its own range deletion iterator through rangeDelIterSetter. Instead use the levelIter's getTombstone method to read the current tombstone within each level, avoiding opening the duplicate range deletion iterator. This commit is a subset of the changes in the unmerged PR #3600. Future work will apply the same technique to the merging iterator, allowing ordinary user iterators to avoid the duplicate range deletion iterator too.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rework the merging iterator's handling of range deletions to use the
interleaved range deletion boundary keys to determine when the iterator is
positioned within a level's range deletion. This removes the direct
manipulation of a range deletion keyspan.FragmentIterator from the mergingIter,
delegating that to the child iterator's keyspan.InterleavingIter.
This factoring is a little cleaner and decouples the mergingIter from the
details of range deletion iterators, and in particular, the levelIter's
individual sstables. It also should reduce key comparisons, especially during
scans, by avoiding unnecessary key comparisons with range deletions that are
loaded but outside the keyspace being iterated over.
Close #2863.