Skip to content

Fix spurious slice is not sorted assert in Subset::intersect (#971) - #973

Open
yihozhang wants to merge 1 commit into
mainfrom
fix/971-intersect-transient-unsorted
Open

Fix spurious slice is not sorted assert in Subset::intersect (#971)#973
yihozhang wants to merge 1 commit into
mainfrom
fix/971-intersect-transient-unsorted

Conversation

@yihozhang

Copy link
Copy Markdown
Collaborator

Fixes #971.

Problem

Subset::intersect's (Sparse, Sparse) "other is much smaller: iterate other and gallop in cur" branch compacts the intersection into cur in place, but re-derived a SortedOffsetSlice over the whole vector on every iteration:

let result = cur.slice().scan_for_offset(ci, target);   // asserts all of cur is sorted
match result {
    Ok(found) => { cur.0[write] = target; write += 1; ci = found + 1; }

Once the first match is written to cur.0[write], the region between write and ci still holds stale originals, so cur as a whole is transiently unsorted and slice()'s debug_assert! fires on the next iteration.

That is exactly the panic in #971. cur was [0,1,2,3,4,5,11..=21] (17 rows) and the first match was RowId(13), written over index 0:

[RowId(13), RowId(1), ..., RowId(5), RowId(11), RowId(12), RowId(13), ..., RowId(21)]
 ^^^^^^^^^ written result   ^^^^^^^^^^^^^^^^^^ untouched original tail

Despite the identical assertion message, this is a different bug from #911/#914 — that one was in hash_index; this one is in offsets, reached via Database::process_constraints.

This was only the assertion, not the results

write <= ci always holds (a match at found >= ci writes to write <= found, then sets ci = found + 1) and scan_for_offset never reads below its start, so every read already landed in the untouched suffix. Confirmed empirically: with the assertion compiled out, the pre-fix code produces correct intersections. So release builds were unaffected; debug builds panicked.

Fix

Search cur.0[ci..] — the suffix that is guaranteed untouched — instead of all of cur, and make the returned offsets absolute again. scan_for_offset's duplicate back-walk floors at start, which becomes suffix-relative 0, so the semantics are unchanged. Same complexity, no extra work in the hot path.

Testing

The existing offsets::tests::intersect corpus missed this because add_row_sorted collapses contiguous row sets to Dense, so its skewed pairs never put two Sparse subsets into this branch. Added:

  • offsets::tests::intersect_sparse_sparse_skewed, a targeted regression test that reproduces Bug: debug_assert failure: slice is not sorted #971's slice byte-for-byte in ~20 lines with no external dependencies.
  • Two entries (a 17-row sparse subset and a 3-row sparse subset) to the exhaustive intersect corpus, so all-pairs coverage now reaches this branch.

Verified at 53b9721:

  • cargo test --workspace: 1132 passed, 0 failed (including the 792-test integration suite).
  • The reporter's reproducer in ProteusLab/LIRA@egglog_reprocd rust && cargo test -- repro — passes against the patched egglog, as does the rest of that suite.
  • cargo fmt --check and cargo clippy --all-targets -- -D warnings clean.

I also audited the other SortedOffsetVector::slice() / scan_for_offset call sites; this was the only one whose receiver can be mid-mutation. One adjacent fragility, left alone as it is not a live bug: binary_search_from's duplicate back-walk is bounded by found > 0 rather than found > start, so it can return an index below start; at the sole start != 0 call site the values make that unreachable.

🤖 Generated with Claude Code

The `(Sparse, Sparse)` "gallop in cur" branch of `Subset::intersect`
compacts the intersection into `cur` in place, but re-derived a
`SortedOffsetSlice` over the *whole* vector on each iteration via
`cur.slice()`. Once the first match has been written to `cur.0[write]`,
the region between `write` and `ci` still holds stale originals, so the
vector as a whole is transiently unsorted and `slice()`'s `debug_assert!`
fires on the next iteration.

The searches themselves were already correct: `write <= ci` always holds
and `scan_for_offset` never reads below its `start`, so every read lands
in the untouched suffix. Only the assertion was wrong, so release builds
(with the assertion compiled out) produced correct intersections.

Search `cur.0[ci..]` — the untouched suffix — instead of all of `cur`,
and make the returned offsets absolute again. Same complexity, no extra
work in the hot path.

The existing `intersect` test corpus missed this because
`add_row_sorted` collapses contiguous row sets to `Dense`, so its skewed
pairs never put two `Sparse` subsets into this branch. Add a targeted
regression test plus two entries to the exhaustive corpus that do.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@yihozhang
yihozhang requested a review from a team as a code owner August 3, 2026 21:10
@yihozhang
yihozhang requested review from saulshanabrook and removed request for a team August 3, 2026 21:10
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 86.63%. Comparing base (53b9721) to head (79ffa4b).

Files with missing lines Patch % Lines
core-relations/src/offsets/mod.rs 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #973      +/-   ##
==========================================
+ Coverage   86.59%   86.63%   +0.04%     
==========================================
  Files          95       95              
  Lines       29676    29678       +2     
==========================================
+ Hits        25699    25713      +14     
+ Misses       3977     3965      -12     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed-hq

codspeed-hq Bot commented Aug 3, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 37 untouched benchmarks
⏩ 227 skipped benchmarks1


Comparing fix/971-intersect-transient-unsorted (79ffa4b) with main (53b9721)

Open in CodSpeed

Footnotes

  1. 227 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: debug_assert failure: slice is not sorted

2 participants