fix(index): error on missing partition column in RQ transform - #9391
Open
LuciferYang wants to merge 3 commits into
Open
LuciferYang wants to merge 3 commits into
LuciferYang wants to merge 3 commits into
Conversation
Two branches of RQTransformer::transform fetched PART_ID_COLUMN via RecordBatch indexing (panicking with an undecorated arrow error when absent) while the same function returned descriptive errors for every other column. Use column_by_name with the same error contract. Assisted-by: GLM-5.3
github-actions
Bot
added
A-index
![lance-gatekeeper[bot]](https://avatars.githubusercontent.com/in/4433141?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
✅ Gate recommendation: approve.
Both partition-ID reads now use the transform’s existing fallible column contract while preserving the residual-query/L2 path that does not require the column. The regression test verifies the current raw-query path returns an Index error naming the missing column. I found no acceptance blocker.
lance-gatekeeper
Bot
added
the
K-approved
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #9390
RQTransformer::transformread__ivf_part_idwithbatch[PART_ID_COLUMN], which panics when the column is absent, while the vector column and__ivf_centroid_distin the same function return acolumn not found in batcherror. Both partition-id reads now go throughcolumn_by_nameand return that same error, so a public transformer handed an incomplete batch fails instead of panicking.Nothing changes for a batch that has the column, which is every batch the IVF transform chain produces.
How was this patch tested?
test_rq_transformer_reports_missing_partition_columncalls the transformer with the vector and centroid-distance columns but no partition column and asserts the error names the column. It sits next to the existing multi-bit factor test and reuses its fixture shape. Putting the indexing operator back makes the test panic at that line instead.