fix(index): error on non-fsl input in NormalizeTransformer - #9393
Open
LuciferYang wants to merge 3 commits into
Open
LuciferYang wants to merge 3 commits into
LuciferYang wants to merge 3 commits into
Conversation
NormalizeTransformer downcast the input with as_fixed_size_list, which panics for non-fixed-size-list columns, while sibling KeepFiniteVectors in the same file returns a descriptive Error::index for the same input. Match the sibling's error contract. Assisted-by: GLM-5.3
github-actions
Bot
added
A-index
![lance-gatekeeper[bot]](https://avatars.githubusercontent.com/in/4433141?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
✅ Gate recommendation: approve.
This change closes the caller-controlled type boundary by returning a contextual Error::Index instead of allowing Arrow’s fixed-size-list downcast to panic. The focused wrong-type regression test passes, as do the existing normalization tests for fixed-size-list inputs and output-column mode.
lance-gatekeeper
Bot
added
the
K-approved
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #9392
NormalizeTransformer::transformerrored when its column was missing and then panicked if the column was the wrong type, because the downcast usedas_fixed_size_list(). The column is named by string on a public transformer, so the type is the caller's to get wrong. It now uses the_optvariant and reports the column and the type it found, which is whatKeepFiniteVectorsin the same file already does for its own column.How was this patch tested?
test_normalize_transformer_rejects_non_fsl_columnpoints the transformer at aFloat32column and asserts the error names the problem; it mirrors the existingtest_keep_finite_vectors_rejects_non_list_column. With the panicking downcast restored the test panics insidearrow::array::castinstead.