Repository navigation
fix(engine): leave the row address out of the stored-schema check - #906
Merged
ragnorc merged 1 commit intoOct 9, 2026
Merged
Conversation
Since typed reads check every column a node read consumes against the table's declared stored schema, a hydrated binding's scan failed with `consumed node:<Type>._rowaddr has no declared field`: it reads Lance's row address, which is no stored field. `ScanColumns::stored_columns` is now the one list of stored fields a node read consumes, without `_rowaddr`; the table scan and the destination read by id both pass it to the check. `HydrateExec` runs the same check on the columns it takes from each binding's pinned dataset.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
mainis red since #887 (4c7a160a). #887 merged before CI finished on its last head, which included the merge with #901. Every query that hydrates return columns after a limit fails:#901 checks every column a node read consumes against the table's declared stored schema. #887 makes a hydrated binding's scan read Lance's row address
_rowaddr, which is not a stored field, so the check refuses it. The failing tests onmainare the hydration owners from #887:engine::report::tests::a_hydrating_top_k_builds_one_operator_above_its_limitengine_v2_memory::a_wide_column_read_under_a_limit_follows_its_resultengine_v2_memory::hydrated_copies_of_a_joined_row_stay_within_the_chunk_boundengine_v2_plan_replay::a_hydrating_top_k_replaysv2/planner/hydrate_return_columns_above_the_limit.gqtWhat changes
ScanColumns::stored_columnsis the one list of stored fields a node read consumes: its projection, or every non-Blob column, without_rowaddr. The table scan (NodeRead::resolve) and the destination read by id (operators/scan/input.rs) both pass it tocheck_stored_schema, so neither treats the row address as a stored field.HydrateExecnow runs the same stored-schema check on the columns it takes. It checks each binding's pinned dataset against the declared node schema before reading, as feat(query): type the IR and plan every cast and aggregate #901 requires for every consumed physical column.Evidence
mainpass: the report test, bothengine_v2_memoryhydration tests,a_hydrating_top_k_replaysand the GQT case.nested_hydration_uses_the_query_pool_and_releases_on_refusalpasses too.hydrate $varclaims cover both changed read paths: its traversal steps hydrate a destination through a hash join and through id lookup.