Extract the per-chromosome dynamic-SQL IEJoin machinery into a shared helper — Closes #217#218
Merged
Merged
Conversation
…red helper The DuckDB per-chromosome dynamic-SQL machinery -- the chromosome string-literal interpolation, the collision-resistant session-variable name, the SET VARIABLE ... = COALESCE((string_agg per chromosome), empty) builder, and the query(getvariable(...)) relation reference -- moves from IntersectsDuckDBIEJoinTransformer into a new giql.expanders._per_chrom module. The operator-specific parts (the per-chromosome SQL template, the partition source, the empty-schema fallback, and the outer projection) stay in the expander. Emitted SQL is byte-identical (no behavior change), readying the scaffolding for reuse by DISJOIN (#216) and NEAREST. Claude-Session: https://claude.ai/code/session_01TERWBHov76DQM3nQeT8yyy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The DuckDB per-chromosome dynamic-SQL scaffolding — build a
string_aggover the distinct chromosomes into aSET VARIABLE __giql_..._<token> = COALESCE((...), '<empty schema>'), then read it back viaquery(getvariable(...))— lived privately insideIntersectsDuckDBIEJoinTransformer. It's the mechanism that routes a range join through DuckDB'sIE_JOINby removing the low-cardinalitychromequality per partition, and two more operators need it: DISJOIN's breakpoint join (#216) and NEAREST's k-nearest sweep.This extracts the target-agnostic pieces into a new
giql/expanders/_per_chrom.py: theCHROM_LITERALinterpolation constant,new_variable_name,set_variable_statement, anddynamic_relation. The operator-specific parts (the per-chromosome SQL template, the partition source, the empty-schema fallback, and the outer projection) stay with each expander.intersects_duckdb.pyis refactored to use the helper.This is a pure refactor: the emitted SQL is byte-identical (all SQL-shape test assertions pass unchanged), so there is no behavior change. DISJOIN (#216) and NEAREST then reuse the same helper instead of duplicating the scaffolding.
Closes #217
Proposed changes
giql/expanders/_per_chrom.py(CHROM_LITERAL,sql_escape,new_variable_name,set_variable_statement,dynamic_relation) and rewrite theSET VARIABLE/getvariableconstruction inintersects_duckdb.pyto call it.No test changes are needed — the full non-integration suite (1031 passed) stays green with the byte-identical SQL, which is the correctness check for this refactor.
https://claude.ai/code/session_01TERWBHov76DQM3nQeT8yyy