Skip to content

Extract the per-chromosome dynamic-SQL IEJoin machinery into a shared helper #217

Description

@conradbzura

Description

The DuckDB per-chromosome dynamic-SQL machinery — build a string_agg over the distinct chromosomes into a SET VARIABLE __giql_..._<token> = COALESCE((...), '<empty schema>'), then read it back via SELECT ... FROM query(getvariable(...)) AS <alias> — currently lives privately inside src/giql/expanders/intersects_duckdb.py. It routes a column-to-column INTERSECTS join through DuckDB's IE_JOIN by removing the low-cardinality chrom equality per partition.

Two more operators need the same partitioning to reach IE_JOIN: DISJOIN's breakpoint join (#216) and NEAREST's k-nearest sweep. Rather than duplicate the scaffolding, extract the target-agnostic pieces into a shared helper.

Motivation

Avoid three copies of the SET VARIABLE/string_agg/getvariable scaffolding. The operator-specific parts (the per-chromosome SQL template, the chromosome-partition source, the empty-schema fallback, and the outer projection) stay with each expander; only the shared scaffolding is centralized.

Expected Outcome

A small shared utility (e.g. giql/expanders/_per_chrom.py) exposing: the chrom string-literal interpolation constant, a collision-resistant session-variable name generator, the SET VARIABLE ... = COALESCE((SELECT string_agg(<template>, ' UNION ALL ') FROM (<partition>)), '<empty>') builder, and the query(getvariable('<var>')) AS <alias> relation reference. intersects_duckdb.py is refactored to use it with byte-identical emitted SQL (all existing tests stay green — this is a pure refactor with no behavior change). DISJOIN (#216) and NEAREST then reuse the same helper.

Metadata

Metadata

Assignees

Labels

refactorCode restructuring without behavior change

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions