test(parity): corpus tiers 08-10; file P13-P26; pin DuckDB

[TimelordUK]

Discovery push against the DuckDB reference engine. No engine changes — this is corpus, docs and CI only, so the only job that can move is the parity gate.

corpus   83 -> 150 cases        open findings   1 -> 15
tiers    07 -> 10               decisions taken  P17, P20

Why now

The P-log was exhausted apart from P3, so new findings had to come from widening the corpus rather than working the backlog — which is how P9–P12 arrived in the first place. Tiers 08 (ordering & limiting), 09 (window functions & QUALIFY) and 10 (aggregate & NULL edges) were the uncovered surface.

Findings

Fourteen, ten of them silent — wrong answers with no error:

	Finding	Silent?
P13	Trailing unparsed tokens discarded, taking later clauses with them	yes
P14	Ungrouped aggregate over an empty set returns no row	yes
P15	QUALIFY rejects an inline window function	no
P16	ORDER BY <ordinal> ignored — rows come back in insertion order	yes
P17	Default NULL placement differs on ASC	yes
P18	= NULL matches NULL rows instead of yielding UNKNOWN	yes
P19	NOT IN doesn't exclude NULLs, though <> does	yes
P20	|| treats NULL as empty string while + propagates it	yes
P21	Window functions evaluated before WHERE	yes
P22	Five window functions return NULL instead of erroring	yes
P23	LAG/LEAD drop the third (default) argument	yes
P24	RANGE frame treated as ROWS, incl. the implicit default frame	yes
P25	A window's ORDER BY accepts only a plain column	no
P26	Window over an aggregate rejected by the GROUP BY check	no

P21 is the widest. With a WHERE present, windows see the unfiltered row set — COUNT(*) OVER (PARTITION BY team) reports pre-filter partition sizes. The identical query without the WHERE agrees with DuckDB exactly, which is what proves partitioning and the functions are correct and the defect is purely pipeline position.

P13 turned out far broader than its symptom. It surfaced as ORDER BY ... NULLS LAST LIMIT 3 returning every row, but there is no NULLS handling in src/sql/ at all — the parser silently ignores everything after the first token it can't place. ... ORDER BY amount DESC FROBNICATE LIMIT 3 runs clean and drops the LIMIT. Any typo becomes a different query that succeeds.

The fixture

data/null_edges.csv, 12 rows, purpose-built — every other corpus CSV is NULL-free, so NULL behaviour was unassertable. NULLs in a sort key, a partition key, a string column, a join key and an entirely-NULL column, plus deliberate ties and a unique never-NULL id as a total-order tiebreak.

Verified before building on it that both engines see the same NULLs in all five nullable columns — otherwise these cases would test the CSV reader, not the engine.

Two harness properties shaped the design and are recorded in the tier headers: normalize.py canonicalises "" to NULL so ''-vs-NULL is invisible to the comparison, and has_order_by() is a substring check so OVER (ORDER BY ...) alone forces ordered comparison — hence a total ordering on every case.

Two near-misses worth reading

Both are in the tier comments, because they show how this could have concluded the opposite:

• SUM(...) OVER (PARTITION BY ...) AGREEs under the same WHERE that exposes P21 — the filtered-out rows carry NULL scores that SUM ignores anyway. COUNT(*) is the discriminating probe; a SUM-only tier would have declared windows fine.
• ROWS and RANGE coincide entirely without ties, so P24 was undetectable before this fixture existed.

The general pattern: every finding got sharper from a control case. FROBNICATE turned a clause gap into a parser-wide finding; <> working correctly is what makes NOT IN a bug rather than a stance; qualify_select_list_alias moved P15's fix site to a different file entirely.

Checked and clear

Window evaluation has two code paths (SQL_CLI_BATCH_WINDOW, batch on by default). Ran 12 window queries through both — no mismatches, so these cases test one behaviour rather than two. Ruled out rather than assumed.

Decisions and policy

P17 and P20 are both cases the SQL standard leaves implementation-defined. Both resolved as follow the reference engine, now stated once in the doc preamble instead of as two precedents — with the qualification that DuckDB is a reference point, not a specification: the goal is to stop being accidentally different, not to reproduce DuckDB, and genuine idiosyncrasies can be ⚪ WON'T FIX.

That makes DuckDB's version part of the contract, so it is now pinned at 1.5.5 in pyproject.toml with CI resolving from that group. Verified zero bucket drift from 1.5.4 before pinning.

What this does not do

No fixes. Every finding is annotated with expect, so the parity gate stays green — that is what the contract is for. Discovery is now paused in favour of fixing; SQL_PARITY.md opens with the fix order, ranked by silent blast radius rather than P-number, on the principle that a loud failure is cheaper than a quiet one.

Tier 10 is deliberately partial — flagged in the doc so it isn't mistaken for complete. STDDEV, DISTINCT aggregates, FILTER and empty-vs-all-NULL are unexamined.

🤖 Generated with Claude Code