Repository navigation
fix(driver-sql): bucket a Field.date as its calendar day on PostgreSQL and MySQL - #21611
Conversation
…ect and session zone Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…L and MySQL A date has no instant. The PostgreSQL arm cast it to timestamptz and the MySQL arm handed it to convert_tz, both of which invent midnight in the session's zone, so on a session east of UTC a calendar day bucketed into the previous day, month, quarter or year. The declared type now picks the arm: a Field.date buckets its own day with no zone conversion, a Field.datetime and an undeclared column keep the UTC-instant arm unchanged. Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…calendar-day bucket Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…te-bucket-calendar-day
…d of erasing them Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 1 package(s): 9 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 3 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 11 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 1fed374c53c3c6e38de207195f7333b5dd6669b4 && git checkout 1fed374c53c3c6e38de207195f7333b5dd6669b4
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 9a4182a752fd53b24a14bbcd2e1b4270e1174e7a c20518cfd7787f8ba3f82d74fc85ca4254e416e8 && git checkout -B drift-repro 9a4182a752fd53b24a14bbcd2e1b4270e1174e7a && git merge --no-ff c20518cfd7787f8ba3f82d74fc85ca4254e416e8
node scripts/docs-audit/affected-docs.mjs --json 9a4182a752fd53b24a14bbcd2e1b4270e1174e7a
|
…mporal-pg-url Brings in PR #21611 (440cd32), which buckets a Field.date as its calendar day on PostgreSQL and MySQL -- the fix for the four live-PostgreSQL objectql-face-order-limit cells this branch's wiring exposed. Claude-Session: https://claude.ai/code/session_01T9u38rswFp5Rw8DswRUReJ Co-authored-by: Claude <noreply@anthropic.com>
…eSQL URL (objectstack-ai#21577) Fixes objectstack-ai#21564 Clause-②: no ## What this changes The `Temporal Conformance (live PG + MySQL)` job's step "Run the non-SQL temporal backends under the skewed process zone" now sets `OS_TEST_POSTGRES_URL` in its `env:`, with the same literal the job's three SQL steps use (`postgres://postgres:postgres@127.0.0.1:5432/postgres`), for the postgres service the job already provisions. Until now that step set only `TZ`, so every `service-analytics` live-PostgreSQL cell (`describe.skipIf(!config)`) was a named skip there while the job read green. The header clause "no CI step provisions that variable for this package" in 15 `service-analytics` test files becomes false with this wiring. It is corrected, comment-only, to name the step that now sets it. No other line of those files changes: every changed line in them is a ` * ` comment line. The 15th file, `objectql-echo-date-bucket.test.ts`, landed on `main` after this PR was opened, and the takeover added it (claim amendment `5973803595`). No other job, no other step, and no timeout changes. No cell is re-skipped. ## What the step needed, measured (A2) Only the one `env:` line: - **Build closure.** `turbo run build --filter=@objectstack/service-analytics... --dry=json` selects 16 packages, including `@objectstack/driver-sql`, `@objectstack/objectql` and `@objectstack/driver-sqlite-wasm` (dev dependencies of `service-analytics`). The step's existing build already builds what the cells import. - **PostgreSQL client.** `pg` is an optional peer of `driver-sql`, resolved by pnpm into `driver-sql`'s `knex` instance; `require.resolve('pg')` from that `knex` answers `pg@8.22.0`. The cells reach `pg` through `SqlDriver`, so nothing is added to `service-analytics`. - **Database and tables.** Every cell connects to the default `postgres` database and creates and drops its own tables. The table names are distinct across the 17 files that read the URL (`os21316_deal`, `os21505_coercion`, `os21441_bucket_deal`, ...), so parallel vitest workers do not collide. - **Server zone.** The job's existing step "Point both servers at a non-UTC timezone" sets the server to `Asia/Shanghai` before any test step. ## Readings - **A3.** No `service-analytics` file reads `OS_EXPECT_LIVE_DIALECT_MATRIX` (`git grep` exit 1), and no code in the step's other four packages reads it. So the flag is not added to the step. - **A4.** No `service-analytics` file reads `OS_TEST_MYSQL_URL` or any other server variable the job provides. Its only `process.env` reads are `OS_TEST_POSTGRES_URL` (34, in 17 files), `NODE_ENV` (10) and `TZ` (4). Nothing is listed for MySQL. - **A5.** `core`, `formula`, `driver-memory` and `driver-mongodb` have zero `OS_TEST_POSTGRES_URL` hits (`git grep` exit 1 for each). Adding the variable to the step starts no test in them. ## Acceptance - **CI, on this head `6e12d6934a`:** `Temporal Conformance (live PG + MySQL)` (job `111301243023`) is success, and step 14, the step this PR wires, is success (3m38s). All 34 check runs are complete: 32 success, 2 skipped (`Check Changeset` under `skip-changeset`, and the opt-in packed-tarball smoke). - **The cells run in CI, by contrast with the head before the fix:** - At `a3fb6c0334`, with this same wiring, step 14 failed. Four `objectql-face-order-limit` live-PostgreSQL cells went red on objectstack-ai#21485's defect: a `date` month bucket landed a day early on a non-UTC server. - objectstack-ai#21485 was fixed by PR objectstack-ai#21611 (`440cd329a9`), which this branch now carries. With the same wiring, step 14 passes at `25fb56f1b7` and at `6e12d6934a`. - That red-then-green on the same step shows the cells run there and now pass. - **NOT MEASURED: a quoted step-14 log line.** The job-log host answers Forbidden from the seat containers, and the API's log tail does not reach step 14. A seat whose egress reaches the log host can read step 14's per-file vitest lines for `objectql-face-order-limit` and `objectql-echo-date-bucket`. - **Local reproduction of the job's two zones** (PostgreSQL 16.14 with `timezone=Asia/Shanghai`, process `TZ=America/New_York`, the full `service-analytics` suite on the merged tree `25fb56f1b7`, from vitest's JSON reporter): - With the URL: 4419 passed, 0 failed, 2 skipped of 4421. - Without it: 4160 passed, 261 skipped. - So 259 cells move from skipped to run. - `objectql-face-order-limit`: 26 / 0 / 0, including the four formerly red cells. - `objectql-echo-date-bucket`: 16 / 0 / 0. - The two remaining skips are cell-scoped by design: the SQLite cell's "the server is not on UTC" check in `read-scope-temporal-coercion`, and a SQLite-only case in `analytics-faces-one-lowering`. Their PostgreSQL twin, "(pg) the server is not on UTC", passed, so the run was not vacuous. ## Acceptance notes - The ruling says each of these cells "asserts a non-UTC server". At this tree, 1 of the 17 files does (`read-scope-temporal-coercion.test.ts`, `show timezone`). The other 16 rely on the job's provisioning step for the server's zone. Noted, not changed. - `packages/plugins/plugin-security/src/rls-boolean-comparand-door.test.ts:40` carries the same "no CI step provisions" clause about its own package. This PR's step does not run that package, so the clause there stays true and is not touched. - **Stall guard.** `check:stall-guard-budget` reads this step's static budget as window 10m, cap 20m, job budget 30m, slack 10m. The step ran 3m38s on this head. - **Release.** Nothing publishes: `.github/workflows/ci.yml`, plus comments in test files outside `service-analytics`'s `files[]`. No changeset; `skip-changeset`. - **Governed.** `check-governed-merges.mjs`: 0 of 16 paths hit the register, NOT governed; 118 changed lines (+80 / −38). ## Gates Run on head `6e12d6934a`: the merge `1866c0185c` of `origin/main` `5b5e83f446`, plus the header commit. Each exit code was recorded before any pipe. - `dispatch-gates.mjs --commands` derived 87 commands. All 87 ran, and all exited 0. `--ran` reconciled them as 87 derived, 87 run, 0 NOT-MEASURED. - `check:changeset-gate-self-tests` was re-run once, alone, after the container's commit-signing hop answered 503 inside its scratch repository on the first pass. The re-run exited 0. - `service-analytics`: its build closure built, and `objectql-echo-date-bucket.test.ts` passed without the URL (10 passed, 8 PostgreSQL cells skipped locally). Its full suite with the URL is in Acceptance above. Opened by `domain:spec` seat 2 (`session_01YDt3PzwfrkuFzUBF89WPmM`); taken over by `domain:spec` seat 1 (`session_01T9u38rswFp5Rw8DswRUReJ`, claim `5973375306`). --------- Co-authored-by: Claude <noreply@anthropic.com>
Fixes #21485
Clause-②: no
What changes
SqlDriver.buildDateBucketExprnow reads the column's declared type from the object it already receives. A declaredField.dateis bucketed as its own calendar day with no zone conversion. AField.datetime, or a column with no declaration, keeps the UTC-instant expression byte for byte.Field.date(new)Field.datetime/ undeclared (unchanged)to_char((col)::date::timestamp, FMT)to_char((col)::timestamptz AT TIME ZONE 'UTC', FMT)date_format(col, FMT)date_format(convert_tz(col, @@session.time_zone, '+00:00'), FMT)strftimeon TEXT, no zone)Both callers render this one function, so both follow without a second edit:
aggregate()'s GROUP BY (buildDateBucketExpr(g.field, g.dateGranularity, table), withtable = coercionKey(builder)), and the publicdateBucketSql(objectName, …)member that the analytics SQL echo reads. Noservice-analyticssource is touched.The mechanism, measured
PostgreSQL 16.14 with the server's
TimeZoneatAsia/Shanghai, at the expression:datevalue2026-06-012026-05-312026-W222026-052026-06-01/2026-W23/2026-06/2026-Q2/20262026-01-012025-12-312026-W012025-122026-01-01/2026-W01/2026-01/2026-Q1/20262024-12-302024-12-292024-W522024-122024-12-30/2025-W01/2024-12/2024-Q4/2024Casting a
datetotimestamptzmakes midnight in the session zone;AT TIME ZONE 'UTC'then reads the previous UTC day on any session east of UTC. A session west of UTC does not shift (midnight local is later the same UTC day), which is why only east-of-UTC servers show it.Why
::date::timestampand not the bare::datethe dispatch suggested.EXPLAIN VERBOSEshowsto_char((d)::date, …)resolves toto_char((d)::timestamp with time zone, …)through the implicit cast, so the bare form still round-trips the session zone. Measured over every day 1900..2100 in ten zones, that round trip differs from the zone-free form on exactly one day each inPacific/Apia(2011-12-30prints2011-12-31) andPacific/Kiritimati(1994-12-31prints1995-01-01), the days those zones skipped.::date::timestampresolvesto_char(timestamp without time zone, …)and consults no zone.MySQL (H2), measured on MySQL 8.0.46 (the Ubuntu
mysql-server-core-8.0binary, run from a private datadir in this container) at a global+08:00. At the expression,convert_tz(d, '+08:00', '+00:00')shifts aDATEexactly like PostgreSQL:2026-06-01→ day2026-05-31, month2026-05;2024-12-30→2024-W52. Through the driver's default composition it does not, because the driver pins its own session to+00:00(withUtcSession), which makesconvert_tzthe identity. A hostpool.afterCreatethat sets the session zone (the driver chains it after its own hook) brings the shift back. Per triage's ruling ("gets the same split if aDATEshifts"; "No server-timezone dependence for adateon any dialect"), the MySQL arm gets the same split.Pins
packages/drivers/driver-sql/src/sql-driver-21485-date-bucket-calendar-day.test.ts(new). One matrix overDIALECT_CELLSthroughdeclareDialectCell:dateanddatetime× every granularity the dialect buckets in SQL (declared per dialect and asserted equal tosupports.queryDateGranularity, so the iterated set cannot shrink silently) × session zone. Each cell asserts both doors:aggregate()and thedateBucketSql()text run as SQL. The rows sit on month, quarter, year and ISO-week boundaries, plus the empty bucket.assertThreeWayZoneSkew) and session at +08:00 through a hostpool.afterCreate(thedateaxis, after asserting the session really is at+08:00). The second run is red-capable on any server, and it is the only red-capable route on MySQL.sql-driver-temporal-dialect.test.ts: a no-server pin (runs in Test Core too). Onpgandmysql2, a declaredField.dateexpression names notimestamptz,time zone,convert_tzortime_zone; onpgit carries::date::timestamp; the undeclared control keeps the UTC-instant arm.Where the live pins run in CI (H4). Job
Temporal Conformance (live PG + MySQL)(required), stepRun driver-sql suite against both live servers. It runs the whole driver-sql package withOS_TEST_POSTGRES_URL,OS_TEST_MYSQL_URL,OS_EXPECT_LIVE_DIALECT_MATRIX=1andTZ=America/New_York, with PostgreSQL atAsia/Shanghaiand MySQL at a global+08:00. No workflow edit is needed. PR #21577 (#21564) wires the separateservice-analyticsstep and is not part of this PR.Reverse verification and ablation
All runs: PostgreSQL 16.14 at
Asia/Shanghai, MySQL 8.0.46 at+08:00, processTZ=America/New_York,OS_EXPECT_LIVE_DIALECT_MATRIX=1; the new file plussql-driver-temporal-dialect.test.ts.26b8310fb): the new file 15 failed / 32 passed of 47. Thedateaxis was red at all five granularities on live postgres as provisioned, live postgres at +08:00, and live mysql at +08:00. Live mysql as provisioned (the driver's UTC pin), everydatetimecell and sqlite were green.aa061685b), thensql-driver.tsreverted to the base text (git restore --source=f6b752083; on-disk anchorscalendarDay3 → 0, the old PostgreSQL month text 0 → 1): 16 failed / 45 passed of 61. That is the same 15 live cells, plus the no-server pin. Restored withgit checkout HEAD -- packages/drivers/driver-sql/src/sql-driver.ts. Thehash-objectwasbe60fd469…, equal to the HEAD blob;git diff HEADandgit status --porcelainwere empty.::timestamphop (scripts/ablation-replace.mjs, anchor 1 → 0):(??)::date::timestampreplaced by the bare(??)::date. Predicted: the live pins stay green atAsia/Shanghai(the round trip is the identity there) and only the no-server pin reds. Observed: 1 failed / 60 passed, the no-server pin. Restored blob = HEAD,git diff HEADempty.service-analyticscells the card measured (objectql-face-order-limit.test.ts, which resolves@objectstack/driver-sqlthroughdist/): the fix was disabled indist/(const calendarDay = false, rebuilt,ablation-dist-preflightmarker present in 2 built files). Result: 4 failed / 22 passed of 26, exactly the card's four. After restore + rebuild +--absentpreflight: the file plusobjectql-echo-date-bucket.test.tsgave 42 of 42 green against live PostgreSQL atAsia/Shanghai. No assertion change is needed inobjectql-face-order-limit.test.ts, so this PR does not touch it.Dispatch hypotheses
aggregate:coercionKey(builder);dateBucketSql:objectName, which the analytics echo fills fromextractObjectName(cube)).temporalFieldKind(table, field)answers'date'fromdateFields, which onlytype: 'date'populates. No new parameter.DATEshifts under any east-of-UTC session; through the driver only with a host-set session zone.type: 'date'feedsdateFields. SQLite stores adateas TEXT, andstrftimereads it with no zone, so the SQLite arm has no instance of this class; its sqlite cells are the control. The SQLiteweekarm is untouched (analytics on SQLite: a week-bucketed (or non-UTC zone) dimension still echoes date_trunc, which SQLite refuses; driver-sql has no SQLite week expression #21595).Driver conformance ledger
pnpm check:driver-conformance, before (f6b752083) and after (c20518cfd7): 50 covered, 0 DEBT, 0 exempt, both times. The dialect axis was 8 suites (7 matrix, 1 named cell), 0 DIALECT ledger, both times. The new file imports no sharedspec/datacase-set, so it adds no cell. A spec-level shared case-set would have made every one of the five drivers consume it or carry DEBT. That is outside this card's file surface and against the no-new-DEBT commitment, so the matrix is driver-local.Local verification
pnpm --filter @objectstack/driver-sql exec vitest run --maxWorkers=2, with all three dialects live as in CI, ateab3f3bde(after mergingorigin/mainb610eabf7): 228 files, 5521 passed, 1 skipped (the skip is pre-existing, inschema-drift.base-type-mismatch.test.ts). The run printed "all 3 dialects were exercised".c20518cfd7, whose last commit only removes theas anycasts in the new test:pnpm --filter @objectstack/driver-sql typecheckwas green (--listFileslists all 228 test files). The new file,sql-driver-temporal-dialect.test.tsandlive-dialect-matrix.isolation.test.tsgave 81 of 81 green with all three dialects live.c20518cfd7:node scripts/pm/dispatch-gates.mjs --commandsderived 65, and all 65 were run with exit 0.--ranprinted: "65 derived famil(ies) accounted for, 65 run, 0 NOT-MEASURED". One gate was red on the way:check:query-options-erasure, because the new test'sas anyraised the test surface 236 → 237. It was answered by typing the query and the config, not by raising the number..tsfiles, all in the lint population (--print-configresolves a config for each), 3 files / 0 errors / 0 warnings in the--format jsonoutput atc20518cfd7.eslint.config.mjsenables no type-aware linting (noparserOptions.project), so this diff cannot move any untouched file's verdict. The fullpnpm lintis CI's.Acceptance notes
datetimeunder a host-set session zone (outside this card's ruling, not changed). Measured through the driver on MySQL 8.0.46 with a hostpool.afterCreatesettingtime_zone = '+08:00':2026-06-01T03:00:00.000Zstores2026-06-01 03:00:00.000inDATETIME(3), andfind()reads it back correctly, butaggregate()bydayanswers2026-05-31.convert_tz(…, @@session.time_zone, '+00:00')is right for a legacyTIMESTAMPcolumn and wrong for aDATETIME(3)that already holds the UTC wall clock. Under the driver's default composition it is the identity. No in-repo host sets the session zone, so this is noted here, not filed.type: 'date'over a remotetimestamptzcolumn would now bucket by the session-local day (::dateon atimestamptz). That is an inference only, not measured, and no producer is named.service-analyticscomments (strategies/objectql-strategy.ts,strategies/types.ts) and theobjectql-echo-date-bucket.test.tsdocblock quote the PostgreSQL… AT TIME ZONE 'UTC' …text. That is still thedatetimeexpression; they are not edited, because aservice-analyticssource edit is outside this PR's surface.TimeZone. The non-UTC readings here are a private server in this container.origin/mainb610eabf7(two commits,packages/specand docs only, disjoint from this diff) before the full-suite run.Generated by Claude Code