fix(core,objectql)!: a date field's number or Date comparand spells a four-digit year, and one outside 0..9999 is refused INVALID_FILTER / 400 (#20240)

.changeset/20203-epoch-ms-date-comparand.md

@@ -1,3 +1,3 @@
 ---
 "@objectstack/core": patch
 "@objectstack/driver-sql": patch
@@ -28,4 +28,4 @@
 - `create()` / `update()` on either driver, given a number for a `date` field, stores its UTC day. Before, `driver-memory` and SQLite stored the number, and PostgreSQL refused the statement. The engine and REST write doors refuse a number on a `date` field before a driver sees it (`VALIDATION_FAILED`), as before.
 - A number already stored in a SQLite `date` column is read back as its UTC day by `find()`, a `groupBy` key and `distinct()`. Only a direct driver write could have put one there.
 
-Not changed, measured identical before and after: `NaN`, ±Infinity, a number outside the `Date` range (past ±8.64e15), a bigint, an epoch-millisecond string, every `Date` and every string on a `date` field, and every `datetime` and `time` reading. `driver-mongodb` keeps its own copy of the `date` rule and is not changed here.
+Not changed, measured identical before and after: `NaN`, ±Infinity, a number outside the `Date` range (past ±8.64e15), a bigint, an epoch-millisecond string, every `Date` and every string on a `date` field (#20240, in the same release, then pads a `Date`'s or a number's year 0..999 to four digits and refuses one whose year falls outside 0..9999, a number past the `Date` range included), and every `datetime` and `time` reading. `driver-mongodb` keeps its own copy of the `date` rule and is not changed here.

.changeset/20240-date-year-four-digits.md

@@ -0,0 +1,38 @@
+---
+"@objectstack/core": minor
+"@objectstack/objectql": minor
+"@objectstack/driver-sql": patch
+"@objectstack/driver-memory": patch
+---
+
+fix(core,objectql)!: a number or `Date` compared against a `date` field spells its year with four digits, and one whose UTC year falls outside 0..9999 is refused `INVALID_FILTER` / 400, as its ISO string already was (#20240)
+
+Clause-②: no (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) a query-time refusal of a comparand VALUE on a `date` field: no authorable key, spelling or stored shape moves, `packages/spec` is untouched, and a stored row keeps the form it has. What is refused is a number or `Date` whose day has no `YYYY-MM-DD` form, and which in-range day the caller meant is not something a ledger entry can decide. The other categories are closed on facts: the packages publish (not `unpublished`); no ADR-0087 id covers a comparand value (not `registered` / `already-registered`); and the change is runtime behaviour, not a declaration (not `runtime-interface-only` / `type-surface-only`). -->
+
+**BREAKING**: this narrows what a filter on a `date` field accepts. A number or `Date` whose UTC calendar day falls in a year below 0 or above 9999 used to answer 200 with the wrong rows, or a 500 on PostgreSQL; it now answers `INVALID_FILTER` / 400. It ships as `minor` under the launch-window convention for accept-set narrowings.
+
+`temporalStorageForm(value, 'date')` in `@objectstack/core` spelled the year of a `Date` or an epoch-millisecond number unpadded: `999-06-15`, `10000-01-01`, `-1-01-01`. The ISO string and the bare day of the same instant spelled `0999-06-15`, and as text an unpadded year sorts as no day does. Measured through `engine.find` / `engine.aggregate` and `POST /data/:object/query` (the two doors agree), on a `date` field holding six 2026 days and 0999-06-15, `$gt` / `$lt` / `$eq`:
+
+| position | comparand | before: memory · SQLite · PostgreSQL | now, on all three |
+|:--|:--|:--|:--|
+| `where` | a number (or, in-process, a `Date`) for 0999-06-15 | 0/7/0 · 0/7/0 · 6/0/1 | 6/0/1 |
+| per-aggregation `filter` | the same | 0/7/0 on all three | 6/0/1 |
+| `having` on `max(date)` | the same | no group / every group / no group | the three 2026 groups / none / the 0999 group |
+| `where` | a number (or `Date`) for 10000-01-01 | 6/1/0 · 6/1/0 · 0/7/0 | `INVALID_FILTER` / 400 |
+| `where` | a number (or `Date`) for -1-01-01 | 7/0/0 · 7/0/0 · `DATABASE_ERROR` (500) | `INVALID_FILTER` / 400 |
+| per-aggregation `filter` | either of those two | 6/1/0 and 7/0/0 on all three | `INVALID_FILTER` / 400 |
+| `where`, per-aggregation `filter` | the ISO string of either | `INVALID_FILTER` / 400 | unchanged |
+
+What changes:
+
+- The rule pads a year from 0 to 999 to four digits, for a `Date` and a number alike, so a number, its `Date` and its ISO string spell one day. `driver-sql` (`toDateOnly`, `temporalFilterValue`), `driver-memory` (`coerceTemporalValue`) and the engine's per-aggregation `filter` and `having` all call it.
+- `isUninterpretableTemporalComparand('date', value)` is now also true for a finite number or a valid `Date` whose UTC year is below 0 or above 9999, a finite number past the `Date` range (±8.64e15) included. The engine's temporal-comparand door refuses such a comparand on `where` for every verb (`find`, `findOne`, `count`, `aggregate`, `update`, `delete`), in both the object and the array spelling, and in a per-aggregation `filter`, before any driver read. `IObjectQLEngine.judgeFilter` runs the same door.
+- The write path: `create()` / `update()` on `driver-memory` or SQLite, given a year-0..999 number or `Date` for a `date` field, now stores `0999-06-15` where it stored `999-06-15`; `engine.insert` of such a `Date` does the same. PostgreSQL and MySQL already stored a three-digit year's day, but not a shorter one: under its default `DateStyle` (`ISO, MDY`) PostgreSQL stored the unpadded `9-03-04` as 2004-09-03 and refused `99-03-04` (`22008`), and MySQL 8.0 stored `99-03-04` as 1999-03-04. All three dialects now store the day. A year outside 0..9999 keeps the spelling it had on the write and read paths; no ordered form is invented for it.
+
+**Who is affected.** A caller that compares a `date` field with an epoch-millisecond number or a `Date` in a year below 0 or above 9999. No writer that stores or queries such a day has been measured; the reach is the public query door.
+
+**Fix.** Compare against a `YYYY-MM-DD` day, or a number or `Date` whose UTC calendar day falls in a four-digit year.
+
+**Unchanged**, measured identical before and after on memory, SQLite and PostgreSQL through the engine and REST: every `datetime` and `time` cell, the same numbers included; every string comparand on a `date` field; every number and `Date` in the years 1000 to 9999; `NaN`, ±Infinity and an Invalid Date, which name no year and are not judged; and every read-path presentation on those three. On MySQL, measured at the driver door, a stored year from 100 to 999 now reads back padded (`0999-06-15`, where it read `999-06-15`); a stored year below 100 still reads back a century late (`0009-03-04` as `1909-03-04`, mysql2's `Date.UTC` reading of a `DATE`), which this change does not touch. `having` does not reach the temporal-comparand door for any comparand, so a number or `Date` outside 0..9999 there is still compared as written. `service-analytics`' raw-SQL decline reads a time dimension by the `datetime` rule, so its answer does not move. `driver-mongodb` keeps its own copy of the `date` rule and is not changed here.

packages/core/src/utils/temporal-comparand.test.ts

@@ -0,0 +1,105 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+//
+// [#20240] `isUninterpretableTemporalComparand` on a `date` column judges one
+// non-string class: a finite number or a `Date` whose UTC calendar day falls in
+// a year below 0 or above 9999. Such a day has no `YYYY-MM-DD` spelling, so the
+// storage rule's text for it (`10000-01-01`, `-1-01-01`) orders as no day does.
+// Measured over REST before this, the number for 10000-01-01 counted `$gt` 6 /
+// `$lt` 1 on driver-memory and SQLite (the day's answer is 0 / 7) and the
+// number for -1-01-01 answered 500 on PostgreSQL, while the ISO string of either
+// instant was already refused `INVALID_FILTER` / 400. The door that calls this
+// predicate (`@objectstack/objectql`, `temporal-comparand-door.ts`) now refuses
+// the number and the `Date` too; its own suite pins the envelope.
+
+import { describe, it, expect } from 'vitest';
+import { isUninterpretableTemporalComparand } from './temporal-comparand.js';
+import { temporalStorageForm } from './temporal-storage-form.js';
+
+const at = (iso: string) => Date.parse(iso);
+
+/** Finite numbers whose UTC calendar day falls outside the four-digit years. */
+const OUT_OF_RANGE: ReadonlyArray<readonly [string, number]> = [
+  ['the first day of year 10000', 253402300800000],
+  ['year -1 (the card\'s number)', -62198755200000],
+  ['the last millisecond of year -1', at('-000001-12-31T23:59:59.999Z')],
+  ['the Date range maximum', 8.64e15],
+  ['the Date range minimum', -8.64e15],
+];
+
+/** Numbers past the `Date` range — an instant with a year past ±271821. */
+const PAST_THE_DATE_RANGE: ReadonlyArray<readonly [string, number]> = [
+  ['one past the maximum', 8.64e15 + 1],
+  ['one before the minimum', -8.64e15 - 1],
+  ['9e15', 9e15],
+  ['MAX_SAFE_INTEGER', Number.MAX_SAFE_INTEGER],
+];
+
+/** Finite numbers whose UTC calendar day has a four-digit year — read as before. */
+const IN_RANGE: ReadonlyArray<readonly [string, number]> = [
+  ['the first millisecond of year 0', at('0000-01-01T00:00:00.000Z')],
+  ['0999-06-15', -30627504000000],
+  ['1000-01-01', at('1000-01-01T00:00:00.000Z')],
+  ['2026-02-01T10:00Z', 1769940000000],
+  ['the epoch', 0],
+  ['the last millisecond of year 9999', at('9999-12-31T23:59:59.999Z')],
+];
+
+describe('[#20240] isUninterpretableTemporalComparand — a date column\'s number or Date outside the four-digit years', () => {
+  it('refuses a number and the Date of the same value, for a year below 0 or above 9999', () => {
+    for (const [name, ms] of OUT_OF_RANGE) {
+      expect(isUninterpretableTemporalComparand('date', ms), `number, ${name}`).toBe(true);
+      expect(isUninterpretableTemporalComparand('date', new Date(ms)), `Date, ${name}`).toBe(true);
+    }
+  });
+
+  it('refuses a finite number past the Date range — its year is past ±271821', () => {
+    for (const [name, ms] of PAST_THE_DATE_RANGE) {
+      expect(isUninterpretableTemporalComparand('date', ms), name).toBe(true);
+    }
+  });
+
+  it('reads every number and Date whose year has four digits, as before — the discriminating half', () => {
+    for (const [name, ms] of IN_RANGE) {
+      expect(isUninterpretableTemporalComparand('date', ms), `number, ${name}`).toBe(false);
+      expect(isUninterpretableTemporalComparand('date', new Date(ms)), `Date, ${name}`).toBe(false);
+    }
+  });
+
+  it('agrees with the rule: a finite number is refused exactly when the rule cannot spell it YYYY-MM-DD', () => {
+    for (const [name, ms] of [...OUT_OF_RANGE, ...PAST_THE_DATE_RANGE, ...IN_RANGE]) {
+      const spelled = temporalStorageForm(ms, 'date');
+      const isDay = typeof spelled === 'string' && /^\d{4}-\d{2}-\d{2}$/.test(spelled);
+      expect(isUninterpretableTemporalComparand('date', ms), name).toBe(!isDay);
+    }
+  });
+
+  it('joins the verdict the ISO string of the same instant already had', () => {
+    // The string arm refused these before this card; the number and the Date
+    // were the two spellings of the same instant it let through.
+    expect(isUninterpretableTemporalComparand('date', '+010000-01-01T00:00:00.000Z')).toBe(true);
+    expect(isUninterpretableTemporalComparand('date', '-000001-01-01T00:00:00.000Z')).toBe(true);
+    expect(isUninterpretableTemporalComparand('date', '0999-06-15T00:00:00.000Z')).toBe(false);
+    expect(isUninterpretableTemporalComparand('date', '0999-06-15')).toBe(false);
+  });
+
+  it('leaves the datetime and time rules alone — they read an instant', () => {
+    for (const kind of ['datetime', 'time'] as const) {
+      for (const [name, ms] of [...OUT_OF_RANGE, ...PAST_THE_DATE_RANGE, ...IN_RANGE]) {
+        expect(isUninterpretableTemporalComparand(kind, ms), `${kind}, number, ${name}`).toBe(false);
+        expect(isUninterpretableTemporalComparand(kind, new Date(ms)), `${kind}, Date, ${name}`).toBe(false);
+      }
+    }
+  });
+
+  it('does not judge NaN, ±Infinity or an Invalid Date — they name no instant and no year', () => {
+    for (const value of [Number.NaN, Number.POSITIVE_INFINITY, Number.NEGATIVE_INFINITY, new Date(Number.NaN)]) {
+      expect(isUninterpretableTemporalComparand('date', value), String(value)).toBe(false);
+    }
+  });
+
+  it('judges nothing else that is not a string, as before', () => {
+    for (const value of [null, undefined, true, 1769940000000n, Object(1769940000000), {}, ['2026-02-01']]) {
+      expect(isUninterpretableTemporalComparand('date', value), String(value)).toBe(false);
+    }
+  });
+});

packages/core/src/utils/temporal-comparand.ts

@@ -44,9 +44,19 @@
  * if the door let it through — `temporalStorageForm` (`temporal-storage-form.ts`,
  * the one rule both drivers read since #20176; before it, a copy in each).
  * That function is total on purpose: an input it cannot interpret is returned
- * UNCHANGED rather than becoming an invented instant. So "the driver would
- * return it unchanged" IS the definition of uninterpretable, and defining it
- * any other way would refuse comparands that work today.
+ * UNCHANGED rather than becoming an invented instant. So, for a STRING, "the
+ * driver would return it unchanged" IS the definition of uninterpretable, and
+ * defining it any other way would refuse comparands that work today.
+ *
+ * [#20240] One non-string class is uninterpretable by the same test's other
+ * half — the rule cannot put it in the column's form. On a `date` column a
+ * finite number or a `Date` becomes its UTC calendar day, and a year below 0 or
+ * above 9999 has no `YYYY-MM-DD` spelling: the rule keeps writing
+ * `10000-01-01` / `-1-01-01` for the write and read paths, but as a comparand
+ * that text orders as no day does (`10000-01-01` sorts below `2026-…`), and
+ * PostgreSQL refuses `-1-01-01` outright. So such a comparand is judged
+ * uninterpretable here, exactly as an unparseable string is, and refused by
+ * the same doors.
  *
  * That is why this is not `utcInstantMs` (`@objectstack/spec/data`), which is
  * the stricter canonical reader: it rejects a bare epoch-millisecond string and
@@ -56,8 +66,18 @@
  *
  * ## Two things it deliberately does NOT judge
  *
- * - **Non-string comparands.** A number is epoch milliseconds, a `Date` is an
- *   instant, `null` is a null test. The refusal scopes to strings by ruling.
+ * - **Non-string comparands, save the one `date` class above.** A number is
+ *   epoch milliseconds, a `Date` is an instant, `null` is a null test, and the
+ *   `datetime` and `time` rules read every finite one; so does the `date` rule
+ *   for a year from 0 to 9999. The #8690 refusal was scoped to strings by that
+ *   card's own ruling — its triage queued "a non-interpretable bare string", and
+ *   the maintainer ruling scoped its two options "to non-empty strings" so the
+ *   empty-string cell stayed its own card. That scoped that change; it is not a
+ *   standing rule that a non-string is never refused. [#20240] extends the
+ *   refusal to the `date` class above by the triage direction on that card.
+ *   `NaN`, ±Infinity and an Invalid Date name no instant and no year, so they
+ *   are not that class and stay unjudged, as before; no JSON body can carry
+ *   one (JSON spells them `null`).
  * - **The EMPTY string.** Measured, `$gte ""` binds as `''` and every canonical
  *   UTC text sorts at or above it, so it returns every non-null row — a third
  *   behaviour again, and one the maintainer ruled stays its own card: "B and C
@@ -130,13 +150,35 @@ function readsAsWallClock(s: string): boolean {
   return Number(m[1]) <= 23 && Number(m[2]) <= 59 && Number(m[3] ?? '0') <= 59;
 }
 
+/**
+ * [#20240] `temporalStorageForm`'s `date` reading of a NUMBER or a `Date` lands
+ * outside the four-digit years `YYYY-MM-DD` can spell.
+ *
+ * Both name an instant, and the rule takes that instant's UTC calendar day.
+ * Its year must fall from 0 to 9999: `0999-06-15` is a day, `10000-01-01` and
+ * `-1-01-01` are not. A finite number past ±8.64e15 names an instant the
+ * `Date` type cannot hold at all — a year past ±271821 — so it is outside the
+ * range too, and the rule hands it back unchanged. `NaN`, ±Infinity and an
+ * Invalid Date name no instant and no year, and are not judged here.
+ */
+function isOutsideCalendarDayYears(value: number | Date): boolean {
+  if (typeof value === 'number' && !Number.isFinite(value)) return false;
+  const instant = typeof value === 'number' ? new Date(value) : value;
+  if (Number.isNaN(instant.getTime())) return typeof value === 'number';
+  const year = instant.getUTCFullYear();
+  return year < 0 || year > 9999;
+}
+
 /**
  * Is `value` a comparand that a `kind` column's storage rule cannot read?
  *
- * `true` ONLY for a non-empty, non-placeholder STRING that the kind's rule
- * would hand back unchanged. Everything else — a number, a `Date`, `null`, a
- * `{ $field }` reference, filter structure, the empty string, a `{token}` —
- * answers `false`, each for a reason recorded in the module note or below.
+ * `true` for a non-empty, non-placeholder STRING that the kind's rule would
+ * hand back unchanged, and — on a `date` column only — for a finite number or
+ * a `Date` whose UTC calendar day falls in a year below 0 or above 9999
+ * ({@link isOutsideCalendarDayYears}). Everything else — any other number or
+ * `Date`, `null`, a `{ $field }` reference, filter structure, the empty
+ * string, a `{token}` — answers `false`, each for a reason recorded in the
+ * module note or below.
  *
  * A `{placeholder}` is stepped around rather than judged because it is another
  * layer's vocabulary and that layer already refuses the unknown ones loudly
@@ -149,6 +191,9 @@ export function isUninterpretableTemporalComparand(
   kind: TemporalComparandKind,
   value: unknown,
 ): boolean {
+  if (kind === 'date' && (typeof value === 'number' || value instanceof Date)) {
+    return isOutsideCalendarDayYears(value);
+  }
   if (typeof value !== 'string') return false;
   const s = value.trim();
   // The empty-string cell is its own card — see the module note.