Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .changeset/20722-import-time-fraction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
"@objectstack/rest": minor
---

fix(rest)!: `POST /api/v1/data/:object/import` reads a `time` cell by `@objectstack/core`'s one `time` rule, the rule the write door asks, so the `10:00:00.250` that `/export` writes for a `time` with milliseconds re-imports as itself instead of failing its row as `invalid_date`, and a cell the write door refuses is refused with the write door's code, `invalid_time` (#20722)

Clause-②: no (narrowing)

<!-- adr-0087: not-required (no-migration-prescription) a refusal of an import CELL value at the `/import` reader's time arm, and the field code that refusal reports: no authorable key, spelling or stored shape of metadata moves, and no schema, type or export changes; `packages/spec` is untouched. The code a refused `time` cell reports moves between two existing members of the closed `FieldErrorCode` catalog (ADR-0114), so nothing is minted or retired, and nothing branches on a field code. An import file is caller input, not stored metadata, and a row already stored keeps whatever it holds, so there is nothing a ledger entry could rewrite. The other categories are closed on facts: `@objectstack/rest` publishes (not `unpublished`); no ADR-0087 id covers an import cell check (not `registered` / `already-registered`); and the change is runtime behaviour, not a TypeScript declaration (not `runtime-interface-only` / `type-surface-only`). -->

**BREAKING**: this narrows what `/import` accepts in a `time` cell. An ISO 8601 instant whose UTC year has no four-digit spelling (`"9999-12-31T23:00:00-02:00"`, which is UTC year 10000) was stored as its UTC time of day (`01:00:00`). It now fails its row with `invalid_time`, as the write door has refused the same value since #20671. A refused `time` cell's row code also moves from `invalid_date` to `invalid_time`, the code the write door gives for the same value. It ships as `minor` under the launch-window convention for accept-set narrowings (`check-changeset-no-major` refuses `major` until GA; the breaking-ness is carried by this banner and the ADR-0087 disposition above).

The import's `time` reader was a private pattern with no fractional part. A `time` stored with milliseconds (`10:00:00.250`, which the write door accepts and `/export` writes as it is stored) failed its row on re-import with `Clock: "10:00:00.250" is not a valid time`, on every backend, so an export did not re-import. The reader now asks the same rule the write door asks of a written `time`: `isUninterpretableTemporalComparand('time', …)` for the verdict and `temporalStorageForm(…, 'time')` for the stored value. A cell is admitted exactly when the write door admits the same value, and stored as the same wall clock.

Through the route, the process in America/New_York, on memory, SQLite and PostgreSQL 16 (at `Asia/Shanghai`), before and after:

| `time` cell | before | now |
|:--|:--|:--|
| `10:00:00.250`, `23:59:59.999` (what `/export` writes) | row failed, `invalid_date` | stored as written |
| `10:00:00.5`, `10:00:00.000` | row failed, `invalid_date` | `10:00:00.500`, `10:00:00` |
| `2026-07-15T10:00:00.250Z`, `2026-07-15 10:00:00.250` | stored `10:00:00`, the fraction dropped | `10:00:00.250`, as the write door stores it |
| `9999-12-31T23:00:00-02:00` (an instant in UTC year 10000) | stored `01:00:00` | row failed, `invalid_time`, as the write door refuses it |
| `10:00Z`, `10:00+08:00`, `07/15/2026 10:00`, `25:00` | row failed, `invalid_date` | row failed, `invalid_time` |
| `10:00`, `10:00:00`, `2026-07-15T18:00:00+08:00` | `10:00:00` | unchanged |
| `2026/7/15 9:00` (the year-first form) | `09:00:00` | unchanged |

A zone-naive `2026-07-15 24:00` in a `time` cell, refused before, is now read as `00:00:00`: core's rule reads it so, and the write door admits it. A `date` or `datetime` cell keeps `invalid_date`, and the row's sentence is unchanged for every kind. A time of day with a `Z` or an offset stays refused: a `time` carries no zone. The year-first date-time (`2026/7/15 9:00`) is still read as its wall clock; it is the one reading the import has that the write door has not. The dry run reports the same verdicts.

**Who is affected.** A caller of `POST /api/v1/data/:object/import`, including the dry run, whose file carries such an instant in a `time` column now gets that row refused. A client that matched the row report's `code` for a refused `time` cell now reads `invalid_time`. Rows already stored are not rewritten.

This supersedes the note in the `@objectstack/objectql` entry for #20671 that the server import turns a `time` cell into `HH:MM:SS`: the import now keeps a non-zero fraction (`HH:MM:SS.fff`).
2 changes: 1 addition & 1 deletion packages/rest/src/import-coerce.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ describe('parseDateCell', () => {
* [#20534] A text cell is read only in ISO 8601, the export's own
* `YYYY-MM-DD HH:mm:ss` or a year-first date (`2026/7/15`, `2026/7/15 9:00`,
* by the maintainer ruling on the card), on a calendar day that exists;
* everything else is refused (`undefined`, so the row's `invalid_date`), never rolled over, never
* everything else is refused (`undefined`, so the row's `invalid_date`, a `time` cell's `invalid_time`), never rolled over, never
* read in the host's zone and never read month-first. A `date`'s year keeps
* four digits. Every case runs under two host zones that disagree by twelve
* hours, and must answer the same under both — the host-zone reading this
Expand Down
81 changes: 66 additions & 15 deletions packages/rest/src/import-coerce.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
* - number / currency / percent / rating / slider → a finite `number`
* - boolean / toggle → a real `boolean`
* - date / datetime → an ISO-8601 string
* - time → `HH:MM` / `HH:MM:SS`
* - time → `HH:MM:SS`, `.fff` when non-zero
* - select / radio → an option *value*
* - multiselect / checkboxes / tags → an array of option values
* - lookup / master_detail / user / reference → a record id (resolved async)
Expand All @@ -31,7 +31,12 @@
* untouched, so an import stays byte-identical to the pre-coercion behaviour.
*/

import { temporalStorageForm, zonedWallClockToUtcMs, type WallClockParts } from '@objectstack/core';
import {
isUninterpretableTemporalComparand,
temporalStorageForm,
zonedWallClockToUtcMs,
type WallClockParts,
} from '@objectstack/core';
import type { ExportFieldMeta } from './export-format.js';
import {
SINGLE_OPTION_TYPES as OPTION_TYPES,
Expand All @@ -43,6 +48,7 @@ import {
IMPORT_BOOLEAN_TRUE_TOKENS,
IMPORT_BOOLEAN_FALSE_TOKENS,
IMPORT_REFERENCE_TYPES,
classifyFilterToken,
} from '@objectstack/spec/data';
import type { FieldErrorCode } from '@objectstack/spec/api';
import {
Expand Down Expand Up @@ -270,7 +276,38 @@ function pad2(n: number): string {
return n < 10 ? `0${n}` : String(n);
}

const TIME_OF_DAY = /^([01]\d|2[0-3]):[0-5]\d(:[0-5]\d)?$/;
/**
* [#20722] A `time` cell read by `@objectstack/core`'s one `time` rule, or
* `undefined` when that rule refuses it. It is the rule the write door asks of
* a written `time` (#20671) and the comparand door asks of a filter value, so a
* cell is admitted here exactly when the same value is admitted there, and is
* stored as the same wall clock:
*
* - the verdict is `isUninterpretableTemporalComparand('time', …)`: a bare
* `HH:MM[:SS[.f…]]` in range, or an instant in an ISO 8601 spelling a
* `datetime` is written in, on a day that exists, whose UTC year has four
* digits. A time of day with a `Z` or an offset (`10:00Z`) is refused: a
* `time` carries no zone (ADR-0053 D-C1);
* - the value is `temporalStorageForm(…, 'time')`: `HH:MM:SS`, `.fff` kept
* when non-zero (a fraction past milliseconds truncated), and an instant's
* UTC time of day, fraction included.
*
* The storage form is not a verdict on its own: it hands a string it cannot
* read back unchanged, and it reads `07/15/2026 10:00` through `Date.parse`
* in the host's zone and rolls `2026-02-30T10:00:00Z` over to March 2, both of
* which the verdict refuses. A `{placeholder}` is filter vocabulary, which the
* verdict steps around rather than judges, so it is refused here, as the write
* door refuses it.
*
* This replaced a private pattern with no fractional part, which refused the
* `10:00:00.250` that `/export` writes for a `time` with milliseconds, so such
* a row did not re-import.
*/
function readTimeOfDayCell(s: string): string | undefined {
if (classifyFilterToken(s) !== null || isUninterpretableTemporalComparand('time', s)) return undefined;
const form = temporalStorageForm(s, 'time');
return typeof form === 'string' ? form : undefined;
}

/**
* [#20534] The text shapes a `date` / `datetime` / `time` cell is read in,
Expand Down Expand Up @@ -416,19 +453,24 @@ function utcClock(t: Date): string {
* Coerce a cell into the string shape the engine accepts for a date-ish field:
* - `date` → `YYYY-MM-DD`
* - `datetime` → full ISO-8601 (`toISOString`)
* - `time` → `HH:MM` / `HH:MM:SS`
* - `time` → `HH:MM:SS`, `.fff` when non-zero ({@link readTimeOfDayCell})
* Returns `undefined` when the cell is not a recognisable date/time, and the
* caller fails the row with `invalid_date` (`import_invalid_date` /
* `import_invalid_datetime` / `import_invalid_time`).
* caller fails the row with the write door's code for the same cell:
* `invalid_date` for a `date` / `datetime` (`import_invalid_date` /
* `import_invalid_datetime`), `invalid_time` for a `time`
* (`import_invalid_time`).
*
* ## Which text is read at all (#20534)
*
* A text cell is read only in an {@link ISO_TEMPORAL_CELL} shape — ISO 8601,
* or the export's own `YYYY-MM-DD HH:mm:ss` — or as a year-first date
* ({@link YEAR_FIRST_CELL}: `2026/7/15`, `2026/7/15 9:00`, stored padded, the
* clock read exactly as the export shape's), and only on a calendar day that
* exists. A `time` cell may also be a bare `HH:MM` / `HH:MM:SS`. Every other
* cell is refused on every branch; nothing reaches `new Date(s)`:
* exists. A `time` cell is read by core's one `time` rule instead
* ({@link readTimeOfDayCell}, #20722): a bare `HH:MM[:SS[.f…]]` or an ISO
* 8601 instant, as the write door reads it, and otherwise in the year-first
* form. Every other cell is refused on every branch; nothing reaches
* `new Date(s)`:
*
* - **an impossible day** (`2026-02-30`, `2026-02-29`, `2026-04-31`,
* `2026/2/30`, in any of the shapes) — `Date.UTC` and `Date.parse` rolled it
Expand Down Expand Up @@ -482,9 +524,10 @@ function utcClock(t: Date): string {
* see it.
*
* For a naive cell landing in a `date` or `time` field the typed components are
* taken verbatim (`2026-08-01 06:00:00` → `2026-08-01` / `06:00:00`), which is
* both zone-free and host-`TZ`-free. An offset-bearing cell landing in either
* takes the UTC calendar day or UTC clock of the instant it names.
* taken verbatim (`2026-08-01 06:00:00` → `2026-08-01` / `06:00:00`, a `time`
* keeping a non-zero fraction), which is both zone-free and host-`TZ`-free. An
* offset-bearing cell landing in either takes the UTC calendar day or UTC
* clock of the instant it names.
*/
export function parseDateCell(
raw: unknown,
Expand All @@ -503,9 +546,15 @@ export function parseDateCell(
const s = String(raw).trim();
if (s === '') return undefined;

if (kind === 'time' && TIME_OF_DAY.test(s)) return s.length === 5 ? `${s}:00` : s;
if (kind === 'time') {
const clock = readTimeOfDayCell(s);
if (clock !== undefined) return clock;
}

const cell = readIsoTemporalCell(s) ?? readYearFirstCell(s);
// [#20722] Core's rule is the whole verdict on an ISO 8601 `time` cell, so a
// `time` cell it refuses is read in the year-first form alone — the one
// reading the import has beyond the write door (#20534, maintainer ruling).
const cell = (kind === 'time' ? undefined : readIsoTemporalCell(s)) ?? readYearFirstCell(s);
if (!cell) return undefined;

if (cell.wall) {
Expand Down Expand Up @@ -593,9 +642,11 @@ export async function coerceFieldValue(
// The business timezone an offset-free datetime cell is read in (#8485).
const d = parseDateCell(raw, t, ctx.timezone);
if (d === undefined) {
// One code, three sentences — a `time` cell is not "not a valid date".
// [#20722] The write door's code for the same cell — `invalid_time` for a
// `time`, `invalid_date` for a `date` / `datetime` — and three sentences.
const code = t === 'time' ? 'invalid_time' : 'invalid_date';
const key = t === 'datetime' ? 'import_invalid_datetime' : t === 'time' ? 'import_invalid_time' : 'import_invalid_date';
return coerceError(meta, field, 'invalid_date', key, raw, ctx);
return coerceError(meta, field, code, key, raw, ctx);
}
return { value: d };
}
Expand Down
11 changes: 7 additions & 4 deletions packages/rest/src/import-date-cell-iso-real-day.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,9 @@ const REFUSED: ReadonlyArray<readonly [field: Field, cell: string]> = [
['d', '2026/2/30'],
];

/** [#20722] The write door's field code for a refused cell: `invalid_time` for a `time`. */
const codeOf = (field: Field) => (field === 't' ? 'invalid_time' : 'invalid_date');

/** Admitted cells and what they store — the padding, the ISO control and the export shape. */
const ADMITTED: ReadonlyArray<readonly [field: Field, cell: string, stored: string]> = [
['d', '0500-01-01', '0500-01-01'],
Expand Down Expand Up @@ -180,7 +183,7 @@ describe.each(HOST_ZONES)('[#20534] /import date cells, host TZ=$tz', ({ tz, jul
else process.env.TZ = originalTz;
});

it.each(REFUSED)('refuses the %s cell %j as that row\'s invalid_date and writes its sibling row', async (field, cell) => {
it.each(REFUSED)('refuses the %s cell %j as that row\'s invalid_date (a time cell: invalid_time) and writes its sibling row', async (field, cell) => {
const res = await ctx.importRows({
format: 'json', writeMode: 'insert',
rows: [{ id: 'bad', [field]: cell }, { id: 'good', d: '2026-07-15' }],
Expand All @@ -189,7 +192,7 @@ describe.each(HOST_ZONES)('[#20534] /import date cells, host TZ=$tz', ({ tz, jul
expect(res._status ?? 200).toBe(200);
expect(res._json).toMatchObject({ total: 2, ok: 1, errors: 1, created: 1 });
expect(res._json.results[0]).toMatchObject({
row: 1, ok: false, action: 'failed', field, code: 'invalid_date',
row: 1, ok: false, action: 'failed', field, code: codeOf(field),
});
// Refused, not stored as some other day or instant.
expect(await ctx.engine.findOne(OBJECT, { where: { id: 'bad' } })).toBeNull();
Expand Down Expand Up @@ -267,7 +270,7 @@ describe.each(HOST_ZONES)('[#20534] /import date cells, host TZ=$tz', ({ tz, jul

expect(res._json).toMatchObject({ total: REFUSED.length, ok: 0, errors: REFUSED.length });
const failedFields = res._json.results.map((r: any) => [r.field, r.code]);
expect(failedFields).toEqual(REFUSED.map(([field]) => [field, 'invalid_date']));
expect(failedFields).toEqual(REFUSED.map(([field]) => [field, codeOf(field)]));
});

it('dry run predicts the same refusals and persists nothing', async () => {
Expand All @@ -278,7 +281,7 @@ describe.each(HOST_ZONES)('[#20534] /import date cells, host TZ=$tz', ({ tz, jul

expect(res._json).toMatchObject({ dryRun: true, total: REFUSED.length, ok: 0, errors: REFUSED.length });
for (const [i, r] of res._json.results.entries()) {
expect(r).toMatchObject({ ok: false, field: REFUSED[i][0], code: 'invalid_date' });
expect(r).toMatchObject({ ok: false, field: REFUSED[i][0], code: codeOf(REFUSED[i][0]) });
}
for (const [i] of REFUSED.entries()) {
expect(await ctx.engine.findOne(OBJECT, { where: { id: `d${i}` } })).toBeNull();
Expand Down
Loading
Loading