diff --git a/.changeset/7189-grouped-grid-partial-disclosure.md b/.changeset/7189-grouped-grid-partial-disclosure.md index 16e6a9461b..946c03e7cd 100644 --- a/.changeset/7189-grouped-grid-partial-disclosure.md +++ b/.changeset/7189-grouped-grid-partial-disclosure.md @@ -48,3 +48,22 @@ toward it or changes the fetch. ten locale packs; `GroupRow` gains two optional props (`partialLabel`, `partialTitle`) and is otherwise unchanged. No metadata schema key was added: the condition is derived from data the grid already has. + +⚠️ **Dated note, 2026-09-28 — grid grouping is now server-side where the data +source can answer the group header query — objectui#7189, PR objectui#10878.** +The paragraphs above describe this change as it was written. Later in this same +release three of their sentences stopped describing the grid: that +`useGroupedData` buckets the rows the browser already holds, so the set of +groups and every header number are properties of the fetched page; that +client-side grouping is **unchanged** here; and that server-side grouping is not +part of this, is still open on objectui#5560, and that nothing here builds +toward it or changes the fetch. Maintainer ruling A on objectui#7189 settled that +question. On a data source that declares `queryGroupHeaders`, a grouped grid +that fetches its own rows now takes its groups, every group count and every +per-group aggregation from the query, and pages each group's rows on the +server; its counts are the query's own, so the `Partial` marker and the notice +described above never render there. On a data source without +`queryGroupHeaders`, the grid still groups the page it fetched in the browser, +as described above, and the marker and the notice are kept there. The rest of +this entry is kept as the reading of this change; the +`7189-server-side-grid-grouping` entry states what a grouped grid does now. diff --git a/.changeset/7189-server-side-grid-grouping.md b/.changeset/7189-server-side-grid-grouping.md new file mode 100644 index 0000000000..ad7e7ce1b3 --- /dev/null +++ b/.changeset/7189-server-side-grid-grouping.md @@ -0,0 +1,68 @@ +--- +'@object-ui/types': minor +'@object-ui/data-objectstack': minor +'@object-ui/plugin-grid': minor +'@object-ui/plugin-list': minor +--- + +Grid grouping is now **server-side** (objectui#7189, maintainer ruling A): the +set of groups and every number in a group header — the count and any per-group +aggregation — come from the query, and the rows inside a group are paged by the +server. + +Before, a grouped grid fetched one window and bucketed it in the browser, so +both the group set and every header number were properties of the page. On a +store of 186 records over five business units sized 86 / 61 / 31 / 7 / 1 with a +100-row page it rendered **two** headers (86, 14) when the rows were stored +contiguously and five page slices (31 / 31 / 30 / 7 / 1) when interleaved — and +the records past the window were not on a second page, they were unreachable. +The same store now renders five headers reading 86 / 61 / 31 / 7 / 1 in either +order, and every record is reachable through its group's own pager. + +- **`@object-ui/types`** — `DataSource` gains an optional + `queryGroupHeaders(resource, query)`: it answers the group header query + `@objectstack/spec/ui`'s `compileListViewGroupQuery` compiles (an + `EngineAggregateOptions`) with one header row per group. Presence is the + capability; a source that cannot answer it leaves it undeclared. +- **`@object-ui/data-objectstack`** — `ObjectStackAdapter.queryGroupHeaders` + posts the compiled query verbatim to the existing `POST /data/:object/query` + door and answers its `records`. It never degrades: a refusal throws, and a + body without `records` is refused rather than read as "no groups". +- **`@object-ui/plugin-grid`** — a grouped grid that fetches its own rows from a + source declaring `queryGroupHeaders` asks for its headers (one compiled query + per grouping level) and pages each **open** group's rows with the group's + own compiled row query (`compileListViewGroupRowsQuery`: the view's filter AND + the group key, `limit` / `offset` per group); a collapsed group costs no row + query, `aggregations` are the header query's numbers, and a reference-typed + grouping key is labelled from the referenced record. Such a grid's column + sort is the server's, and it no longer shows the `Partial` marker: its counts + are the query's own. Rows handed in whole are still grouped in the browser + (exact), and a source with no header query still groups the page it fetched + and still marks it partial. +- **`@object-ui/plugin-list`** — `ListView` hands a grouped grid its own fetch, + with the view's effective filter, when the data source can answer the header + query, and draws no record-count bar over it. While a toolbar search is + active it keeps hosting the rows as before: `$search` has no counterpart on + the header query. + +**Breaking semantics (declared `minor` per this repo's version policy):** + +- A grouped grid over a capable source sends different requests: one header + query per grouping level plus one row query per open group, instead of one + window. A host that counted or mocked the single window sees the new shape. +- `useGroupedData` takes an optional fifth argument (the server's header rows). +- `GroupEntry` gains two REQUIRED members: `count` (the group's size — use it + instead of `rows.length`, which is empty on a server-grouped grid) and + `keyValues`. Code that only READS a `GroupEntry` is unaffected; code that + CONSTRUCTS a `GroupEntry` literal no longer type-checks until it supplies + both. +- `AggregationResult.value` is `number | null`, `null` being what the server's + aggregate answered over no values (rendered as a dash, never invented as 0). +- `@object-ui/plugin-grid` and `@object-ui/data-objectstack` now require + `@objectstack/spec` `^17.4.0`, the first release that ships the group-query + compilers. + +This supersedes the last paragraph of the pending +`7189-grouped-grid-partial-disclosure` changeset, which said server-side +grouping was not built: it now is, and the `Partial` marker described there +remains only for the data sources that cannot answer the header query. diff --git a/content/docs/api/schema-reference.md b/content/docs/api/schema-reference.md index 67378888de..1c0b22f09c 100644 --- a/content/docs/api/schema-reference.md +++ b/content/docs/api/schema-reference.md @@ -781,7 +781,7 @@ A data grid that auto-fetches from an ObjectQL object definition. Includes searc | `operations` | `object` | Enabled CRUD operations. | | `rowActions` / `bulkActions` | `string[]` | Action identifiers for rows and batch selection. `bulkActions` is the spec-aligned key; `batchActions` is a legacy alias that takes precedence when both are set. | | `editable` | `boolean` | Enable inline cell editing. | -| `grouping` | `GroupingConfig` | Row grouping configuration. **Page-scoped**: the grid groups the rows it has fetched, so group counts are page slices and a group beyond the page is absent — the grid marks the grouping partial when it can tell. | +| `grouping` | `GroupingConfig` | Row grouping configuration. **Server-side**: the set of groups, every group count and every per-group aggregation come from the group header query (`dataSource.queryGroupHeaders`), and each group's rows are paged by the server. Rows handed in whole are grouped in the browser (exact); over a data source with no header query the grid groups the page it fetched and marks the counts partial. | | `frozenColumns` | `number` | Number of columns frozen on scroll. | | `navigation` | `ViewNavigationConfig` | SPA navigation configuration. | diff --git a/content/docs/plugins/plugin-grid.mdx b/content/docs/plugins/plugin-grid.mdx index d4a24bf5cf..5d81c27f75 100644 --- a/content/docs/plugins/plugin-grid.mdx +++ b/content/docs/plugins/plugin-grid.mdx @@ -95,7 +95,7 @@ below for why this plugin deliberately does not claim it. | `navigation` | `NavigationConfig` | What a row click does: `{ mode: 'page' \| 'drawer' \| 'modal' \| 'split' \| 'none', … }`. | | `operations` | `object` | Toggles the built-in CRUD/export/import affordances, e.g. `{ delete: false }`. | | `rowHeight`, `frozenColumns`, `resizable`, `reorderableColumns`, `showColumnTypeIcons`, `rowColor`, `conditionalFormatting`, `aggregations`, `exportOptions`, `className` | | The rest of the declared surface. | -| `grouping` | `GroupingConfig` | Row grouping — **page-scoped**, see [Grouping is page-scoped](#grouping-is-page-scoped). | +| `grouping` | `GroupingConfig` | Row grouping — **server-side**: group set, counts and aggregations come from the query, rows are paged per group; see [Grouping is server-side](#grouping-is-server-side). | **There is no `sortable`, `filterable`, `onRowClick`, `onSelectionChange`, `onCellChange`, `onRowSave`, `onBatchSave` or `object` on this schema.** The two @@ -335,37 +335,50 @@ see [Row callbacks are component props](#row-callbacks-are-component-props). always carries a rows-per-page picker, and `pageSizeOptions` only replaces the choices it offers with your own. -### Grouping is page-scoped - -`grouping` groups **the records the browser has already fetched** — one page — -not the whole result set. The grid buckets the rows in hand and computes every -per-group count and aggregation from that same array, so both the set of groups -and every number in a group header are properties of the page, not of the query. - -Two consequences, and the second is the one to decide against before you ship -the view: - -- Every group count is a page slice, so it can be lower than the group's real - size. -- **A group whose records all fall beyond the page does not appear at all.** On - a grid read as an org lens ("what is outstanding, per business unit"), a unit - that is silently absent is a wrong answer that looks like good news. - -Since objectui#7189 the grid says so on screen rather than leaving it silent. -When the result set is larger than the rows loaded, a short `Partial` marker -appears beside **every group count** — where the authoritative-looking number -is — and a line above the group list carries the whole sentence, e.g. -*"Grouped over the first 100 of 186 records…"*. When no match total is -reachable the wording weakens honestly to *"Grouped over the 100 records -loaded. More may match this view…"*. A grouped grid whose result set fits in -one page shows neither: the marker is conditional, which is what makes it worth -reading. - -The disclosure is not a fix for the scoping — it makes it visible. If a view -needs true totals per group today, the working options are to narrow the query -until the result set fits one page (a `filter` that scopes the lens), to raise -`pagination.pageSize` past the result set, or to compute the aggregation server -side and render it from a dataset rather than from grid grouping. +### Grouping is server-side + +`grouping` is answered by the **server**, not by bucketing the records the +browser happens to hold (objectui#7189, maintainer ruling A). The set of groups +and every number in a group header — the count and any per-group aggregation — +are properties of the query; the rows inside a group are paged. + +A grouped grid that fetches its own rows asks its data source for the group +headers — the query `@objectstack/spec/ui`'s `compileListViewGroupQuery` +compiles, sent through `dataSource.queryGroupHeaders` (the ObjectStack adapter +posts it to `POST /data/:object/query`) — and then pages each **open** group's +rows with that group's own query (`compileListViewGroupRowsQuery`: the view's +filter AND the group's key, `limit` / `offset` per group). So: + +- Every group appears, with its **true size**, whatever the page size and + whatever order the rows are stored in — a store of 186 records over five + units renders five headers reading 86/61/31/7/1. +- A group larger than the page gets its **own pager**, and turning it asks the + server for that group's next page, so every record is reachable. +- A collapsed group costs no row query at all. +- `aggregations` are computed by the same header query, over the group's whole + row set. +- Multi-level grouping asks one header query per level, so an outer header's + numbers are that level's own (an average is never an average of averages). +- A `lookup` / `master_detail` / `user` grouping key is the referenced record's + id on the wire; the grid labels it from the referenced record. + +When a `ListView` hosts the grid (the console's list views), it hands a grouped +grid its own fetch and the view's effective filter, rather than a window of +rows. + +Two cases still group **in the browser**: + +- **Rows handed in whole** (`data: { provider: 'value', items }`, or a host's + whole result set): nothing was withheld, so grouping them there is exact. +- **A data source that declares no `queryGroupHeaders`**: the grid can only + group the page it fetched. Every count is then a page slice and a group whose + records all fall past the page is absent, so the grid says so where the + numbers are — a short `Partial` marker beside every group count and a line + above the group list (*"Grouped over the first 100 of 186 records…"*). With + counts from the server the marker never appears. + +A toolbar **search** has no counterpart on the group header query, so a +`ListView` keeps grouping its own window while a search term is active. ### ObjectQL Integration diff --git a/content/docs/utilities/data-objectstack.mdx b/content/docs/utilities/data-objectstack.mdx index 1927793690..ccee1a2405 100644 --- a/content/docs/utilities/data-objectstack.mdx +++ b/content/docs/utilities/data-objectstack.mdx @@ -225,12 +225,16 @@ adapter): - `bulk?(resource, operation, data)` - Batch create/update/delete on one object - `batchTransaction?(operations)` - Cross-object atomic batch (master-detail) - `onMutation?(listener)` - Subscribe to create/update/delete events - -`bulk`, `batchTransaction` and `onMutation` are **optional** members of -`DataSource`: not every adapter implements them, so through a `DataSource`-typed -value they must be feature-detected (`typeof dataSource.bulk === 'function'`). -`ObjectStackAdapter` implements all three unconditionally, and the factory -declares the class, so a value from either form calls them directly. (`onMutation` +- `queryGroupHeaders?(resource, query)` - Answer a grouped list view's group + header query (`compileListViewGroupQuery`), one row per group with its true + `count` — what a grouped grid groups by on the server (objectui#7189) + +`bulk`, `batchTransaction`, `onMutation` and `queryGroupHeaders` are +**optional** members of `DataSource`: not every adapter implements them, so +through a `DataSource`-typed value they must be feature-detected +(`typeof dataSource.bulk === 'function'`). `ObjectStackAdapter` implements all +four unconditionally, and the factory declares the class, so a value from +either form calls them directly. (`onMutation` was listed under *Adapter-only* here until objectui#7323; it is optional on `DataSource`, not absent from it.) diff --git a/packages/data-objectstack/README.md b/packages/data-objectstack/README.md index 52e8e48c11..5caff6336c 100644 --- a/packages/data-objectstack/README.md +++ b/packages/data-objectstack/README.md @@ -361,6 +361,25 @@ possibly-absent authored values keep working. Keys outside those three are not refused either — the gate names `filter`, `field` and `function`, and only those. +### Group headers for server-side grid grouping + +`queryGroupHeaders(resource, query)` answers a grouped list view's **group +header query** — the `EngineAggregateOptions` that `@objectstack/spec/ui`'s +`compileListViewGroupQuery` compiles from the view's `grouping` (and its column +summaries). It is how a grouped `object-grid` gets its group set and every +header number from the server instead of bucketing a fetched page +(objectui#7189): one row per group, each grouped field under its own name +holding the raw stored value (`null` for the empty group), and the group's +total row count under `count`. + +The query is posted **verbatim** to the existing `POST /data/:object/query` +door (the same one the spec-shape `aggregate()` branch uses), and the answer is +its `records`. Unlike `aggregate()`, it never degrades to a client-side +reduction: a refusal is thrown with the server's code and status, and an answer +without a `records` array is refused rather than read as "no groups". The rows +inside a group are an ordinary `find()` with the group's compiled row query +(`compileListViewGroupRowsQuery`). + ### Sorting ```typescript diff --git a/packages/data-objectstack/package.json b/packages/data-objectstack/package.json index fe5563a973..a8c9152584 100644 --- a/packages/data-objectstack/package.json +++ b/packages/data-objectstack/package.json @@ -32,7 +32,7 @@ "@object-ui/core": "workspace:*", "@object-ui/types": "workspace:*", "@objectstack/client": "^17.3.0", - "@objectstack/spec": "^17.2.0" + "@objectstack/spec": "^17.4.0" }, "devDependencies": { "@object-ui/test-support": "workspace:*", diff --git a/packages/data-objectstack/src/index.ts b/packages/data-objectstack/src/index.ts index 8ba0378268..c2f404aaaf 100644 --- a/packages/data-objectstack/src/index.ts +++ b/packages/data-objectstack/src/index.ts @@ -7,7 +7,8 @@ */ import { ObjectStackClient, type QueryOptions as ObjectStackQueryOptions } from '@objectstack/client'; -import type { DroppedFieldsEvent } from '@objectstack/spec/data'; +import type { DroppedFieldsEvent, EngineAggregateOptions } from '@objectstack/spec/data'; +import type { ListViewGroupHeaderRow } from '@objectstack/spec/ui'; // #4934 — a VALUE import, not a type one: the write-warning boundary parses the // wire's `reason` against the enum the spec itself declares, so the accept set // is read off the pin instead of hand-copied here (a hand copy is the drift @@ -6208,6 +6209,40 @@ export class ObjectStackAdapter implements DataSource { } } + /** + * Answer a grouped list view's GROUP HEADER query (objectui#7189) — see + * `DataSource.queryGroupHeaders` in `@object-ui/types` for the contract. + * + * The compiled query rides the platform's EXISTING data door verbatim: + * `client.data.query()` posts it to `POST /data/:object/query`, whose + * `findData` routes a body carrying `groupBy` / `aggregations` to + * `engine.aggregate` and answers the header rows as `records` (the door the + * spec half of objectstack#14556 names, pinned platform-side by + * objectstack#15330). No new route, no new wire shape. + * + * ⛔ No fallback, deliberately — unlike {@link aggregate}. A header query that + * degraded to bucketing a fetched page would answer page slices under a name + * that promises the query's own numbers, which is the defect this member + * exists to end. A failure is thrown to the caller, and a body without a + * `records` array is refused rather than read as "no groups". + */ + async queryGroupHeaders( + resource: string, + query: EngineAggregateOptions, + ): Promise { + await this.connect(); + const result: unknown = await this.client.data.query(resource, query as any); + const records = (result as { records?: unknown } | null)?.records; + if (!Array.isArray(records)) { + throw new Error( + `queryGroupHeaders('${resource}'): POST /data/${resource}/query answered without a ` + + '`records` array, so no group header can be read from it. The header query was ' + + 'sent verbatim; nothing was approximated from a page of rows.', + ); + } + return records as ListViewGroupHeaderRow[]; + } + /** * Client-side aggregation over a server-scoped `find()` — the fallback used * whenever the analytics endpoint cannot answer (capability absent, network diff --git a/packages/data-objectstack/src/queryGroupHeaders-7189.test.ts b/packages/data-objectstack/src/queryGroupHeaders-7189.test.ts new file mode 100644 index 0000000000..c6ba7cf047 --- /dev/null +++ b/packages/data-objectstack/src/queryGroupHeaders-7189.test.ts @@ -0,0 +1,119 @@ +/** + * ObjectUI + * Copyright (c) 2024-present ObjectStack Inc. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */ + +/** + * `queryGroupHeaders()` — the transport for server-side grid grouping + * (objectui#7189, maintainer ruling A). + * + * A grouped grid that owns its fetch hands this member the header query + * `@objectstack/spec/ui`'s `compileListViewGroupQuery` compiles, and gets one + * row per group back. The spec half of the ruling (objectstack#14556) named the + * door: the EXISTING `POST /data/:object/query`, whose `findData` routes a body + * carrying `groupBy` / `aggregations` to `engine.aggregate` and answers the + * header rows as `records`. These pins hold the adapter to that door: + * + * - the compiled query goes out VERBATIM — the body on the wire IS the + * compiler's output, so a pin here goes red when either end drifts; + * - the answer is the `records` array, untouched; + * - there is NO fallback: a refused query throws, and a body with no + * `records` is refused rather than read as "no groups" — the two ways a + * degrading member would hand a grid page slices under a name that promises + * the query's own numbers. + */ + +import { describe, it, expect, vi } from 'vitest'; +import { compileListViewGroupQuery } from '@objectstack/spec/ui'; +import { ObjectStackAdapter } from './index'; + +const HEADER_ROWS = [ + { business_unit: 'northgate_operations', count: 86 }, + { business_unit: 'northgate_quality', count: 61 }, + { business_unit: 'riverside_plant', count: 31 }, + { business_unit: 'northgate_plant', count: 7 }, + { business_unit: 'harbour_office', count: 1 }, +]; + +function makeAdapter(answer: { status: number; body: unknown }) { + const posted: Array<{ url: string; method?: string; body: unknown }> = []; + const fetchImpl = vi.fn(async (url: any, init?: any) => { + const u = String(url); + if (u.includes('/api/v1/discovery')) { + return { + ok: true, status: 200, statusText: 'OK', + json: async () => ({ success: true, data: { version: 'v1', routes: {} } }), + } as any; + } + posted.push({ url: u, method: init?.method, body: init?.body ? JSON.parse(String(init.body)) : undefined }); + return { + ok: answer.status < 400, status: answer.status, statusText: String(answer.status), + json: async () => answer.body, + } as any; + }); + const adapter = new ObjectStackAdapter({ + baseUrl: 'http://localhost:3000', token: 't', autoReconnect: false, fetch: fetchImpl as any, + }); + return { adapter, posted }; +} + +const VIEW = { + grouping: { fields: [{ field: 'business_unit' }] }, + columns: [{ field: 'amount', summary: 'sum' as const }], +}; + +describe('ObjectStackAdapter.queryGroupHeaders (objectui#7189)', () => { + it('posts the compiled header query verbatim to POST /data/:object/query and answers its records', async () => { + const { adapter, posted } = makeAdapter({ + status: 200, + body: { success: true, data: { object: 'work_item', records: HEADER_ROWS, total: 5, hasMore: false } }, + }); + const query = compileListViewGroupQuery(VIEW, { where: { status: { $eq: 'open' } } }); + + const rows = await adapter.queryGroupHeaders('work_item', query); + + expect(posted).toHaveLength(1); + expect(posted[0].method).toBe('POST'); + expect(posted[0].url).toMatch(/\/api\/v1\/data\/work_item\/query$/); + // The body IS the compiler's output — not a restatement of it. + expect(posted[0].body).toEqual(JSON.parse(JSON.stringify(query))); + expect(rows).toEqual(HEADER_ROWS); + }); + + it('refuses an answer with no `records` array instead of reading it as "no groups"', async () => { + const { adapter } = makeAdapter({ status: 200, body: { success: true, data: { object: 'work_item', total: 0 } } }); + const error = await adapter + .queryGroupHeaders('work_item', compileListViewGroupQuery(VIEW)) + .then(() => null, (e: unknown) => e); + expect(error).toBeInstanceOf(Error); + expect(String((error as Error).message)).toContain('records'); + }); + + it('CONTROL: an empty `records` array IS an answer — zero groups, no refusal', async () => { + const { adapter } = makeAdapter({ + status: 200, + body: { success: true, data: { object: 'work_item', records: [], total: 0, hasMore: false } }, + }); + await expect(adapter.queryGroupHeaders('work_item', compileListViewGroupQuery(VIEW))).resolves.toEqual([]); + }); + + it('throws the server refusal with its ADR-0112 code and status — no client-side fallback', async () => { + const { adapter, posted } = makeAdapter({ + status: 400, + body: { success: false, error: { code: 'INVALID_FIELD', message: "Unknown field 'nope'" } }, + }); + const error = await adapter + .queryGroupHeaders('work_item', compileListViewGroupQuery({ grouping: { fields: [{ field: 'nope' }] } })) + .then(() => null, (e: unknown) => e) as { code?: string; httpStatus?: number; status?: number } | null; + expect(error).not.toBeNull(); + expect(error?.code).toBe('INVALID_FIELD'); + expect(error?.httpStatus ?? error?.status).toBe(400); + // One request, and it was the header query: nothing re-read the rows to + // approximate the groups. + expect(posted).toHaveLength(1); + expect(posted[0].url).toMatch(/\/data\/work_item\/query$/); + }); +}); diff --git a/packages/plugin-grid/README.md b/packages/plugin-grid/README.md index d2dd4e8b11..93b931e286 100644 --- a/packages/plugin-grid/README.md +++ b/packages/plugin-grid/README.md @@ -194,7 +194,7 @@ const grid: ObjectGridSchema = { | `navigation` | `NavigationConfig` | What a row click does, `{ mode: 'page' \| 'drawer' \| 'modal' \| 'split' \| 'none', … }`. | | `operations` | `object` | Toggles the built-in CRUD/export/import affordances, e.g. `{ delete: false }`. | | `rowHeight`, `frozenColumns`, `resizable`, `reorderableColumns`, `showColumnTypeIcons`, `rowColor`, `conditionalFormatting`, `aggregations`, `exportOptions`, `className` | | The rest of the declared surface. | -| `grouping` | `GroupingConfig` | Row grouping — **page-scoped**, see [Grouping is page-scoped](#grouping-is-page-scoped). | +| `grouping` | `GroupingConfig` | Row grouping — **server-side**: group set, counts and aggregations come from the query, rows are paged per group; see [Grouping is server-side](#grouping-is-server-side). | **There is no `sortable`, `filterable`, `onRowClick`, `onSelectionChange`, `onCellChange`, `onRowSave`, `onBatchSave` or `object` on this schema.** The @@ -453,37 +453,50 @@ const schema: ObjectGridSchema = { always carries a rows-per-page picker, and `pageSizeOptions` only replaces the choices it offers with your own. -### Grouping is page-scoped - -`grouping` groups **the records the browser has already fetched** — one page — -not the whole result set. The grid buckets the rows in hand and computes every -per-group count and aggregation from that same array, so both the set of groups -and every number in a group header are properties of the page, not of the query. - -Two consequences, and the second is the one to decide against before you ship -the view: - -- Every group count is a page slice, so it can be lower than the group's real - size. -- **A group whose records all fall beyond the page does not appear at all.** On - a grid read as an org lens ("what is outstanding, per business unit"), a unit - that is silently absent is a wrong answer that looks like good news. - -Since objectui#7189 the grid says so on screen rather than leaving it silent. -When the result set is larger than the rows loaded, a short `Partial` marker -appears beside **every group count** — where the authoritative-looking number -is — and a line above the group list carries the whole sentence, e.g. -*"Grouped over the first 100 of 186 records…"*. When no match total is -reachable the wording weakens honestly to *"Grouped over the 100 records -loaded. More may match this view…"*. A grouped grid whose result set fits in -one page shows neither: the marker is conditional, which is what makes it worth -reading. - -The disclosure is not a fix for the scoping — it makes it visible. If a view -needs true totals per group today, the working options are to narrow the query -until the result set fits one page (a `filter` that scopes the lens), to raise -`pagination.pageSize` past the result set, or to compute the aggregation server -side and render it from a dataset rather than from grid grouping. +### Grouping is server-side + +`grouping` is answered by the **server**, not by bucketing the records the +browser happens to hold (objectui#7189, maintainer ruling A). The set of groups +and every number in a group header — the count and any per-group aggregation — +are properties of the query; the rows inside a group are paged. + +A grouped grid that fetches its own rows asks its data source for the group +headers — the query `@objectstack/spec/ui`'s `compileListViewGroupQuery` +compiles, sent through `dataSource.queryGroupHeaders` (the ObjectStack adapter +posts it to `POST /data/:object/query`) — and then pages each **open** group's +rows with that group's own query (`compileListViewGroupRowsQuery`: the view's +filter AND the group's key, `limit` / `offset` per group). So: + +- Every group appears, with its **true size**, whatever the page size and + whatever order the rows are stored in — a store of 186 records over five + units renders five headers reading 86/61/31/7/1. +- A group larger than the page gets its **own pager**, and turning it asks the + server for that group's next page, so every record is reachable. +- A collapsed group costs no row query at all. +- `aggregations` are computed by the same header query, over the group's whole + row set. +- Multi-level grouping asks one header query per level, so an outer header's + numbers are that level's own (an average is never an average of averages). +- A `lookup` / `master_detail` / `user` grouping key is the referenced record's + id on the wire; the grid labels it from the referenced record. + +When a `ListView` hosts the grid (the console's list views), it hands a grouped +grid its own fetch and the view's effective filter, rather than a window of +rows. + +Two cases still group **in the browser**: + +- **Rows handed in whole** (`data: { provider: 'value', items }`, or a host's + whole result set): nothing was withheld, so grouping them there is exact. +- **A data source that declares no `queryGroupHeaders`**: the grid can only + group the page it fetched. Every count is then a page slice and a group whose + records all fall past the page is absent, so the grid says so where the + numbers are — a short `Partial` marker beside every group count and a line + above the group list (*"Grouped over the first 100 of 186 records…"*). With + counts from the server the marker never appears. + +A toolbar **search** has no counterpart on the group header query, so a +`ListView` keeps grouping its own window while a search term is active. ## Integration with Data Sources diff --git a/packages/plugin-grid/package.json b/packages/plugin-grid/package.json index f76a7baa44..99b3fa48e3 100644 --- a/packages/plugin-grid/package.json +++ b/packages/plugin-grid/package.json @@ -31,7 +31,7 @@ "@object-ui/plugin-detail": "workspace:*", "@object-ui/react": "workspace:*", "@object-ui/types": "workspace:*", - "@objectstack/spec": "^17.0.0", + "@objectstack/spec": "^17.4.0", "@tanstack/react-virtual": "^3.14.9", "exceljs": "^4.4.0", "lucide-react": "^1.43.0" @@ -44,7 +44,7 @@ "@object-ui/data-objectstack": "workspace:*", "@object-ui/sdui-parser": "workspace:*", "@object-ui/test-support": "workspace:*", - "@objectstack/spec": "^17.0.0", + "@objectstack/spec": "^17.4.0", "@tailwindcss/postcss": "^4.3.3", "@vitejs/plugin-react": "^6.0.5", "msw": "^2.15.0", diff --git a/packages/plugin-grid/src/GroupRow.tsx b/packages/plugin-grid/src/GroupRow.tsx index f24cb8b3e0..c96d790b13 100644 --- a/packages/plugin-grid/src/GroupRow.tsx +++ b/packages/plugin-grid/src/GroupRow.tsx @@ -16,7 +16,10 @@ export interface GroupRowProps { groupKey: string; /** Display label for the group (field value or "(empty)") */ label: string; - /** Number of rows in this group */ + /** + * The group's size. On a server-grouped grid this is the header query's + * `count` — the group's TOTAL, not the rows on screen (objectui#7189). + */ count: number; /** Whether the group is collapsed */ collapsed: boolean; @@ -53,6 +56,9 @@ export interface GroupRowProps { * * `count` is then a page slice, not the group's size, and a group whose * records all fall beyond the loaded rows is absent from the list entirely. + * A server-grouped grid never sets it: its counts are the query's own. It + * remains for a grid grouping a page it fetched from a data source that + * cannot answer the group header query. * The marker lives here, next to the number, because that is the number a * reader treats as authoritative — the paging footer says nothing about * what was grouped, and demonstrably does not prevent the wrong reading. @@ -133,7 +139,7 @@ export const GroupRow: React.FC = ({ {aggregations.map((agg) => ( - {agg.type}: {Number.isInteger(agg.value) ? agg.value : agg.value.toFixed(2)} + {agg.type}: {agg.value === null ? '\u2014' : Number.isInteger(agg.value) ? agg.value : agg.value.toFixed(2)} ))} diff --git a/packages/plugin-grid/src/ObjectGrid.tsx b/packages/plugin-grid/src/ObjectGrid.tsx index 597881f3a7..736556ad1b 100644 --- a/packages/plugin-grid/src/ObjectGrid.tsx +++ b/packages/plugin-grid/src/ObjectGrid.tsx @@ -25,6 +25,7 @@ import React, { useEffect, useState, useCallback, useMemo } from 'react'; import type { ObjectGridSchema, DataSource, ListColumn, TableColumn, ViewData, TableSortItem, DataTableSchema, ListViewExportFormat } from '@object-ui/types'; import { isSystemManagedField, normalizeTableColumnType } from '@object-ui/types'; import type { I18nLabel } from '@objectstack/spec/ui'; +import { parseFilterAST, type FilterCondition } from '@objectstack/spec/data'; import { SchemaRenderer, useDataScope, useNavigationOverlay, useAction, useSafeFieldLabel, usePredicateScope, useRelatedRecordActions, useDataInvalidation, useFilterScope } from '@object-ui/react'; import { createSafeTranslation } from '@object-ui/i18n'; // objectui#8920 — the grid reaches a cell renderer through THIS module and @@ -63,7 +64,8 @@ import { } from '@object-ui/plugin-detail'; import { ChevronRight, ChevronLeft, ChevronsLeft, ChevronsRight, Download, Rows2, Rows3, Rows4, AlignJustify, Type, Hash, Calendar, CheckSquare, User, Tag, Clock, Loader2 } from 'lucide-react'; import { useRowColor } from './useRowColor'; -import { useGroupedData, usableGroupingFields } from './useGroupedData'; +import { useGroupedData, usableGroupingFields, type ServerGroupSource } from './useGroupedData'; +import { useServerGroupHeaders, useServerGroupRows, type ServerGroupLeaf } from './useServerGrouping'; import { GroupRow } from './GroupRow'; import { useColumnSummary } from './useColumnSummary'; import { resolveRowCrudAffordances, resolveRowRecordCrudAffordance } from './rowCrudAffordances'; @@ -2021,6 +2023,100 @@ export const ObjectGrid: React.FC = ({ return () => { cancelled = true; }; }, [hasInlineData, objectName, dataSource]); + // objectui#10583 — a MASKED field is REFUSED as a grouping key. + // Grouping by it printed the raw value as each group's label. Masking the + // label would not be enough: the buckets would still show which records + // share a credential, ordered by its raw value. So the entry is dropped (the + // other entries still group, as `usableGroupingFields` does for an unusable + // one) and the drop is reported through the grid's warning channel. The rule + // is the column flag's: `isMaskedGridColumn` over the view column's type and + // the object-declared type. + // + // objectui#10657 (which folded objectui#10706) — the same rule's third + // argument: while the object's field types are unknown (`objectTypesPending`, + // the host-fetched window, or a failed read), an entry on a field whose view + // column authors no type is WITHHELD and so refused too, since its group + // label would print whatever the field holds. The group label follows the + // cell: a withheld cell draws the mask, and a withheld key labels no group. + // Once the types are known the entry groups again, unless its field is + // masked. The refusal of a withheld entry is not warned: it is not a finding + // about the authored grouping, and a failed read reports itself. + const groupingFieldsRaw = schema.grouping?.fields; + const maskedGroupingSignature = React.useMemo(() => { + const cols: ReadonlyArray = normalizeColumns(schema.columns) ?? []; + const columnTypeOf = (field: string) => + cols.find((c): c is ListColumn => typeof c === 'object' && c !== null && c.field === field)?.type; + const refused = usableGroupingFields(groupingFieldsRaw) + .map((gf) => gf.field) + .filter((field) => isMaskedGridColumn(columnTypeOf(field), objectFields?.[field]?.type, objectTypesPending)); + return JSON.stringify({ + refused, + // The ones refused on a KNOWN masked type, which is what the warning names. + warned: refused.filter((field) => isMaskedGridColumn(columnTypeOf(field), objectFields?.[field]?.type, false)), + }); + }, [groupingFieldsRaw, schema.columns, objectFields, objectTypesPending]); + // Keyed on the authored array and the signature STRING, never on a memo's + // identity (AGENTS.md #10). Read only when something was refused: the + // unmasked path below hands `useGroupedData` the authored config itself. + const unmaskedGroupingFields = React.useMemo(() => { + const { refused }: { refused: string[] } = JSON.parse(maskedGroupingSignature); + return usableGroupingFields(groupingFieldsRaw).filter((gf) => !refused.includes(gf.field)); + }, [groupingFieldsRaw, maskedGroupingSignature]); + const groupingRefusesNothing = maskedGroupingSignature === NO_GROUPING_REFUSAL; + useEffect(() => { + const { warned: masked }: { warned: string[] } = JSON.parse(maskedGroupingSignature); + if (masked.length === 0) return; + console.warn( + `[ObjectUI] ObjectGrid grouping: ${schema.objectName ?? 'object-grid'} groups by the masked ` + + `field(s) ${masked.join(', ')}. A masked field cannot be a grouping key: its group labels would ` + + 'show the raw value, and its groups would show which records share it. The entry was ignored.', + ); + }, [maskedGroupingSignature, schema.objectName]); + + // ── Server-side grouping (objectui#7189, maintainer ruling A) ───────────── + // + // Grouping on a list view is server-side: the set of groups and every number + // in a group header are properties of the QUERY, not of a fetched page, and + // the rows inside a group are paged. So a grouped grid that OWNS its fetch + // asks the server for its group headers (`dataSource.queryGroupHeaders`, + // the compiled `compileListViewGroupQuery`) and pages each open group's rows + // (`compileListViewGroupRowsQuery`) instead of fetching one window and + // bucketing it — which answered two headers (86, 14) or five page slices + // (31/31/30/7/1) for the same five units of 86/61/31/7/1, depending on + // nothing but row order, and left every row past the window unreachable. + // + // Presence of `queryGroupHeaders` is the capability. Rows handed in whole + // (`hasInlineData`) are grouped where they are, which is exact because + // nothing was withheld. A data source that cannot answer the header query + // keeps grouping the page it fetched — and the grid says so on screen (the + // "Partial" marker below), because only there are the counts page slices. + // + // Decided on the grouping AS CONFIGURED less the entries refused on a KNOWN + // masked type — NOT less the entries merely withheld while the object's + // types are still loading, or the first render would fetch one flat window + // it is about to throw away. + const groupingRefusedKnownMasked = React.useMemo( + () => (JSON.parse(maskedGroupingSignature) as { warned: string[] }).warned, + [maskedGroupingSignature], + ); + // A grouping field this principal may not READ is not a key the server may + // group by: a header row carries every key's raw value, so grouping by it + // would print what the column itself withholds. Same gate, same deferral + // (an unanswered policy filters nothing), as the projection's. + const groupingKeyReadable = (field: string): boolean => + !perms?.isLoaded || !objectName || perms.checkField(objectName, field, 'read'); + const serverGroupedFetch = + !hasInlineData + && !!objectName + && typeof dataSource?.queryGroupHeaders === 'function' + && usableGroupingFields(groupingFieldsRaw).some( + (gf) => !groupingRefusedKnownMasked.includes(gf.field) && groupingKeyReadable(gf.field), + ); + // The grid's own row query — projection, expansion, order and the view's + // filter — resolved by the load effect below exactly as the flat fetch + // resolves it, and handed to each group's row page. `null` until resolved. + const [groupRowQuery, setGroupRowQuery] = useState | null>(null); + // objectui#10035 — the refresh input this grid had none of, so a host could // show it a write only by remounting it (AGENTS.md #8's corollary: refresh // data, don't rebuild UI). The nonce moves when the data-invalidation bus @@ -2494,6 +2590,18 @@ export const ObjectGrid: React.FC = ({ params.$expand = expand; } + // [objectui#7189] Server-grouped: no flat window is fetched. The + // query is resolved here all the same — every gate above applies to + // each group's rows exactly as it would to the flat page — and each + // group's row page supplies its own filter and window. + if (serverGroupedFetch) { + const { $top: _top, $skip: _skip, ...groupQuery } = params; + setGroupRowQuery(groupQuery); + lastFindParamsRef.current = { ...groupQuery }; + setSelectAllMatching(false); + return; + } + const result = await dataSource.find(objectName, params); if (cancelled) return; setData(result.data || []); @@ -2543,7 +2651,7 @@ export const ObjectGrid: React.FC = ({ // held with `filter`. It changes only when the authored alias changes by // structure or the scope changes (a new signed-in user), and either one // changes the `$filter` this effect sends when `filter` is absent. - }, [objectName, schemaFields, schemaColumns, schemaFilter, schemaFilterRefusal, schemaDefaultFilters, schemaSort, headerSort, searchTerm, schemaPagination, schemaPageSize, serverPage, serverPageSize, dataSource, hasInlineData, dataConfig, refreshKey, perms.isLoaded, groupingProjectionKey, invalidationNonce]); + }, [objectName, schemaFields, schemaColumns, schemaFilter, schemaFilterRefusal, schemaDefaultFilters, schemaSort, headerSort, searchTerm, schemaPagination, schemaPageSize, serverPage, serverPageSize, dataSource, hasInlineData, dataConfig, refreshKey, perms.isLoaded, groupingProjectionKey, invalidationNonce, serverGroupedFetch]); // The same reset, for the path the loader above never runs on (objectui#4501 // clause 2). "All N matching are selected" is a claim about ONE query, so it @@ -2720,63 +2828,76 @@ export const ObjectGrid: React.FC = ({ }; }, [schema.grouping, schema.columns, schema.objectName, objectFields, translateOptions, t]); - // objectui#10583 — a MASKED field is REFUSED as a grouping key. - // Grouping by it printed the raw value as each group's label. Masking the - // label would not be enough: the buckets would still show which records - // share a credential, ordered by its raw value. So the entry is dropped (the - // other entries still group, as `usableGroupingFields` does for an unusable - // one) and the drop is reported through the grid's warning channel. The rule - // is the column flag's: `isMaskedGridColumn` over the view column's type and - // the object-declared type. - // - // objectui#10657 (which folded objectui#10706) — the same rule's third - // argument: while the object's field types are unknown (`objectTypesPending`, - // the host-fetched window, or a failed read), an entry on a field whose view - // column authors no type is WITHHELD and so refused too, since its group - // label would print whatever the field holds. The group label follows the - // cell: a withheld cell draws the mask, and a withheld key labels no group. - // Once the types are known the entry groups again, unless its field is - // masked. The refusal of a withheld entry is not warned: it is not a finding - // about the authored grouping, and a failed read reports itself. - const groupingFieldsRaw = schema.grouping?.fields; - const maskedGroupingSignature = React.useMemo(() => { - const cols: ReadonlyArray = normalizeColumns(schema.columns) ?? []; - const columnTypeOf = (field: string) => - cols.find((c): c is ListColumn => typeof c === 'object' && c !== null && c.field === field)?.type; - const refused = usableGroupingFields(groupingFieldsRaw) - .map((gf) => gf.field) - .filter((field) => isMaskedGridColumn(columnTypeOf(field), objectFields?.[field]?.type, objectTypesPending)); - return JSON.stringify({ - refused, - // The ones refused on a KNOWN masked type, which is what the warning names. - warned: refused.filter((field) => isMaskedGridColumn(columnTypeOf(field), objectFields?.[field]?.type, false)), - }); - }, [groupingFieldsRaw, schema.columns, objectFields, objectTypesPending]); - // Keyed on the authored array and the signature STRING, never on a memo's - // identity (AGENTS.md #10). Read only when something was refused: the - // unmasked path below hands `useGroupedData` the authored config itself. - const unmaskedGroupingFields = React.useMemo(() => { - const { refused }: { refused: string[] } = JSON.parse(maskedGroupingSignature); - return usableGroupingFields(groupingFieldsRaw).filter((gf) => !refused.includes(gf.field)); - }, [groupingFieldsRaw, maskedGroupingSignature]); - const groupingRefusesNothing = maskedGroupingSignature === NO_GROUPING_REFUSAL; - useEffect(() => { - const { warned: masked }: { warned: string[] } = JSON.parse(maskedGroupingSignature); - if (masked.length === 0) return; - console.warn( - `[ObjectUI] ObjectGrid grouping: ${schema.objectName ?? 'object-grid'} groups by the masked ` - + `field(s) ${masked.join(', ')}. A masked field cannot be a grouping key: its group labels would ` - + 'show the raw value, and its groups would show which records share it. The entry was ignored.', - ); - }, [maskedGroupingSignature, schema.objectName]); + const groupingForRender = groupingRefusesNothing || !schema.grouping + ? schema.grouping + : { ...schema.grouping, fields: unmaskedGroupingFields }; + // [objectui#7189] The entries the SERVER groups by: the render grouping less + // any key this principal may not read (see `groupingKeyReadable`). Keyed on + // the names, so a host rebuilding `grouping` every render re-asks nothing. + const serverGroupingKey = serverGroupedFetch + ? JSON.stringify( + usableGroupingFields(groupingForRender?.fields) + .filter((gf) => groupingKeyReadable(gf.field)) + .map((gf) => [gf.field, gf.order ?? null, gf.collapsed ?? null]), + ) + : '[]'; + const serverGroupingFields = React.useMemo( + () => (JSON.parse(serverGroupingKey) as Array<[string, 'asc' | 'desc' | null, boolean | null]>) + .map(([field, order, collapsed]) => ({ + field, + ...(order ? { order } : {}), + ...(collapsed !== null ? { collapsed } : {}), + })), + [serverGroupingKey], + ); + // The view's composed filter as ONE `FilterCondition` — what both compiled + // queries take, so the header numbers and the rows they head answer the + // same question. Lowered from the AST the flat fetch would have sent; a + // filter that does not lower is refused, never dropped (dropping it would + // group the unfiltered object). + const groupRowQueryKey = JSON.stringify(groupRowQuery); + const groupWhere = React.useMemo((): { where?: FilterCondition; error?: Error } => { + const query = JSON.parse(groupRowQueryKey) as Record | null; + const ast = query?.$filter; + if (ast === undefined || ast === null) return {}; + try { + const where = parseFilterAST(ast); + if (where === undefined) { + return { error: new Error(`[ObjectGrid] The view filter could not be lowered for server-side grouping: ${JSON.stringify(ast)}`) }; + } + return { where }; + } catch (err) { + return { error: err instanceof Error ? err : new Error(String(err)) }; + } + }, [groupRowQueryKey]); + const groupReloadKey = `${refreshKey}:${invalidationNonce}`; + const groupHeaders = useServerGroupHeaders({ + enabled: serverGroupedFetch && !objectTypesPending && groupRowQuery !== null && !groupWhere.error, + dataSource, + objectName, + fields: serverGroupingFields, + where: groupWhere.where, + aggregations: schema.aggregations, + objectFields, + reloadKey: groupReloadKey, + }); + // Keyed on the hook's STATE values (a stable identity React promises), so + // the group tree is rebuilt when the server answers, not on every render. + const groupHeaderRows = groupHeaders.headers; + const groupKeyLabels = groupHeaders.keyLabels; + const serverGroupSource: ServerGroupSource | undefined = React.useMemo( + () => (serverGroupedFetch ? { headers: groupHeaderRows ?? [], keyLabels: groupKeyLabels } : undefined), + [serverGroupedFetch, groupHeaderRows, groupKeyLabels], + ); const { groups, isGrouped, toggleGroup } = useGroupedData( - groupingRefusesNothing || !schema.grouping - ? schema.grouping - : { ...schema.grouping, fields: unmaskedGroupingFields }, + serverGroupedFetch + ? (schema.grouping ? { ...schema.grouping, fields: serverGroupingFields } : undefined) + : groupingForRender, data, schema.aggregations, groupValueFormatter, + serverGroupSource, ); // Reset grouped pagination to page 1 whenever the grouping config, page size @@ -2789,6 +2910,47 @@ export const ObjectGrid: React.FC = ({ setGroupedPage(1); }, [groupingKey, groupedPageSize, refreshKey]); + // [objectui#7189] The open LEAF groups on the current page of groups — the + // only ones whose rows are on screen, and so the only ones whose rows are + // asked for. A collapsed group costs no row query at all. + const serverGroupLeaves: ServerGroupLeaf[] = []; + if (serverGroupedFetch) { + const groupPages = Math.max(1, Math.ceil(groups.length / groupedPageSize)); + const onPage = Math.min(groupedPage, groupPages); + const collectOpenLeaves = (group: typeof groups[number]) => { + if (group.collapsed) return; + if (group.subgroups.length === 0) serverGroupLeaves.push({ key: group.key, keyValues: group.keyValues }); + else group.subgroups.forEach(collectOpenLeaves); + }; + groups.slice((onPage - 1) * groupedPageSize, onPage * groupedPageSize).forEach(collectOpenLeaves); + } + const groupRows = useServerGroupRows({ + enabled: serverGroupedFetch && !!groupHeaders.headers, + dataSource, + objectName, + fields: serverGroupingFields, + where: groupWhere.where, + baseParams: groupRowQuery, + pageSize: serverPageSize, + leaves: serverGroupLeaves, + reloadKey: groupReloadKey, + }); + // The rows in hand are the open groups' pages: what everything else that + // reads `data` (column inference, the client export, selection) sees. + const serverLeafKeys = JSON.stringify(serverGroupLeaves.map((leaf) => leaf.key)); + useEffect(() => { + if (!serverGroupedFetch) return; + const keys = JSON.parse(serverLeafKeys) as string[]; + setData(keys.flatMap((key) => groupRows.pages[key]?.rows ?? [])); + }, [serverGroupedFetch, serverLeafKeys, groupRows.pages]); + // …and the match total is the QUERY's: the top-level group counts summed. + const serverGroupTotal = serverGroupedFetch && groupHeaders.headers + ? (groupHeaders.headers[0] ?? []).reduce((sum, row) => sum + (Number(row.count) || 0), 0) + : undefined; + useEffect(() => { + if (serverGroupTotal !== undefined) setTotalMatching(serverGroupTotal); + }, [serverGroupTotal]); + // --- Column summary support --- const summaryColumns = React.useMemo(() => { const cols = normalizeColumns(schema.columns); @@ -3857,16 +4019,23 @@ export const ObjectGrid: React.FC = ({ ); } - if (error) { + // [objectui#7189] A server-grouped grid fails like any other load: the + // header query refused, or a view filter that would not lower for it. + const loadError = error + ?? (serverGroupedFetch ? (groupWhere.error ?? groupHeaders.error ?? null) : null); + if (loadError) { return (

{t('grid.errorLoading')}

-

{error.message}

+

{loadError.message}

); } - if (loading && data.length === 0) { + // …and it is loading until its group set has first arrived: the flat fetch + // it skips never clears `data`'s emptiness, so `loading` alone says nothing. + const serverGroupsFirstLoad = serverGroupedFetch && !groupHeaders.headers; + if ((loading && data.length === 0) || serverGroupsFirstLoad) { if (useCardView) { return (
@@ -4178,19 +4347,26 @@ export const ObjectGrid: React.FC = ({ cellClassName: [rowHeightCellClass, col.cellClassName].filter(Boolean).join(' '), }); - // Server-side pagination applies to the flat, server-fetched list only. - // Inline/static data and the grouped view paginate in-memory (grouped mode - // keeps whole groups together via its own groupedPage state), so they stay - // on DataTable's default client-side slicing. + // Server-side pagination applies to every list whose rows the server pages: + // the flat, server-fetched list, and a server-grouped one (per group). + // Inline/static data paginates in-memory, and so does a grid grouping a page + // it fetched (grouped mode keeps whole groups together via its own + // groupedPage state), so they stay on DataTable's client-side slicing. // // Declared here rather than beside the rest of the manual-mode wiring below // because `orderedColumns` needs it: which columns may be sorted at all // depends on whether the sort is the server's. - const useServerPagination = !hasInlineData && !isGrouped; + // + // [objectui#7189] The grouped exclusion is gone for a grid that groups on + // the SERVER: it pages each group's rows there (see `renderGroup`), so its + // sort is the server's too. A grid grouping a page it fetched from a source + // that cannot answer the header query still holds whole groups in memory. + const useServerPagination = !hasInlineData && (!isGrouped || serverGroupedFetch); // Either we own the server fetch (useServerPagination) or a parent does - // (externalManualPagination). Grouped mode always keeps in-memory slicing so - // whole groups stay together. Both server modes feed DataTable a manual pager + // (externalManualPagination). This is the FLAT table's pager: a grouped grid + // renders no flat table — its pages are per group (`renderGroup`) — so it + // never switches this on. Both server modes feed DataTable a manual pager // backed by the real match total. const manualPaginationOn = (useServerPagination || externalManualPagination) && !isGrouped; @@ -4198,9 +4374,10 @@ export const ObjectGrid: React.FC = ({ // // Tied to who owns the ROWS, not to who owns the pager: whenever `data` is // one window of a larger collection, sorting it in the browser orders that - // window and nothing else. Grouped mode holds every row it groups, so it - // keeps DataTable's own client-side sort. - const manualSortingOn = manualPaginationOn; + // window and nothing else. A server-grouped grid holds one PAGE of each + // group, so its sort is the server's (objectui#7189); a grid grouping rows + // it holds keeps DataTable's own client-side sort. + const manualSortingOn = manualPaginationOn || (isGrouped && serverGroupedFetch); // Server-side search, on exactly the same condition (objectui#3118). Same // question, filter axis: when `data` is one window, a `.filter()` over it @@ -4803,6 +4980,13 @@ export const ObjectGrid: React.FC = ({ // host-driven paging mode still reaches condition 1, through `hostRowCount`. // A result set that fits leaves the grid silent, and that silence is what // makes the marker mean something when it does appear. + // + // ⭐ RETIRED where the counts are server-true (objectui#7189, ruling A — + // enforce-or-remove). A server-grouped grid's group set and every header + // number are the header query's own, so there is nothing partial to + // disclose and the marker would be a lie. What still reaches it is the one + // shape whose counts ARE page slices: a grid grouping the window it fetched + // from a data source that declares no `queryGroupHeaders`. const groupingRowsLoaded = data.length; const groupingTotalKnown = typeof resolvedTotalMatching === 'number'; const groupingPartialWithTotal = @@ -4810,7 +4994,7 @@ export const ObjectGrid: React.FC = ({ const groupingPartialWindowFull = !groupingTotalKnown && !hasInlineData && groupingRowsLoaded >= serverPageSize; const groupingIsPartial = - isGrouped && (groupingPartialWithTotal || groupingPartialWindowFull); + isGrouped && !serverGroupedFetch && (groupingPartialWithTotal || groupingPartialWindowFull); // ONE sentence, used in both places it belongs: the notice above the group // list, and the accessible name of the marker beside every group count. const groupingPartialNotice = groupingIsPartial @@ -5138,11 +5322,29 @@ export const ObjectGrid: React.FC = ({ } /** Build a per-group data-table schema (inherits everything except data & pagination). */ - const buildGroupTableSchema = (groupRows: any[]): ObjectGridDataTableSchema => ({ + const buildGroupTableSchema = ( + groupRows: any[], + serverPage?: { count: number; page: number; onPageChange: (page: number) => void }, + ): ObjectGridDataTableSchema => ({ ...dataTableSchema, caption: undefined, data: groupRows, pagination: false, + // [objectui#7189] A server-grouped leaf holds ONE page of its group: the + // pager is the group's own, driven by the header's count, and turning it + // asks the server for the group's next page — so every row of every group + // is reachable. Shown only when the group spans more than one page. + ...(serverPage + ? { + pagination: serverPage.count > serverPageSize, + manualPagination: true, + rowCount: serverPage.count, + page: serverPage.page, + pageSize: serverPageSize, + onPageChange: serverPage.onPageChange, + onPageSizeChange: (size: number) => { setServerPageSize(size); }, + } + : {}), searchable: false, // Embedded inside a GroupRow which already provides visual framing. // Drop the table's outer rounded border so groups look like Airtable's @@ -5801,6 +6003,34 @@ export const ObjectGrid: React.FC = ({ return { fieldLabel, labelColorClass, labelColorStyle }; }; + // [objectui#7189] One open leaf group's page of rows, as the server paged it. + const renderServerGroupPage = (group: typeof groups[number]): React.ReactNode => { + const held = groupRows.pages[group.key]; + if (held?.error) { + return ( +

+ {held.error.message} +

+ ); + } + if (!held || (held.loading && held.rows.length === 0)) { + return ( +

+ {t('grid.loading')} +

+ ); + } + return ( + groupRows.setPage(group.key, page), + })} + /> + ); + }; + const renderGroup = (group: typeof groups[number]): React.ReactNode => { const { fieldLabel, labelColorClass, labelColorStyle } = resolveGroupHeader(group.field, group.label); return ( @@ -5808,7 +6038,7 @@ export const ObjectGrid: React.FC = ({ = ({
{group.subgroups.map(renderGroup)}
+ ) : serverGroupedFetch ? ( + renderServerGroupPage(group) ) : ( )} diff --git a/packages/plugin-grid/src/__tests__/groupedPartialDisclosure-7189.test.tsx b/packages/plugin-grid/src/__tests__/groupedPartialDisclosure-7189.test.tsx index c411cf4cca..dba87632ab 100644 --- a/packages/plugin-grid/src/__tests__/groupedPartialDisclosure-7189.test.tsx +++ b/packages/plugin-grid/src/__tests__/groupedPartialDisclosure-7189.test.tsx @@ -10,6 +10,18 @@ * objectui#7189 — a grouped grid says, WHERE THE GROUP COUNTS ARE, that it * grouped a page. * + * ## ⭐ SCOPE SINCE RULING A LANDED — this file now pins the FALLBACK + * + * Maintainer ruling A made grouping server-side: a grid whose data source + * answers the group header query (`DataSource.queryGroupHeaders`) groups on the + * server, its counts are the query's own, and the marker below is RETIRED there + * — `serverGrouping-7189.test.tsx` pins both, including the marker's absence. + * What still reaches this disclosure is the one shape whose counts ARE page + * slices: a grid grouping the window it fetched from a data source that + * declares no `queryGroupHeaders`. Every data source in this file is that + * shape (`find` only), which is why every pin below still holds unchanged; the + * history that follows is the page-scoped behaviour those sources still get. + * * ## The defect this discloses (measured on `main` at a6d8b8d44) * * `useGroupedData` buckets the rows the browser already holds and computes diff --git a/packages/plugin-grid/src/__tests__/serverGrouping-7189.test.tsx b/packages/plugin-grid/src/__tests__/serverGrouping-7189.test.tsx new file mode 100644 index 0000000000..530189c443 --- /dev/null +++ b/packages/plugin-grid/src/__tests__/serverGrouping-7189.test.tsx @@ -0,0 +1,348 @@ +/** + * ObjectUI + * Copyright (c) 2024-present ObjectStack Inc. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */ + +/** + * objectui#7189 — a grouped grid groups on the SERVER. + * + * Maintainer ruling A (「7189 A 其他同意」): grouping on a list view is + * server-side. The set of groups and every number in a group header (the + * count, and any per-group aggregation) are properties of the query, not of + * the fetched page; rows inside a group are paged. The acceptance carried from + * objectstack#14556: the 186-row, five-unit, `$top: 100` fixture renders five + * group headers reading 86/61/31/7/1 regardless of row order, opening the + * 86-row group pages its rows, no "Partial" marker renders when counts are + * true, and every row is reachable through the UI. + * + * ## What the double answers, and why that is enough here + * + * The data source below answers `queryGroupHeaders` by reducing EVERY stored + * row under the compiled query's own `groupBy` / `aggregations` / `where`, and + * `find` by applying the compiled row query's `$filter` / `$top` / `$skip` — + * the two faces the platform serves through `POST /data/:object/query`. That + * the real door answers the compiled queries this way (five groups 86/61/31/7/1 + * in both orders, the 86-row group paged 50 + 36) is the platform half's pin + * (objectstack#15330, `list-view-grouping-query-door.test.ts`) and was + * re-measured for this card against the installed `@objectstack/spec` 17.4.0; + * what THIS file pins is the grid consuming those answers — and not the page. + * + * Every positive pin has a control that must come out differently: the SAME + * fixture through a data source that cannot answer the header query still + * groups the fetched page (86 + 14 contiguous) and still says so. + */ +import { describe, it, expect, vi, afterEach, beforeAll } from 'vitest'; +import { render, cleanup, screen, within, fireEvent } from '@testing-library/react'; +import '@testing-library/jest-dom'; +import React from 'react'; +import { compileListViewGroupQuery } from '@objectstack/spec/ui'; + +import { ObjectGrid } from '../ObjectGrid'; +import { ActionProvider } from '@object-ui/react'; +import { registerAllFields } from '@object-ui/fields'; + +beforeAll(() => { + registerAllFields(); +}); + +const OBJECT = 'duly_task'; + +const OBJECT_FIELDS = { + id: { type: 'text', label: 'Id' }, + subject: { type: 'text', label: 'Subject' }, + business_unit: { type: 'text', label: 'Business Unit' }, + amount: { type: 'number', label: 'Amount' }, +}; + +/** The card's distribution: 186 rows over five units sized 86/61/31/7/1. */ +const UNITS: Array<[string, number]> = [ + ['Northgate Operations', 86], + ['Northgate Plant', 61], + ['Northgate Quality', 31], + ['Riverside Plant', 7], + ['Riverside Depot', 1], +]; +const EXPECTED: Record = Object.fromEntries(UNITS); + +interface Row { id: string; subject: string; business_unit: string; amount: number } + +const buildRows = (interleaved: boolean): Row[] => { + const buckets = UNITS.map(([unit, n]) => + Array.from({ length: n }, (_, i) => ({ id: `${unit}-${i}`, subject: `${unit} #${i}`, business_unit: unit, amount: i + 1 })), + ); + if (!interleaved) return buckets.flat(); + const out: Row[] = []; + for (let i = 0; ; i++) { + let took = false; + for (const b of buckets) if (i < b.length) { out.push(b[i]); took = true; } + if (!took) return out; + } +}; + +/** The `FilterCondition` subset the compiled queries use: `$and`, `$eq`, `$null`, equality. */ +function matches(row: Record, cond: unknown): boolean { + if (cond === undefined || cond === null) return true; + return Object.entries(cond as Record).every(([key, value]) => { + if (key === '$and') return (value as unknown[]).every((c) => matches(row, c)); + if (value && typeof value === 'object' && !Array.isArray(value)) { + const op = value as Record; + if ('$eq' in op) return row[key] === op.$eq; + if ('$null' in op) return op.$null ? row[key] == null : row[key] != null; + throw new Error(`double: unsupported operator ${JSON.stringify(op)}`); + } + return row[key] === value; + }); +} + +type Agg = { function: string; field?: string; alias: string }; + +/** A data source serving BOTH compiled queries over the whole row store. */ +const makeServerDataSource = (rows: Row[]) => { + const ds = { + queryGroupHeaders: vi.fn(async (_object: string, query: { where?: unknown; groupBy?: string[]; aggregations?: Agg[] }) => { + const groupBy = query.groupBy ?? []; + const buckets = new Map(); + for (const row of rows.filter((r) => matches(r as any, query.where))) { + const key = JSON.stringify(groupBy.map((f) => (row as any)[f] ?? null)); + if (!buckets.has(key)) buckets.set(key, []); + buckets.get(key)!.push(row); + } + return [...buckets.entries()].map(([key, members]) => { + const values = JSON.parse(key) as unknown[]; + const out: Record = Object.fromEntries(groupBy.map((f, i) => [f, values[i]])); + for (const agg of query.aggregations ?? []) { + const nums = members.map((m) => (agg.field ? (m as any)[agg.field] : null)).filter((v) => typeof v === 'number') as number[]; + out[agg.alias] = + agg.function === 'count' ? (agg.field ? members.filter((m) => (m as any)[agg.field!] != null).length : members.length) + : agg.function === 'sum' ? nums.reduce((a, b) => a + b, 0) + : agg.function === 'count_distinct' ? new Set(members.map((m) => (m as any)[agg.field!])).size + : null; + } + return out as { count: number }; + }); + }), + find: vi.fn(async (_object: string, params: Record) => { + if (Array.isArray(params.$filter)) throw new Error('double: the grouped grid sent an AST filter, i.e. a flat window'); + const matching = rows.filter((r) => matches(r as any, params.$filter)); + const skip = (params.$skip as number | undefined) ?? 0; + const top = (params.$top as number | undefined) ?? matching.length; + return { data: matching.slice(skip, skip + top), total: matching.length }; + }), + findOne: vi.fn(), + create: vi.fn(), + update: vi.fn(), + delete: vi.fn(), + getObjectSchema: vi.fn(async () => ({ name: OBJECT, fields: OBJECT_FIELDS })), + }; + return ds; +}; + +/** The CONTROL: the same store, a data source that declares no header query. */ +const makePageOnlyDataSource = (rows: Row[]) => { + const { queryGroupHeaders: _none, ...rest } = makeServerDataSource(rows); + rest.find = vi.fn(async (_object: string, params: Record) => { + const skip = (params.$skip as number | undefined) ?? 0; + const top = (params.$top as number | undefined) ?? rows.length; + return { data: rows.slice(skip, skip + top), total: rows.length }; + }); + return rest; +}; + +const groupRows = () => [...document.querySelectorAll('[data-testid^="group-row-"]')]; +const headerOf = (el: Element) => ({ + label: el.querySelector('.group-label')?.textContent ?? '', + count: el.querySelector('.group-count')?.textContent ?? '', +}); +const headerCounts = () => Object.fromEntries(groupRows().map((r) => { const h = headerOf(r); return [h.label, Number(h.count)]; })); +const groupRowEl = (label: string) => groupRows().find((r) => headerOf(r).label === label)!; +const subjectsIn = (el: Element, unit: string) => + within(el as HTMLElement).queryAllByText(new RegExp(`^${unit} #\\d+$`)).map((n) => n.textContent); +const partialMarkers = () => [...document.querySelectorAll('.group-count-partial')]; + +const renderGrid = (dataSource: unknown, schemaExtra: Record = {}) => + render( + + + , + ); + +afterEach(() => cleanup()); + +describe('ObjectGrid groups on the server (objectui#7189)', () => { + it.each([ + ['contiguous', false], + ['interleaved', true], + ])('%s rows: five headers reading 86/61/31/7/1, no Partial marker', async (_name, interleaved) => { + const ds = makeServerDataSource(buildRows(interleaved)); + renderGrid(ds); + + await vi.waitFor(() => expect(groupRows()).toHaveLength(5)); + expect(headerCounts()).toEqual(EXPECTED); + // Ordered by label (`GroupingField.order`, default asc) over the whole set. + expect(groupRows().map((r) => headerOf(r).label)).toEqual([ + 'Northgate Operations', 'Northgate Plant', 'Northgate Quality', 'Riverside Depot', 'Riverside Plant', + ]); + expect(partialMarkers()).toHaveLength(0); + expect(screen.queryByTestId('grouping-partial-notice')).not.toBeInTheDocument(); + }); + + it('asks the header query the SPEC compiles, and never fetches a flat window', async () => { + const ds = makeServerDataSource(buildRows(false)); + renderGrid(ds); + await vi.waitFor(() => expect(groupRows()).toHaveLength(5)); + + expect(ds.queryGroupHeaders).toHaveBeenCalledTimes(1); + const [object, query] = ds.queryGroupHeaders.mock.calls[0]; + expect(object).toBe(OBJECT); + expect(query).toEqual(compileListViewGroupQuery( + { grouping: { fields: [{ field: 'business_unit' }] }, columns: [] }, + { depth: 1 }, + )); + // Every row read is ONE group's page: its filter is the compiled group + // predicate, never the flat window the page-scoped grouping bucketed. + await vi.waitFor(() => expect(ds.find).toHaveBeenCalledTimes(5)); + for (const [, params] of ds.find.mock.calls) { + expect(params.$filter).toMatchObject({ $and: [{ business_unit: { $eq: expect.any(String) } }] }); + expect(params.$skip).toBe(0); + expect(params.$top).toBe(100); + } + }); + + it('opening the 86-row group pages its rows: 86 of them under a 100-row window', async () => { + const ds = makeServerDataSource(buildRows(true)); + renderGrid(ds); + await vi.waitFor(() => expect(subjectsIn(groupRowEl('Northgate Operations'), 'Northgate Operations')).toHaveLength(86)); + }); + + it('pages a group larger than the window: 50, then the next 36 — all 86 reachable', async () => { + const ds = makeServerDataSource(buildRows(false)); + renderGrid(ds, { pagination: { pageSize: 50 } }); + await vi.waitFor(() => expect(groupRows()).toHaveLength(5)); + const group = () => groupRowEl('Northgate Operations'); + await vi.waitFor(() => expect(subjectsIn(group(), 'Northgate Operations')).toHaveLength(50)); + const firstPage = subjectsIn(group(), 'Northgate Operations'); + + const pageInfo = within(group() as HTMLElement).getByText('Page 1 of 2'); + const next = pageInfo.parentElement!.querySelectorAll('button')[2]; + fireEvent.click(next); + + await vi.waitFor(() => expect(within(group() as HTMLElement).getByText('Page 2 of 2')).toBeInTheDocument()); + await vi.waitFor(() => expect(subjectsIn(group(), 'Northgate Operations')).toHaveLength(36)); + const secondPage = subjectsIn(group(), 'Northgate Operations'); + expect(new Set([...firstPage, ...secondPage]).size).toBe(86); + // The second page was asked of the server, for this group only. + const lastCall = ds.find.mock.calls[ds.find.mock.calls.length - 1][1]; + expect(lastCall).toMatchObject({ + $filter: { $and: [{ business_unit: { $eq: 'Northgate Operations' } }] }, + $skip: 50, + $top: 50, + }); + }); + + it('every one of the 186 rows is reachable through the group pagers', async () => { + const ds = makeServerDataSource(buildRows(true)); + renderGrid(ds, { pagination: { pageSize: 50 } }); + await vi.waitFor(() => expect(groupRows()).toHaveLength(5)); + const seen = new Set(); + for (const [unit, size] of UNITS) { + const group = () => groupRowEl(unit); + await vi.waitFor(() => expect(subjectsIn(group(), unit).length).toBe(Math.min(size, 50))); + subjectsIn(group(), unit).forEach((s) => seen.add(s!)); + if (size > 50) { + const next = within(group() as HTMLElement).getByText('Page 1 of 2').parentElement!.querySelectorAll('button')[2]; + fireEvent.click(next); + await vi.waitFor(() => expect(subjectsIn(group(), unit).length).toBe(size - 50)); + subjectsIn(group(), unit).forEach((s) => seen.add(s!)); + } + } + expect(seen.size).toBe(186); + }); + + it('a per-group aggregation is the QUERY\'s number, not the page\'s', async () => { + const rows = buildRows(false); + const ds = makeServerDataSource(rows); + renderGrid(ds, { aggregations: [{ field: 'amount', type: 'sum' }] }); + await vi.waitFor(() => expect(groupRows()).toHaveLength(5)); + const chip = groupRowEl('Northgate Operations').querySelector('.group-aggregations')?.textContent ?? ''; + // 1 + 2 + … + 86 over the whole unit. + expect(chip).toContain(`sum: ${(86 * 87) / 2}`); + // …and the query asked for it by the spec's own alias. + const [, query] = ds.queryGroupHeaders.mock.calls[0]; + expect(query.aggregations).toContainEqual({ function: 'sum', field: 'amount', alias: 'sum_amount' }); + }); + + it('a collapsed group costs no row query', async () => { + const ds = makeServerDataSource(buildRows(false)); + renderGrid(ds, { grouping: { fields: [{ field: 'business_unit', collapsed: true }] } }); + await vi.waitFor(() => expect(groupRows()).toHaveLength(5)); + expect(headerCounts()).toEqual(EXPECTED); + expect(ds.find).not.toHaveBeenCalled(); + }); + + it('every LEVEL is the query\'s: an outer header counts its whole subtree, from its own depth query', async () => { + const rows = buildRows(true).map((r) => ({ ...r, region: r.business_unit.startsWith('Northgate') ? 'North' : 'River' })); + const ds = makeServerDataSource(rows as Row[]); + renderGrid(ds, { grouping: { fields: [{ field: 'region' }, { field: 'business_unit' }] } }); + await vi.waitFor(() => expect(groupRows()).toHaveLength(2 + 5)); + const outer = groupRows().filter((r) => !(r.getAttribute('data-testid') ?? '').includes('__')); + expect(Object.fromEntries(outer.map((r) => [headerOf(r).label, Number(headerOf(r).count)]))).toEqual({ + North: 86 + 61 + 31, + River: 7 + 1, + }); + // One header query per depth — `count` would fold, `avg` would not. + expect(ds.queryGroupHeaders.mock.calls.map(([, q]) => q.groupBy)).toEqual([['region'], ['region', 'business_unit']]); + // A leaf's rows are asked for with BOTH levels of its key. + await vi.waitFor(() => expect(ds.find).toHaveBeenCalled()); + expect((ds.find.mock.calls[0][1].$filter as { $and: unknown[] }).$and).toHaveLength(2); + }); + + it('a reference-typed key is labelled from the referenced record, not printed as its id', async () => { + const rows = buildRows(false).map((r) => ({ ...r, business_unit: `bu_${UNITS.findIndex(([u]) => u === r.business_unit)}` })); + const ds = makeServerDataSource(rows as Row[]); + const units = UNITS.map(([name], i) => ({ id: `bu_${i}`, name })); + const rowsFind = ds.find; + // The referenced object answers the label read; every other read is a row page. + const withLabels = { + ...ds, + find: vi.fn(async (object: string, params: Record) => + object === 'business_unit' + ? { + data: units.filter((u) => ((params.$filter as { id: { $in: string[] } }).id.$in).includes(u.id)), + total: units.length, + } + : rowsFind(object, params)), + getObjectSchema: vi.fn(async () => ({ + name: OBJECT, + fields: { ...OBJECT_FIELDS, business_unit: { type: 'lookup', label: 'Business Unit', reference: 'business_unit' } }, + })), + }; + renderGrid(withLabels); + await vi.waitFor(() => expect(groupRows()).toHaveLength(5)); + await vi.waitFor(() => expect(headerCounts()).toEqual(EXPECTED)); + }); + + // ── CONTROL: the instrument can see the defect ────────────────────────── + it('CONTROL — a data source with no header query still groups the page, and says so', async () => { + const ds = makePageOnlyDataSource(buildRows(false)); + renderGrid(ds); + await vi.waitFor(() => expect(groupRows().length).toBeGreaterThan(0)); + // The page-scoped answer the ruling retired where the server can answer: + // two headers for five units, and the marker on every one of them. + expect(headerCounts()).toEqual({ 'Northgate Operations': 86, 'Northgate Plant': 14 }); + expect(partialMarkers()).toHaveLength(2); + expect(screen.getByTestId('grouping-partial-notice')).toBeInTheDocument(); + }); +}); diff --git a/packages/plugin-grid/src/useGroupedData.ts b/packages/plugin-grid/src/useGroupedData.ts index eeb8efc45f..9778ecbfee 100644 --- a/packages/plugin-grid/src/useGroupedData.ts +++ b/packages/plugin-grid/src/useGroupedData.ts @@ -8,6 +8,7 @@ import { useState, useMemo, useCallback } from 'react'; import type { GroupingConfig } from '@object-ui/types'; +import { deriveColumnSummary, type ListViewGroupHeaderRow } from '@objectstack/spec/ui'; /** Supported aggregation function types. */ export type AggregationType = 'sum' | 'count' | 'avg' | 'min' | 'max' | 'count_distinct'; @@ -26,8 +27,12 @@ export interface AggregationResult { field: string; /** The aggregation function used. */ type: AggregationType; - /** The computed value. */ - value: number; + /** + * The computed value. `null` only on a server-grouped grid, where it is what + * the aggregate itself answered (`avg` / `min` / `max` over no values) — + * shown as a dash rather than invented as `0`. + */ + value: number | null; } export interface GroupEntry { @@ -39,8 +44,25 @@ export interface GroupEntry { field: string; /** Nesting depth (0 = top-level) */ depth: number; - /** Rows belonging to this group (flattened across subgroups) */ + /** + * Rows belonging to this group (flattened across subgroups). Empty on a + * server-grouped grid: the rows there are PAGED per group by the grid, + * which holds each open group's current page — `count` is the group's size. + */ rows: any[]; + /** + * The group's size. On a server-grouped grid this is the header query's + * `count` — the group's TOTAL row count, a property of the query rather than + * of any page (objectui#7189). Grouping rows the grid holds whole, it is + * `rows.length`. + */ + count: number; + /** + * The raw key of every level from the outermost down to this group, by + * field name — what the group's own row query is compiled from. `null` is + * the empty group's key. + */ + keyValues: Record; /** Whether the group section is collapsed */ collapsed: boolean; /** Computed aggregations for this group (empty when no aggregations configured). */ @@ -266,24 +288,109 @@ export function usableGroupingFields(fields: unknown): UsableGroupingField[] { } /** - * Hook that groups a flat data array by the fields specified in GroupingConfig. + * The server's answer for a grouped grid that owns its fetch (objectui#7189): + * the group set and every header number, as the group header query answered + * them. The rows inside each group are paged by the grid itself, per group, + * so they are not part of this tree. + */ +export interface ServerGroupSource { + /** + * Index `d` holds the header rows of the depth-`d + 1` query — one row per + * group at that level, every grouped field under its own name holding the + * RAW stored value (`null` for the empty group), and `count`. + */ + headers: ReadonlyArray>; + /** + * Display labels for keys that are not their own label (a lookup key is the + * referenced record's id), per field, by the raw key. + */ + keyLabels?: Readonly>>>; +} + +/** + * The grid's own `aggregations` vocabulary → the spec's `ColumnSummary` + * members, so a per-group number rides the header query the spec compiles + * (`compileListViewGroupQuery`) and is read back through the spec's + * `deriveColumnSummary` — one mapping, both directions. `count` maps to the + * summary that RIDES the group count (`COUNT(*)`), which is what this key has + * always meant (a group's row count, whatever `field` names); + * `count_distinct` is the spec's `count_unique`. + */ +const SERVER_SUMMARY: Readonly> = { + sum: 'sum', + count: 'count', + avg: 'avg', + min: 'min', + max: 'max', + count_distinct: 'count_unique', +}; + +/** The spec summary member one grid aggregation type reads back through. */ +export function serverSummaryOf(type: AggregationType): (typeof SERVER_SUMMARY)[AggregationType] | undefined { + return Object.prototype.hasOwnProperty.call(SERVER_SUMMARY, type) ? SERVER_SUMMARY[type] : undefined; +} + +/** + * Read the grid's per-group aggregations off ONE header row. A configured + * aggregation the header query carried no column for is left out rather than + * reported as a number nobody computed. + */ +function readServerAggregations( + row: ListViewGroupHeaderRow, + configs: AggregationConfig[] | undefined, +): AggregationResult[] { + if (!configs || configs.length === 0) return []; + const out: AggregationResult[] = []; + for (const { field, type } of configs) { + const summary = serverSummaryOf(type); + if (!summary || typeof field !== 'string' || field === '') continue; + const value = deriveColumnSummary(row, summary, field); + if (value === undefined) continue; + out.push({ field, type, value: value === null ? null : Number(value) }); + } + return out; +} + +/** Two raw group keys name the same group (a stored value, or both empty). */ +function sameGroupKey(a: unknown, b: unknown): boolean { + const aEmpty = a === null || a === undefined; + const bEmpty = b === null || b === undefined; + if (aEmpty || bEmpty) return aEmpty && bEmpty; + return a === b || String(a) === String(b); +} + +/** + * Hook that groups a grid's rows by the fields specified in GroupingConfig. * * Supports multi-level grouping, per-field sort order, and per-field default * collapsed state. Collapse state is managed internally so the consumer only * needs to wire `toggleGroup` to the UI. * + * Two sources, one tree (objectui#7189, maintainer ruling A): + * + * - `server` given — the grid owns its fetch, and the group set, every + * count and every aggregation come from the server's header query. `data` + * is not read: grouping a fetched page is exactly the answer the ruling + * retired, since a group whose rows all fall past the page would not + * appear at all and every count would be a page slice. + * - `server` absent — the rows in `data` are grouped here. That is exact + * when the grid holds every row (inline rows, a host's whole result set); + * over a fetched page it is page-scoped, and the grid says so on screen. + * * @param config - GroupingConfig from the grid schema (optional) - * @param data - flat data rows + * @param data - flat data rows (grouped only when `server` is absent) * @param aggregations - optional aggregation definitions to compute per group * @param formatValue - optional per-field formatter that maps raw values to * display labels (e.g. resolves select option codes * to their human-readable labels) + * @param server - the server's header rows and per-group row pages */ export function useGroupedData( config: GroupingConfig | undefined, data: any[], aggregations?: AggregationConfig[], formatValue?: GroupValueFormatter, + server?: ServerGroupSource, ): UseGroupedDataResult { // [objectui#7217] The SAME normalized list `ObjectGrid`'s formatter memo // reads. Memoized on the raw array rather than on `config`: hosts rebuild @@ -299,6 +406,61 @@ export function useGroupedData( const groups: GroupEntry[] = useMemo(() => { if (!isGrouped) return []; + if (server) { + /** + * One level of the server tree: the depth-`depth` header rows whose + * outer keys are `prefix`, each becoming one group whose `count` and + * aggregations are the row's own — never recomputed from rows. + */ + const buildServerLevel = ( + depth: number, + parentKey: string, + prefix: Record, + ): GroupEntry[] => { + if (depth >= fields.length) return []; + const f = fields[depth]; + const outer = fields.slice(0, depth); + const rowsAtDepth = (server.headers[depth] ?? []).filter((row) => + outer.every((pf) => sameGroupKey(row[pf.field], prefix[pf.field])), + ); + const entries = rowsAtDepth.map((row) => { + const raw = row[f.field] ?? null; + const resolved = raw === null ? undefined : server.keyLabels?.[f.field]?.[String(raw)]; + return { + row, + raw, + segment: extractValueKey(raw), + label: resolved ?? buildSegmentLabel(raw, f.field, formatValue), + }; + }); + const order = f.order ?? 'asc'; + entries.sort((a, b) => compareGroups(a.label, b.label, order)); + + return entries.map(({ row, raw, segment, label }) => { + const compositeKey = parentKey ? `${parentKey}__${depth}:${segment}` : `${depth}:${segment}`; + const collapsedDefault = !!f.collapsed; + const collapsed = + compositeKey in toggledKeys ? toggledKeys[compositeKey] : collapsedDefault; + const keyValues = { ...prefix, [f.field]: raw }; + const isLeaf = depth + 1 >= fields.length; + const count = Number(row.count); + return { + key: compositeKey, + label, + field: f.field, + depth, + rows: [], + count: Number.isFinite(count) ? count : 0, + keyValues, + collapsed, + aggregations: readServerAggregations(row, aggregations), + subgroups: isLeaf ? [] : buildServerLevel(depth + 1, compositeKey, keyValues), + }; + }); + }; + return buildServerLevel(0, '', {}); + } + /** * Recursively build a tree of groups for the slice of rows at the current * nesting depth. Each level partitions rows by `fields[depth]` and then @@ -309,6 +471,7 @@ export function useGroupedData( rowsAtLevel: any[], depth: number, parentKey: string, + prefix: Record, ): GroupEntry[] => { if (depth >= fields.length) return []; const f = fields[depth]; @@ -344,8 +507,9 @@ export function useGroupedData( const agg = aggregations && aggregations.length > 0 ? computeAggregations(entry.rows, aggregations) : []; + const keyValues = { ...prefix, [f.field]: entry.rows[0]?.[f.field] ?? null }; const subgroups = depth + 1 < fields.length - ? buildLevel(entry.rows, depth + 1, compositeKey) + ? buildLevel(entry.rows, depth + 1, compositeKey, keyValues) : []; return { key: compositeKey, @@ -353,6 +517,8 @@ export function useGroupedData( field: f.field, depth, rows: entry.rows, + count: entry.rows.length, + keyValues, collapsed, aggregations: agg, subgroups, @@ -360,8 +526,8 @@ export function useGroupedData( }); }; - return buildLevel(data, 0, ''); - }, [data, fields, isGrouped, toggledKeys, aggregations, formatValue]); + return buildLevel(data, 0, '', {}); + }, [data, fields, isGrouped, toggledKeys, aggregations, formatValue, server]); const toggleGroup = useCallback((key: string) => { setToggledKeys((prev) => { diff --git a/packages/plugin-grid/src/useServerGrouping.ts b/packages/plugin-grid/src/useServerGrouping.ts new file mode 100644 index 0000000000..19e40e616f --- /dev/null +++ b/packages/plugin-grid/src/useServerGrouping.ts @@ -0,0 +1,350 @@ +/** + * ObjectUI + * Copyright (c) 2024-present ObjectStack Inc. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */ + +/** + * Server-side grouping for a grid that owns its fetch (objectui#7189, + * maintainer ruling A: 「7189 A 其他同意」). + * + * ## The contract this module consumes + * + * Grouping on a list view is server-side: the set of groups and every number + * in a group header are properties of the QUERY, not of a fetched page, and + * the rows inside a group are paged. `@objectstack/spec/ui` compiles a grouped + * view into the two queries that say so, and this module sends exactly what + * they compile — it never re-derives either: + * + * - {@link compileListViewGroupQuery} — the GROUP HEADER query, one + * `EngineAggregateOptions` per nesting depth, answered through + * `DataSource.queryGroupHeaders` (`@object-ui/types`). One query per depth + * rather than one leaf query folded upward, because `avg` and + * `count_distinct` do not fold across levels (the spec module's own + * "Multi-level grouping" note); `count`, the number every header shows, is + * exact either way. + * - {@link compileListViewGroupRowsQuery} — ONE group's row page: the view's + * composed filter AND-ed with that group's key predicate, `limit` / + * `offset` per group, answered through the ordinary `DataSource.find`. + * + * Both take the view's composed filter as the same `FilterCondition`, so the + * header numbers and the rows they head are one question asked twice rather + * than two questions that might disagree. + * + * ## What a header row carries, and what the grid still has to do + * + * Every grouped field under its own name holding the RAW stored value — a + * lookup key is the referenced record's id, never the record, and the empty + * group's key is `null`. Resolving a key to a label is the consumer's, exactly + * as for a cell, so a reference-typed grouping key is resolved here with one + * `find` on the referenced object per grouped field (the same resolution + * `ObjectChart` applies to a grouped series); a failed resolution keeps the id + * rather than failing the grid. Ordering the groups is the consumer's too: + * `useGroupedData` applies `GroupingField.order` over the header set. + */ + +import { useCallback, useEffect, useState } from 'react'; +import type { DataSource, QueryParams } from '@object-ui/types'; +import type { FilterCondition } from '@objectstack/spec/data'; +import { + compileListViewGroupQuery, + compileListViewGroupRowsQuery, + type ListViewGroupHeaderRow, + type ListViewGroupQuerySource, +} from '@objectstack/spec/ui'; +import { isExpandableFieldType } from '@object-ui/core'; +import { serverSummaryOf, type AggregationConfig, type UsableGroupingField } from './useGroupedData'; + +/** + * `aggregations` as the summary columns the header query compiles from. An + * entry naming no field or an unknown type contributes no node, so it renders + * no number rather than a fabricated one. + */ +function summaryColumnsOf(aggregations: AggregationConfig[] | undefined): NonNullable { + const out: Array<{ field: string; summary: NonNullable> }> = []; + for (const agg of aggregations ?? []) { + if (!agg || typeof agg.field !== 'string' || agg.field === '') continue; + const summary = serverSummaryOf(agg.type); + if (summary) out.push({ field: agg.field, summary }); + } + return out as NonNullable; +} + +/** The grouping block the two compilers read, keyed on field names alone. */ +function groupingOf(fields: readonly UsableGroupingField[]): ListViewGroupQuerySource['grouping'] { + return { fields: fields.map((f) => ({ field: f.field })) } as ListViewGroupQuerySource['grouping']; +} + +export interface ServerGroupHeadersInput { + /** Whether the grid groups server-side at all. */ + enabled: boolean; + dataSource?: DataSource; + objectName?: string; + /** The grouping entries the grid may group by, in nesting order. */ + fields: readonly UsableGroupingField[]; + /** The view's composed filter, lowered to a `FilterCondition`. */ + where?: FilterCondition; + /** `object-grid.aggregations` — the per-group numbers besides the count. */ + aggregations?: AggregationConfig[]; + /** The object's field catalogue, for reference-typed grouping keys. */ + objectFields?: Record; + /** Any change re-asks the server (refresh, data invalidation). */ + reloadKey: string; +} + +export interface ServerGroupHeaders { + /** + * Index `d` holds the answer of the depth-`d + 1` header query. `undefined` + * until every depth has answered. + */ + headers?: ListViewGroupHeaderRow[][]; + /** + * Display labels for reference-typed grouping keys, per field, by the raw id + * a header row carries — read off the referenced record. An id with no entry + * keeps the id as its label. + */ + keyLabels: Record>; + loading: boolean; + error: Error | null; +} + +const IDLE: ServerGroupHeaders = { headers: undefined, keyLabels: {}, loading: false, error: null }; + +/** + * Ask the server for the group set and every header number — one compiled + * header query per nesting depth. + */ +export function useServerGroupHeaders(input: ServerGroupHeadersInput): ServerGroupHeaders { + const { enabled, dataSource, objectName, fields, where, aggregations, objectFields, reloadKey } = input; + const [state, setState] = useState(IDLE); + + // Keyed on CONTENT, never on the identity of an object a host may rebuild + // every render (AGENTS.md #10). + const fieldsKey = JSON.stringify(fields.map((f) => f.field)); + const whereKey = JSON.stringify(where ?? null); + const aggregationsKey = JSON.stringify(summaryColumnsOf(aggregations)); + const referenceKey = JSON.stringify( + fields.map((f) => { + const def = objectFields?.[f.field]; + return isExpandableFieldType(def) && typeof def?.reference === 'string' && def.reference + ? [def.reference, typeof def.displayField === 'string' ? def.displayField : null] + : null; + }), + ); + const canAsk = enabled && !!objectName && typeof dataSource?.queryGroupHeaders === 'function' && fields.length > 0; + + useEffect(() => { + if (!canAsk || !dataSource || !objectName) { + // Leaving server grouping drops the last answer rather than keeping a + // group set that no longer describes the view. + setState((prev) => (prev === IDLE ? prev : IDLE)); + return; + } + let cancelled = false; + setState((prev) => ({ ...prev, loading: true, error: null })); + const fieldNames: string[] = JSON.parse(fieldsKey); + const references: Array<[string, string | null] | null> = JSON.parse(referenceKey); + const grouping = groupingOf(fieldNames.map((field) => ({ field }))); + const columns = JSON.parse(aggregationsKey) as NonNullable; + const composedWhere = JSON.parse(whereKey) as FilterCondition | null; + + (async () => { + try { + const headers = await Promise.all( + fieldNames.map((_, depthIndex) => + dataSource.queryGroupHeaders!( + objectName, + compileListViewGroupQuery( + { grouping, columns }, + { ...(composedWhere ? { where: composedWhere } : {}), depth: depthIndex + 1 }, + ), + ), + ), + ); + + // Reference-typed keys are ids; read the records their labels come + // from. Keyed off the deepest answer, which carries every key. + const keyLabels: ServerGroupHeaders['keyLabels'] = {}; + const deepest = headers[headers.length - 1] ?? []; + await Promise.all( + fieldNames.map(async (field, i) => { + const target = references[i]; + if (!target) return; + const [reference, displayField] = target; + const ids = [...new Set(deepest.map((row) => row[field]).filter((v) => v !== null && v !== undefined && v !== ''))]; + if (ids.length === 0) return; + try { + const result = await dataSource.find(reference, { + $filter: { id: { $in: ids } }, + $top: ids.length, + }); + const byId: Record = {}; + for (const rec of (result?.data ?? []) as Array>) { + const id = rec?.id; + if (id === undefined || id === null) continue; + const label = (displayField ? rec[displayField] : undefined) + ?? rec.name ?? rec.label ?? rec.title; + if (label !== undefined && label !== null && label !== '') byId[String(id)] = String(label); + } + keyLabels[field] = byId; + } catch (err) { + // The ids still group correctly; only their labels are missing. + console.warn(`[ObjectGrid] Failed to resolve group labels for '${field}' (${reference}):`, err); + } + }), + ); + + if (!cancelled) setState({ headers, keyLabels, loading: false, error: null }); + } catch (err) { + if (!cancelled) { + setState({ headers: undefined, keyLabels: {}, loading: false, error: err instanceof Error ? err : new Error(String(err)) }); + } + } + })(); + + return () => { cancelled = true; }; + }, [canAsk, dataSource, objectName, fieldsKey, whereKey, aggregationsKey, referenceKey, reloadKey]); + + return state; +} + +/** One leaf group whose rows the grid wants on screen. */ +export interface ServerGroupLeaf { + /** The composite key `useGroupedData` gave the group. */ + key: string; + /** The raw key of every level from the outermost down to this group. */ + keyValues: Readonly>; +} + +export interface ServerGroupRowsInput { + enabled: boolean; + dataSource?: DataSource; + objectName?: string; + fields: readonly UsableGroupingField[]; + where?: FilterCondition; + /** + * The grid's own row query — projection, expansion and order — WITHOUT a + * filter or a window: each group supplies its own. `null` until the grid has + * resolved it. + */ + baseParams: Record | null; + pageSize: number; + /** The leaf groups that are expanded and on screen. */ + leaves: readonly ServerGroupLeaf[]; + reloadKey: string; +} + +export interface ServerGroupRowsPage { + rows: Record[]; + page: number; + loading: boolean; + error: Error | null; +} + +export interface ServerGroupRows { + /** The page of rows held for each leaf group, by its composite key. */ + pages: Readonly>; + /** Turn one group's page. */ + setPage: (key: string, page: number) => void; +} + +interface HeldPage extends ServerGroupRowsPage { + /** The request this page answers — a different one means re-ask. */ + signature: string; +} + +/** + * Page the rows INSIDE each visible, expanded leaf group — one compiled row + * query per group, `limit` / `offset` per group. + */ +export function useServerGroupRows(input: ServerGroupRowsInput): ServerGroupRows { + const { enabled, dataSource, objectName, fields, where, baseParams, pageSize, leaves, reloadKey } = input; + const [held, setHeld] = useState>({}); + + const fieldsKey = JSON.stringify(fields.map((f) => f.field)); + const whereKey = JSON.stringify(where ?? null); + const paramsKey = JSON.stringify(baseParams ?? null); + const leavesKey = JSON.stringify(leaves.map((l) => [l.key, l.keyValues])); + const queryKey = JSON.stringify([fieldsKey, whereKey, paramsKey, pageSize, reloadKey]); + + // The page each group is on, recorded AGAINST the question it was turned + // under: a different question (filter, sort, page size, a refresh) makes + // every recorded page stale, so each group starts over at its first page — + // read that way at once, rather than reset one render late. + const [turned, setTurned] = useState<{ queryKey: string; pages: Record }>({ queryKey, pages: {} }); + const pageByKey = turned.queryKey === queryKey ? turned.pages : {}; + + const canAsk = enabled && !!objectName && !!dataSource && baseParams !== null && fields.length > 0; + const pagesKey = JSON.stringify(pageByKey); + + useEffect(() => { + if (!canAsk || !dataSource || !objectName || !baseParams) return; + const fieldNames: string[] = JSON.parse(fieldsKey); + const grouping = groupingOf(fieldNames.map((field) => ({ field }))); + const composedWhere = JSON.parse(whereKey) as FilterCondition | null; + const wanted: Array<[string, Record]> = JSON.parse(leavesKey); + + for (const [key, keyValues] of wanted) { + const page = (JSON.parse(pagesKey) as Record)[key] ?? 1; + const signature = JSON.stringify([queryKey, key, page]); + // Already asked (answered or in flight) for exactly the page on screen. + // A response to any OTHER request for this group is dropped on arrival + // by the same signature, so paging back and forth never shows a stale + // page under the pager's number. + if (held[key]?.signature === signature) continue; + setHeld((prev) => ({ + ...prev, + [key]: { rows: prev[key]?.rows ?? [], page, loading: true, error: null, signature }, + })); + + const compiled = compileListViewGroupRowsQuery({ grouping }, keyValues, { + ...(composedWhere ? { where: composedWhere } : {}), + limit: pageSize, + offset: (page - 1) * pageSize, + }); + // The grid's projection / expansion / order, the group's filter and + // window. A text search has no counterpart on the header query, so it + // is never sent here either — the rows must stay the rows the header + // counted. + const { $search: _search, $searchFields: _searchFields, $filter: _filter, ...rest } = baseParams as Record; + const params = { + ...rest, + $filter: compiled.where, + $top: compiled.limit, + $skip: compiled.offset, + }; + + dataSource + .find(objectName, params as QueryParams) + .then((result) => { + setHeld((prev) => (prev[key]?.signature === signature + ? { ...prev, [key]: { rows: result?.data ?? [], page, loading: false, error: null, signature } } + : prev)); + }) + .catch((err) => { + setHeld((prev) => (prev[key]?.signature === signature + ? { ...prev, [key]: { rows: [], page, loading: false, error: err instanceof Error ? err : new Error(String(err)), signature } } + : prev)); + }); + } + // `held` is read to skip a request already answered; naming it would + // re-run this effect on every answer, which asks nothing new. + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [canAsk, dataSource, objectName, fieldsKey, whereKey, leavesKey, pagesKey, queryKey, pageSize]); + + const setPage = useCallback((key: string, page: number) => { + setTurned((prev) => { + const pages = prev.queryKey === queryKey ? prev.pages : {}; + return pages[key] === page && prev.queryKey === queryKey + ? prev + : { queryKey, pages: { ...pages, [key]: page } }; + }); + }, [queryKey]); + + // The held STATE itself, not a projection memoised over it: a consumer + // keys an effect on it, and a state value's identity is a promise React + // keeps where a memo's is not (AGENTS.md #10). + return { pages: held, setPage }; +} diff --git a/packages/plugin-list/src/ListView.tsx b/packages/plugin-list/src/ListView.tsx index 2fc23db134..76833a87ff 100644 --- a/packages/plugin-list/src/ListView.tsx +++ b/packages/plugin-list/src/ListView.tsx @@ -2084,6 +2084,36 @@ export const ListView = React.forwardRef(({ !Array.isArray(schema.data) && (schema.data as any).provider === 'api'; + /** + * Does a GROUPED grid query for itself? (objectui#7189, maintainer ruling A) + * + * Grouping on a list view is server-side: the set of groups and every number + * in a group header are properties of the query, and the rows inside a + * group are paged. This component's fetch is one window (`$top: + * effectivePageSize`), and a grid handed that window as `data` can only + * group the window — measured on 186 rows in five units of 86/61/31/7/1 with + * `$top: 100`: two headers (86, 14) or five page slices (31/31/30/7/1), by + * row order alone, every row past the window unreachable, and nothing on + * screen saying so. So when the data source can answer the group header + * query (`queryGroupHeaders`), the grid is handed NO rows: it asks the server + * for its groups and pages each group's rows itself, with the SAME effective + * filter this component would have sent (see `selfQueryFilter` below). + * + * ⛔ Not while a toolbar search is active. `$search` has no counterpart on + * the header query the platform answers — its aggregate branch composes + * `where` / `groupBy` / `aggregations` / `having` and nothing else — so + * searched group counts would not be the searched rows' counts. The + * searched view keeps grouping this component's window, as before. + */ + const gridOwnsGroupedFetch = + currentView === 'grid' && + (groupingConfig?.fields?.length ?? 0) > 0 && + !Array.isArray(schema.data) && + (schema.data as any)?.provider !== 'value' && + !!schema.objectName && + typeof dataSource?.queryGroupHeaders === 'function' && + !searchTerm; + /** * Does the surface rendered below draw the rows THIS component fetched? * @@ -2122,8 +2152,13 @@ export const ListView = React.forwardRef(({ * is what the client-side CSV/JSON export writes out. Measured with the paged * response delayed: while it was in flight the skeleton was up and the chart * had 0 rows, though its own unbounded response had already arrived. + * + * objectui#7189 adds the second surface that draws rows it queried itself: a + * grouped grid grouping on the server (`gridOwnsGroupedFetch`). Its group + * headers carry the true counts and every group pages its own rows, so this + * bar's "N records · Showing first N" would describe a window it never drew. */ - const surfaceDrawsFetchedRows = currentView !== 'gantt'; + const surfaceDrawsFetchedRows = currentView !== 'gantt' && !gridOwnsGroupedFetch; // objectui#10572 — the data-invalidation bus (`notifyDataChanged` from // `@object-ui/react`), read the objectui#10494 way: the nonce moves when a @@ -3093,7 +3128,7 @@ export const ListView = React.forwardRef(({ */ const selfQueryFilterRef = React.useRef<{ key: string; value: unknown } | null>(null); let selfQueryFilter: unknown = authoredFilter; - if (currentView === 'gantt' || currentView === 'tree' || currentView === 'chart') { + if (currentView === 'gantt' || currentView === 'tree' || currentView === 'chart' || gridOwnsGroupedFetch) { try { const value = buildEffectiveFilter(authoredFilter, appliedFilters, appliedUserFilterConditions); const key = JSON.stringify(value ?? null); @@ -3193,6 +3228,9 @@ export const ListView = React.forwardRef(({ ...(schema.selection ? { selection: schema.selection } : {}), ...(schema.pagination ? { pagination: schema.pagination } : {}), ...(groupingConfig ? { grouping: groupingConfig } : {}), + // objectui#7189 — a grid grouping on the server runs its own query, + // so it carries the effective filter. See `gridOwnsGroupedFetch`. + ...(gridOwnsGroupedFetch ? { filter: selfQueryFilter } : {}), ...(rowColorConfig ? { rowColor: rowColorConfig } : {}), ...(schema.rowActions ? { rowActions: schema.rowActions } : {}), /** @@ -3669,7 +3707,7 @@ export const ListView = React.forwardRef(({ // asynchronously (`/me/permissions`) and `objectDef` loads into state, so a // grid schema built before either resolved must be rebuilt when they do — // otherwise `editable` keeps the pre-verdict answer for the session. - }, [currentView, schema, authoredFilter, currentSort, effectiveFields, hasAuthoredColumns, groupingConfig, rowColorConfig, navigation.handleClick, density.mode, galleryCardSize, inlineEdit, inlineEditOffered, objectDef, selfQueryFilter, ganttSearchTerm]); + }, [currentView, schema, authoredFilter, currentSort, effectiveFields, hasAuthoredColumns, groupingConfig, rowColorConfig, navigation.handleClick, density.mode, galleryCardSize, inlineEdit, inlineEditOffered, objectDef, selfQueryFilter, ganttSearchTerm, gridOwnsGroupedFetch]); const hasFilters = currentFilters.conditions && currentFilters.conditions.length > 0; @@ -5034,10 +5072,12 @@ export const ListView = React.forwardRef(({ ({ + find: vi.fn(async (_object: string, params: any) => { + const top = params?.$top ?? 100; + const data = Array.from({ length: Math.min(top, 186) }, (_, i) => ({ id: `t-${i}`, subject: `Task ${i}`, business_unit: `bu_${i % 5}` })); + return { data, total: 186 }; + }), + ...(withHeaderQuery ? { queryGroupHeaders: vi.fn(async () => []) } : {}), + findOne: vi.fn(), + create: vi.fn(), + update: vi.fn(), + delete: vi.fn(), + getObjectSchema: async (name: string) => ({ + name, + fields: { + id: { type: 'text' }, + subject: { type: 'text' }, + status: { type: 'text' }, + business_unit: { type: 'text' }, + }, + }), +}) as any; + +const listSchema = (over: Record = {}): ListViewSchema => ({ + type: 'list-view', + objectName: OBJECT, + columns: ['subject'], + pagination: { pageSize: 100 }, + grouping: { fields: [{ field: 'business_unit', order: 'asc', collapsed: false }] }, + filter: [{ field: 'status', operator: 'equals', value: 'open' }], + ...over, +} as unknown as ListViewSchema); + +let prevObjectGrid: any; +beforeAll(() => { + prevObjectGrid = ComponentRegistry.get('object-grid'); + ComponentRegistry.register('object-grid', (props: any) => { + lastGridProps = props; + return
; + }); +}); +afterAll(() => { + if (prevObjectGrid) ComponentRegistry.register('object-grid', prevObjectGrid); + else ComponentRegistry.unregister('object-grid'); +}); +beforeEach(() => { lastGridProps = null; }); +afterEach(() => { cleanup(); lastGridProps = null; }); + +const renderList = (ds: any, schema: ListViewSchema) => + render( + + + , + ); + +describe('ListView → grouped grid: the grid owns its fetch when the server can group (objectui#7189)', () => { + it('hands the grouped grid NO rows, and the effective filter to query with', async () => { + const ds = makeDataSource(true); + renderList(ds, listSchema()); + await waitFor(() => expect(lastGridProps).toBeTruthy()); + await waitFor(() => expect(ds.find).toHaveBeenCalled()); + await waitFor(() => expect(screen.getByTestId('grid-stub')).toBeInTheDocument()); + + expect(lastGridProps.data, 'a window of rows would turn the grid back into page-scoped grouping').toBeUndefined(); + // The filter the grid queries with is the one THIS component queried + // with — lowered, so the grid's own fetch sends the same predicate. + const listFilter = ds.find.mock.calls[ds.find.mock.calls.length - 1][1].$filter; + expect(lastGridProps.schema.filter).toEqual(listFilter); + expect(lastGridProps.schema.grouping.fields.map((f: any) => f.field)).toEqual(['business_unit']); + }); + + it('draws no record-count bar over a grid that drew something else', async () => { + const ds = makeDataSource(true); + renderList(ds, listSchema()); + await waitFor(() => expect(ds.find).toHaveBeenCalled()); + await waitFor(() => expect(screen.getByTestId('grid-stub')).toBeInTheDocument()); + expect(screen.queryByTestId('record-count-bar')).toBeNull(); + }); + + // ── CONTROLS: nothing moves where the server cannot group ─────────────── + it('CONTROL — a data source with no header query still hands the grid its window', async () => { + const ds = makeDataSource(false); + renderList(ds, listSchema()); + await waitFor(() => expect(Array.isArray(lastGridProps?.data) && lastGridProps.data.length > 0).toBe(true)); + // …with the AUTHORED filter, exactly as before. + expect(lastGridProps.schema.filter).toEqual([{ field: 'status', operator: 'equals', value: 'open' }]); + await waitFor(() => expect(screen.getByTestId('record-count-bar')).toBeInTheDocument()); + }); + + it('CONTROL — an UNGROUPED grid is still handed its window, header query or not', async () => { + const ds = makeDataSource(true); + renderList(ds, listSchema({ grouping: undefined })); + await waitFor(() => expect(Array.isArray(lastGridProps?.data) && lastGridProps.data.length > 0).toBe(true)); + expect(ds.queryGroupHeaders).not.toHaveBeenCalled(); + }); +}); diff --git a/packages/types/src/data.ts b/packages/types/src/data.ts index b648dd356e..3a3c5c8729 100644 --- a/packages/types/src/data.ts +++ b/packages/types/src/data.ts @@ -25,8 +25,9 @@ import type { ImportRowResult, ImportWriteMode, } from '@objectstack/spec/api'; -import type { FilterArray } from '@objectstack/spec/data'; +import type { EngineAggregateOptions, FilterArray } from '@objectstack/spec/data'; import type { ValidationError } from '@objectstack/spec/kernel'; +import type { ListViewGroupHeaderRow } from '@objectstack/spec/ui'; export type { ImportJobStatus, ImportWriteMode, ValidationError }; @@ -776,6 +777,35 @@ export interface DataSource { */ aggregate?(resource: string, params: AggregateParams): Promise; + /** + * Answer a grouped list view's GROUP HEADER query (objectui#7189). + * + * `query` is what `compileListViewGroupQuery` (`@objectstack/spec/ui`) + * compiles a grouped view into — an `EngineAggregateOptions` whose `groupBy` + * is the grouping fields in nesting order and whose `aggregations` carry the + * group count plus any per-group summary. The answer is one header row per + * group, every grouped field under its own name holding the RAW stored value + * (`null` for the empty group) and the group's TOTAL row count under + * `count`. Those numbers are properties of the query, not of any fetched + * page: that is the whole contract, and it is what lets a grouped grid show + * every group with its true size and page the rows inside each one. + * + * Optional, and presence is the capability: a grid only asks a data source + * that declares this member. A source that cannot answer it must leave it + * undeclared rather than approximate it — a grouped grid over such a source + * keeps grouping the rows it fetched, and says on screen that it did. + * + * ⛔ Not a second spelling of {@link aggregate}: that member takes the + * analytics-shaped {@link AggregateParams} (one field, one function, one + * `groupBy`) and may degrade to a client-side reduction; this one takes the + * compiled header query verbatim and never degrades. + * + * @param resource - Object name + * @param query - The compiled header query (`EngineAggregateOptions`) + * @returns One header row per group, as the platform answers it + */ + queryGroupHeaders?(resource: string, query: EngineAggregateOptions): Promise; + /** * Subscribe to mutation events. * When implemented, data-bound views (ListView, ObjectView) can auto-refresh diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index d1e7ba5503..77d2d0fe29 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -1249,7 +1249,7 @@ importers: specifier: ^17.3.0 version: 17.4.0(ai@7.0.65(zod@4.4.3)) '@objectstack/spec': - specifier: ^17.2.0 + specifier: ^17.4.0 version: 17.4.0(ai@7.0.65(zod@4.4.3)) devDependencies: '@object-ui/test-support': @@ -2123,7 +2123,7 @@ importers: specifier: workspace:* version: link:../types '@objectstack/spec': - specifier: ^17.0.0 + specifier: ^17.4.0 version: 17.4.0(ai@7.0.65(zod@4.4.3)) '@tanstack/react-virtual': specifier: ^3.14.9