feat(plugin-grid): server-side grid grouping — group headers from the query, rows paged per group (objectui#7189)

.changeset/7189-grouped-grid-partial-disclosure.md

@@ -48,3 +48,22 @@ toward it or changes the fetch.
 ten locale packs; `GroupRow` gains two optional props (`partialLabel`,
 `partialTitle`) and is otherwise unchanged. No metadata schema key was added:
 the condition is derived from data the grid already has.
+
+⚠️ **Dated note, 2026-09-28 — grid grouping is now server-side where the data
+source can answer the group header query — objectui#7189, PR objectui#10878.**
+The paragraphs above describe this change as it was written. Later in this same
+release three of their sentences stopped describing the grid: that
+`useGroupedData` buckets the rows the browser already holds, so the set of
+groups and every header number are properties of the fetched page; that
+client-side grouping is **unchanged** here; and that server-side grouping is not
+part of this, is still open on objectui#5560, and that nothing here builds
+toward it or changes the fetch. Maintainer ruling A on objectui#7189 settled that
+question. On a data source that declares `queryGroupHeaders`, a grouped grid
+that fetches its own rows now takes its groups, every group count and every
+per-group aggregation from the query, and pages each group's rows on the
+server; its counts are the query's own, so the `Partial` marker and the notice
+described above never render there. On a data source without
+`queryGroupHeaders`, the grid still groups the page it fetched in the browser,
+as described above, and the marker and the notice are kept there. The rest of
+this entry is kept as the reading of this change; the
+`7189-server-side-grid-grouping` entry states what a grouped grid does now.

.changeset/7189-server-side-grid-grouping.md

@@ -0,0 +1,68 @@
+---
+'@object-ui/types': minor
+'@object-ui/data-objectstack': minor
+'@object-ui/plugin-grid': minor
+'@object-ui/plugin-list': minor
+---
+
+Grid grouping is now **server-side** (objectui#7189, maintainer ruling A): the
+set of groups and every number in a group header — the count and any per-group
+aggregation — come from the query, and the rows inside a group are paged by the
+server.
+
+Before, a grouped grid fetched one window and bucketed it in the browser, so
+both the group set and every header number were properties of the page. On a
+store of 186 records over five business units sized 86 / 61 / 31 / 7 / 1 with a
+100-row page it rendered **two** headers (86, 14) when the rows were stored
+contiguously and five page slices (31 / 31 / 30 / 7 / 1) when interleaved — and
+the records past the window were not on a second page, they were unreachable.
+The same store now renders five headers reading 86 / 61 / 31 / 7 / 1 in either
+order, and every record is reachable through its group's own pager.
+
+- **`@object-ui/types`** — `DataSource` gains an optional
+  `queryGroupHeaders(resource, query)`: it answers the group header query
+  `@objectstack/spec/ui`'s `compileListViewGroupQuery` compiles (an
+  `EngineAggregateOptions`) with one header row per group. Presence is the
+  capability; a source that cannot answer it leaves it undeclared.
+- **`@object-ui/data-objectstack`** — `ObjectStackAdapter.queryGroupHeaders`
+  posts the compiled query verbatim to the existing `POST /data/:object/query`
+  door and answers its `records`. It never degrades: a refusal throws, and a
+  body without `records` is refused rather than read as "no groups".
+- **`@object-ui/plugin-grid`** — a grouped grid that fetches its own rows from a
+  source declaring `queryGroupHeaders` asks for its headers (one compiled query
+  per grouping level) and pages each **open** group's rows with the group's
+  own compiled row query (`compileListViewGroupRowsQuery`: the view's filter AND
+  the group key, `limit` / `offset` per group); a collapsed group costs no row
+  query, `aggregations` are the header query's numbers, and a reference-typed
+  grouping key is labelled from the referenced record. Such a grid's column
+  sort is the server's, and it no longer shows the `Partial` marker: its counts
+  are the query's own. Rows handed in whole are still grouped in the browser
+  (exact), and a source with no header query still groups the page it fetched
+  and still marks it partial.
+- **`@object-ui/plugin-list`** — `ListView` hands a grouped grid its own fetch,
+  with the view's effective filter, when the data source can answer the header
+  query, and draws no record-count bar over it. While a toolbar search is
+  active it keeps hosting the rows as before: `$search` has no counterpart on
+  the header query.
+
+**Breaking semantics (declared `minor` per this repo's version policy):**
+
+- A grouped grid over a capable source sends different requests: one header
+  query per grouping level plus one row query per open group, instead of one
+  window. A host that counted or mocked the single window sees the new shape.
+- `useGroupedData` takes an optional fifth argument (the server's header rows).
+- `GroupEntry` gains two REQUIRED members: `count` (the group's size — use it
+  instead of `rows.length`, which is empty on a server-grouped grid) and
+  `keyValues`. Code that only READS a `GroupEntry` is unaffected; code that
+  CONSTRUCTS a `GroupEntry` literal no longer type-checks until it supplies
+  both.
+- `AggregationResult.value` is `number | null`, `null` being what the server's
+  aggregate answered over no values (rendered as a dash, never invented as 0).
+- `@object-ui/plugin-grid` and `@object-ui/data-objectstack` now require
+  `@objectstack/spec` `^17.4.0`, the first release that ships the group-query
+  compilers.
+
+This supersedes the last paragraph of the pending
+`7189-grouped-grid-partial-disclosure` changeset, which said server-side
+grouping was not built: it now is, and the `Partial` marker described there
+remains only for the data sources that cannot answer the header query.

content/docs/api/schema-reference.md

@@ -781,7 +781,7 @@ A data grid that auto-fetches from an ObjectQL object definition. Includes searc
 | `operations` | `object` | Enabled CRUD operations. |
 | `rowActions` / `bulkActions` | `string[]` | Action identifiers for rows and batch selection. `bulkActions` is the spec-aligned key; `batchActions` is a legacy alias that takes precedence when both are set. |
 | `editable` | `boolean` | Enable inline cell editing. |
-| `grouping` | `GroupingConfig` | Row grouping configuration. **Page-scoped**: the grid groups the rows it has fetched, so group counts are page slices and a group beyond the page is absent — the grid marks the grouping partial when it can tell. |
+| `grouping` | `GroupingConfig` | Row grouping configuration. **Server-side**: the set of groups, every group count and every per-group aggregation come from the group header query (`dataSource.queryGroupHeaders`), and each group's rows are paged by the server. Rows handed in whole are grouped in the browser (exact); over a data source with no header query the grid groups the page it fetched and marks the counts partial. |
 | `frozenColumns` | `number` | Number of columns frozen on scroll. |
 | `navigation` | `ViewNavigationConfig` | SPA navigation configuration. |
 

content/docs/plugins/plugin-grid.mdx

@@ -95,7 +95,7 @@ below for why this plugin deliberately does not claim it.
 | `navigation` | `NavigationConfig` | What a row click does: `{ mode: 'page' \| 'drawer' \| 'modal' \| 'split' \| 'none', … }`. |
 | `operations` | `object` | Toggles the built-in CRUD/export/import affordances, e.g. `{ delete: false }`. |
 | `rowHeight`, `frozenColumns`, `resizable`, `reorderableColumns`, `showColumnTypeIcons`, `rowColor`, `conditionalFormatting`, `aggregations`, `exportOptions`, `className` | | The rest of the declared surface. |
-| `grouping` | `GroupingConfig` | Row grouping — **page-scoped**, see [Grouping is page-scoped](#grouping-is-page-scoped). |
+| `grouping` | `GroupingConfig` | Row grouping — **server-side**: group set, counts and aggregations come from the query, rows are paged per group; see [Grouping is server-side](#grouping-is-server-side). |
 
 **There is no `sortable`, `filterable`, `onRowClick`, `onSelectionChange`,
 `onCellChange`, `onRowSave`, `onBatchSave` or `object` on this schema.** The two
@@ -335,37 +335,50 @@ see [Row callbacks are component props](#row-callbacks-are-component-props).
 always carries a rows-per-page picker, and `pageSizeOptions` only replaces the
 choices it offers with your own.
 
-### Grouping is page-scoped
-
-`grouping` groups **the records the browser has already fetched** — one page —
-not the whole result set. The grid buckets the rows in hand and computes every
-per-group count and aggregation from that same array, so both the set of groups
-and every number in a group header are properties of the page, not of the query.
-
-Two consequences, and the second is the one to decide against before you ship
-the view:
-
-- Every group count is a page slice, so it can be lower than the group's real
-  size.
-- **A group whose records all fall beyond the page does not appear at all.** On
-  a grid read as an org lens ("what is outstanding, per business unit"), a unit
-  that is silently absent is a wrong answer that looks like good news.
-
-Since objectui#7189 the grid says so on screen rather than leaving it silent.
-When the result set is larger than the rows loaded, a short `Partial` marker
-appears beside **every group count** — where the authoritative-looking number
-is — and a line above the group list carries the whole sentence, e.g.
-*"Grouped over the first 100 of 186 records…"*. When no match total is
-reachable the wording weakens honestly to *"Grouped over the 100 records
-loaded. More may match this view…"*. A grouped grid whose result set fits in
-one page shows neither: the marker is conditional, which is what makes it worth
-reading.
-
-The disclosure is not a fix for the scoping — it makes it visible. If a view
-needs true totals per group today, the working options are to narrow the query
-until the result set fits one page (a `filter` that scopes the lens), to raise
-`pagination.pageSize` past the result set, or to compute the aggregation server
-side and render it from a dataset rather than from grid grouping.
+### Grouping is server-side
+
+`grouping` is answered by the **server**, not by bucketing the records the
+browser happens to hold (objectui#7189, maintainer ruling A). The set of groups
+and every number in a group header — the count and any per-group aggregation —
+are properties of the query; the rows inside a group are paged.
+
+A grouped grid that fetches its own rows asks its data source for the group
+headers — the query `@objectstack/spec/ui`'s `compileListViewGroupQuery`
+compiles, sent through `dataSource.queryGroupHeaders` (the ObjectStack adapter
+posts it to `POST /data/:object/query`) — and then pages each **open** group's
+rows with that group's own query (`compileListViewGroupRowsQuery`: the view's
+filter AND the group's key, `limit` / `offset` per group). So:
+
+- Every group appears, with its **true size**, whatever the page size and
+  whatever order the rows are stored in — a store of 186 records over five
+  units renders five headers reading 86/61/31/7/1.
+- A group larger than the page gets its **own pager**, and turning it asks the
+  server for that group's next page, so every record is reachable.
+- A collapsed group costs no row query at all.
+- `aggregations` are computed by the same header query, over the group's whole
+  row set.
+- Multi-level grouping asks one header query per level, so an outer header's
+  numbers are that level's own (an average is never an average of averages).
+- A `lookup` / `master_detail` / `user` grouping key is the referenced record's
+  id on the wire; the grid labels it from the referenced record.
+
+When a `ListView` hosts the grid (the console's list views), it hands a grouped
+grid its own fetch and the view's effective filter, rather than a window of
+rows.
+
+Two cases still group **in the browser**:
+
+- **Rows handed in whole** (`data: { provider: 'value', items }`, or a host's
+  whole result set): nothing was withheld, so grouping them there is exact.
+- **A data source that declares no `queryGroupHeaders`**: the grid can only
+  group the page it fetched. Every count is then a page slice and a group whose
+  records all fall past the page is absent, so the grid says so where the
+  numbers are — a short `Partial` marker beside every group count and a line
+  above the group list (*"Grouped over the first 100 of 186 records…"*). With
+  counts from the server the marker never appears.
+
+A toolbar **search** has no counterpart on the group header query, so a
+`ListView` keeps grouping its own window while a search term is active.
 
 ### ObjectQL Integration
 

content/docs/utilities/data-objectstack.mdx

@@ -225,12 +225,16 @@ adapter):
 - `bulk?(resource, operation, data)` - Batch create/update/delete on one object
 - `batchTransaction?(operations)` - Cross-object atomic batch (master-detail)
 - `onMutation?(listener)` - Subscribe to create/update/delete events
-
-`bulk`, `batchTransaction` and `onMutation` are **optional** members of
-`DataSource`: not every adapter implements them, so through a `DataSource`-typed
-value they must be feature-detected (`typeof dataSource.bulk === 'function'`).
-`ObjectStackAdapter` implements all three unconditionally, and the factory
-declares the class, so a value from either form calls them directly. (`onMutation`
+- `queryGroupHeaders?(resource, query)` - Answer a grouped list view's group
+  header query (`compileListViewGroupQuery`), one row per group with its true
+  `count` — what a grouped grid groups by on the server (objectui#7189)
+
+`bulk`, `batchTransaction`, `onMutation` and `queryGroupHeaders` are
+**optional** members of `DataSource`: not every adapter implements them, so
+through a `DataSource`-typed value they must be feature-detected
+(`typeof dataSource.bulk === 'function'`). `ObjectStackAdapter` implements all
+four unconditionally, and the factory declares the class, so a value from
+either form calls them directly. (`onMutation`
 was listed under *Adapter-only* here until objectui#7323; it is optional on
 `DataSource`, not absent from it.)
 

packages/data-objectstack/README.md

@@ -361,6 +361,25 @@ possibly-absent authored values keep working. Keys outside those three are not
 refused either — the gate names `filter`, `field` and `function`, and only
 those.
 
+### Group headers for server-side grid grouping
+
+`queryGroupHeaders(resource, query)` answers a grouped list view's **group
+header query** — the `EngineAggregateOptions` that `@objectstack/spec/ui`'s
+`compileListViewGroupQuery` compiles from the view's `grouping` (and its column
+summaries). It is how a grouped `object-grid` gets its group set and every
+header number from the server instead of bucketing a fetched page
+(objectui#7189): one row per group, each grouped field under its own name
+holding the raw stored value (`null` for the empty group), and the group's
+total row count under `count`.
+
+The query is posted **verbatim** to the existing `POST /data/:object/query`
+door (the same one the spec-shape `aggregate()` branch uses), and the answer is
+its `records`. Unlike `aggregate()`, it never degrades to a client-side
+reduction: a refusal is thrown with the server's code and status, and an answer
+without a `records` array is refused rather than read as "no groups". The rows
+inside a group are an ordinary `find()` with the group's compiled row query
+(`compileListViewGroupRowsQuery`).
+
 ### Sorting
 
 ```typescript

packages/data-objectstack/package.json

@@ -32,7 +32,7 @@
     "@object-ui/core": "workspace:*",
     "@object-ui/types": "workspace:*",
     "@objectstack/client": "^17.3.0",
-    "@objectstack/spec": "^17.2.0"
+    "@objectstack/spec": "^17.4.0"
   },
   "devDependencies": {
     "@object-ui/test-support": "workspace:*",

packages/data-objectstack/src/index.ts

@@ -7,7 +7,8 @@
  */
 
 import { ObjectStackClient, type QueryOptions as ObjectStackQueryOptions } from '@objectstack/client';
-import type { DroppedFieldsEvent } from '@objectstack/spec/data';
+import type { DroppedFieldsEvent, EngineAggregateOptions } from '@objectstack/spec/data';
+import type { ListViewGroupHeaderRow } from '@objectstack/spec/ui';
 // #4934 — a VALUE import, not a type one: the write-warning boundary parses the
 // wire's `reason` against the enum the spec itself declares, so the accept set
 // is read off the pin instead of hand-copied here (a hand copy is the drift
@@ -6208,6 +6209,40 @@ export class ObjectStackAdapter<T = unknown> implements DataSource<T> {
     }
   }
 
+  /**
+   * Answer a grouped list view's GROUP HEADER query (objectui#7189) — see
+   * `DataSource.queryGroupHeaders` in `@object-ui/types` for the contract.
+   *
+   * The compiled query rides the platform's EXISTING data door verbatim:
+   * `client.data.query()` posts it to `POST /data/:object/query`, whose
+   * `findData` routes a body carrying `groupBy` / `aggregations` to
+   * `engine.aggregate` and answers the header rows as `records` (the door the
+   * spec half of objectstack#14556 names, pinned platform-side by
+   * objectstack#15330). No new route, no new wire shape.
+   *
+   * ⛔ No fallback, deliberately — unlike {@link aggregate}. A header query that
+   * degraded to bucketing a fetched page would answer page slices under a name
+   * that promises the query's own numbers, which is the defect this member
+   * exists to end. A failure is thrown to the caller, and a body without a
+   * `records` array is refused rather than read as "no groups".
+   */
+  async queryGroupHeaders(
+    resource: string,
+    query: EngineAggregateOptions,
+  ): Promise<ListViewGroupHeaderRow[]> {
+    await this.connect();
+    const result: unknown = await this.client.data.query(resource, query as any);
+    const records = (result as { records?: unknown } | null)?.records;
+    if (!Array.isArray(records)) {
+      throw new Error(
+        `queryGroupHeaders('${resource}'): POST /data/${resource}/query answered without a `
+        + '`records` array, so no group header can be read from it. The header query was '
+        + 'sent verbatim; nothing was approximated from a page of rows.',
+      );
+    }
+    return records as ListViewGroupHeaderRow[];
+  }
+
   /**
    * Client-side aggregation over a server-scoped `find()` — the fallback used
    * whenever the analytics endpoint cannot answer (capability absent, network