Grid grouping collapses every row into one "(empty)" bucket when the grouping field is not also a displayed column — `select` is built from `columns` alone

[os-warren]

Found by opening a grouped grid in a real ObjectStack application (objectstack-ai/duly) against published @objectstack/* 17.2.0, with 186 real records across 5 business units. Network payloads captured; root cause confirmed in this repo's source.

The claim

A grid view that declares grouping on a field absent from its columns renders one group labelled (empty) containing every row. No error, no warning, no empty state — a grouped grid that looks like it grouped, and did not.

The grouping field is fully populated on every record. The same field, grouped by a dashboard chart on the same data, resolves its labels correctly.

Measured

View declaration:

by_unit: {
  type: 'grid',
  columns,                                    // subject, status, due_date, period_key, owner, source
  grouping: { fields: [{ field: 'business_unit' }] },
}

Rendered: a single collapsible group, (empty) 100, holding every row.

Two payloads captured on that page load:

GET /api/v1/data/duly_task?
  keys: id,…,subject,duty,owner,business_unit,assignment,source,period_key,…
  business_unit: "Q2UJSlOy4ICnWmTg"          ← populated

GET /api/v1/data/duly_task?populate=owner&top=100&select=id,subject,status,due_date,period_key,…
  keys: id,subject,status,due_date,period_key,owner,source
  business_unit: undefined                    ← not requested

The grid's own request builds select from the view's columns and nothing else. business_unit is never asked for, so it is undefined on every row by the time grouping runs.

Storage confirms the data is not the problem — select business_unit, count(*) from duly_task group by 1 gives 5 distinct unit ids over 186 rows, 0 nulls.

Why it lands on (empty) rather than failing

packages/plugin-grid/src/useGroupedData.ts:119, first line of buildSegmentLabel:

if (value === undefined || value === null || value === '') return '(empty)';

That guard is right for a genuinely empty value and cannot distinguish it from a field that was never fetched. Everything below it handles lookups well — an expanded { id, name } resolves through name/label/title/…, and a bare FK string falls through to String(value), which would at least have shown five distinct id buckets. Neither path is reached, because the value is undefined before any of it runs.

So the defect is upstream of the label builder, in the projection: the query should include every field the view groups by.

Why this is worth prioritising

It is a silent wrong answer, not a missing feature, and it is invisible in exactly the conditions authors test under:

• objectstack validate and build are green — business_unit is a real field on the object, so every binding check passes;
• the view renders, the row count is right, the rows are right;
• the only symptom is that the grouping did nothing, and (empty) reads as "these records have no unit" — a plausible, wrong, actionable conclusion. A manager looking at this concludes the org data is unpopulated.

An author whose grouping field happens to also be a displayed column never sees it, which is why this survives casual use.

Reproduce

// any object with a populated field `f`
{ type: 'grid',
  columns: [{ field: 'a' }, { field: 'b' }],   // note: no `f`
  grouping: { fields: [{ field: 'f' }] } }

Open the view. One group, (empty), all rows. Add { field: 'f' } to columns and it groups correctly.

Suggested direction

Union the grouping fields into the select projection where it is built, rather than requiring authors to mirror them in columns — grouping by a field you do not want as a visible column is an ordinary thing to want, and it is what the two config keys being separate implies is supported.

Worth checking the same builder for the neighbouring cases before closing: a sort on a non-column field, and the groupBy fields on kanban and gantt, all take a field name from a config key that is not columns.

If the projection is deliberately columns-only, then the author-time gate is the alternative — refuse a grouping.fields[] entry that names no column — but the projection fix is the one that matches what the schema already lets you write.

Application impact

In the app that found this, the affected view is the manager's org lens: "show me what is outstanding, by business unit". It shipped green, with a passing metadata-binding test that resolves business_unit against the object schema — which it does resolve, correctly, and which is exactly why the test could not catch this.

[os-warren]

Follow-up measurement: the neighbouring cases are fine — this is narrower than I suggested

I closed the report with "worth checking the same builder for the neighbouring cases before closing: a sort on a non-column field, and the groupBy fields on kanban and gantt." We have now checked them, so you do not have to.

Kanban, gantt and timeline each union their groupByField into the select through their own adapter, with no column needed. Four browser ablation legs against a dev server with real data, each mutation confirmed on disk by anchored grep counts and each confirmed at runtime by the changed select= in the captured request:

leg	change	resulting projection
A	board kanban.groupByField → business_unit	select=id,subject,due_date,owner,source,business_unit,status
B	A, plus business_unit added to kanban.columns	unchanged — kanban.columns is the card face, not the fetch
C	schedule gantt.groupByField → business_unit	select=…,due_date,business_unit
D	recent gains timeline.groupByField	select=…,business_unit,due_date

So the grid's grouping block is the only projection-blind path of the four. That should shrink the fix and the regression surface: it is one code path, not a pattern repeated across view types — and three working implementations sit next to it as references for what the grid's should do.

sort was left alone deliberately and I would not read that as cleared. We found no failure, but we also could not construct one, and sorting is applied server-side through the query rather than client-side over the projection, so the mechanism that breaks grouping does not obviously apply. Recording it as unmeasured rather than fine.

The app-side workaround has landed (objectstack-ai/duly#85) with a repo-local guard that fails when a grouped grid does not carry its grouping fields as columns. That guard is explicitly filed for deletion when this issue closes, so it will not linger as a permanent convention once the projection does the right thing.

---

Generated by Claude Code

[os-warren]

Claimed — domain:ui execution seat

• session session_012wwHa4aaFybxXrfmfHioDM
• branch claude/issue-7179-grid-grouping-projection

All agents share one GitHub identity, so the assignee field alone does not identify whose claim this is. If you are reading this before starting work on #7179 and your session ID differs from the one above, this card is taken.

Why this one is dispatchable now

The follow-up measurement in the comment above is what made it so. It converted the body's open question — "worth checking the same builder for the neighbouring cases" — into a bounded answer: kanban, gantt and timeline each union their groupByField into the projection through their own adapter, so the grid's grouping block is the only projection-blind path of the four. That turns a suspected pattern into one code path with three working references sitting next to it.

The reporter also left one thing explicitly unmeasured rather than cleared: sort on a non-column field. That distinction is being honoured — the dispatch scopes sort out rather than treating "no failure found" as "no failure".

On the numbers in this comment

Every count, line number and file path in the issue body and in the comment above is the reporter's measurement, taken against published @objectstack/* 17.2.0 in a browser. I have read neither useGroupedData.ts nor the projection builder. The dispatch carries them as assumptions to falsify, not as findings, and instructs the implementing agent not to copy any figure out of my order or out of this comment into code, comments, or its report without measuring it first.

That instruction is not boilerplate. Earlier this shift I propagated a provider count of "exactly ONE" into a claim comment when the true figure was four — the original census had scoped packages/ examples/ and excluded apps/, which was that very card's own defect class. It was caught only because the dispatch order forbade copying counts out of the claim comment.

---

Generated by Claude Code

[os-warren]

os-dev-report

{
  "issue": 7179,
  "status": "done",
  "branch": "claude/issue-7179-grid-grouping-projection",
  "pr": "https://github.com/objectstack-ai/objectui/pull/7214",
  "premise_still_valid": true,
  "summary": "The grouping fields are now unioned into the query projection at BOTH sites that build one — ObjectGrid.getSelectFields() when the grid fetches for itself, and ListView's selectFields IIFE when ListView fetches and hands rows down — plus into $expand at both, so a lookup grouping field groups by name rather than by raw foreign key. A new shared harvester, collectGroupingFieldRefs, lives in @object-ui/core so one reader serves both halves. The union is gated on the speculative side of each builder: intersected with the object's declared fields (an unknown $select key zeroes the whole list on backends that reject rather than ignore it) and passed through the same FLS gate as columns, so objectui#6898 stays closed. ObjectGrid's fetch effect gained a content key over the grouping field NAMES so that switching grouping at runtime re-issues the query while collapsing a group does not. The (empty) guard, sort, and the kanban/gantt/timeline paths are untouched per the ruling.",
  "tests": "All runs on the final commit 8006103ba (tree clean; ablation restored to HEAD before every measurement). (1) pnpm exec vitest run packages/plugin-grid/ packages/plugin-list/ packages/core/ => 'Test Files 272 passed (272) / Tests 3846 passed (3846)', exit 0. (2) The four suites that matter — the two new projection files, the new core unit file, and projectionFls-6898.test.tsx as regression control => 'Test Files 4 passed (4) / Tests 52 passed (52)', exit 0; the file count is the asserted control that the right suites ran. (3) pnpm lint, the whole farm, eslint . --no-inline-config => 'Tasks: 47 successful, 47 total', exit 0 — NOT narrowed, so no narrowing declaration is owed. (4) turbo run type-check for the three packages => 'Tasks: 16 successful, 16 total', exit 0; each package runs 'tsc --noEmit && tsc -p tsconfig.test.json' and all three new test files were confirmed present in the test program via --listFiles (1/1/2 hits), so the green covers the tests and is not the excluded-test-files false green. (5) Gate family derived BY HAND from this repo's package.json and workflows, since objectui has no scripts/pm/ and objectstack's dispatch-gates script answers only about its own tree: check:control-bytes, check:self-import, check:esm-specifiers, check:phantom-deps, check:vi-mock-specifiers, check:vi-mock-inherit, check:side-effects-array, check:entry-guard, check-changeset-presence, check-changeset-no-major — all exit 0, each read from the gate's own printed verdict line. Every exit code was captured by redirecting to a file first, never after a pipe. ABLATION, three legs, direction AND counts predicted before running, each mutation proven on disk by anchored marker count AND blob hash, each restore proven by state (git diff HEAD, git diff --cached, git status --short all empty) via 'git checkout HEAD -- ABSOLUTE_PATH' (never the bare form, which restores from the index), with restore trapped on EXIT INT TERM and the PASSING count asserted alongside the red: leg A, ObjectGrid projection union removed, predicted 5 failed / 47 passed, observed 5 failed / 47 passed; leg B, ListView projection union removed, predicted 4 failed / 48 passed, observed 4 failed / 48 passed; leg C, both $expand augmentations removed, predicted 2 failed / 50 passed, observed 2 failed / 50 passed. All three matched exactly. Leg A left the ListView file fully green and leg B left the ObjectGrid file fully green — that pair is the measurement showing the two sites are genuinely independent and that a one-site fix would have shipped looking correct. Leg A marker evidence: removed-token 1 to 0, injected-token 0 to 1, blob 0a3f9248 to 488b47ab; restore confirmed both files byte-identical to their HEAD blobs. NOT MEASURED, reported as neither red nor green: check:readme-exports and check:eager-closure both require a full monorepo build this worktree does not have and each says so itself — readme-exports prints 'the population COLLAPSED, this run proves nothing' with 24 packages unbuilt, eager-closure prints a missing apps/console/dist/eager-closure.json and calls itself 'a broken gauge, not a sensitive gate'. Neither reads any file in this diff. CI builds before it runs them.",
  "mcp_calls": "4 — search_issues (failed on quota), create_pull_request, pull_request_read (mandatory body read-back), add_issue_comment. The issue body and both its comments were read through the zero-quota public-repo payload channel, not MCP.",
  "open_questions": [],
  "out_of_scope_findings": [
    "NOT FILED — needs the PM to file: a null entry in grouping.fields[] crashes the whole grid. ObjectGrid's groupValueFormatter memo dereferences gf.field with no null guard, throwing a TypeError before any projection is built. Confirmed PRE-EXISTING at this branch's base commit 4f596e01a (identical code, only the line number moved). Surfaced by my own malformed-block test, which I then narrowed to exclude null rather than ride the fix. A different defect class from this card's silent wrong answer. The new harvester handles null correctly and that is pinned in packages/core/src/utils/__tests__/grouping-fields.test.ts, which needs no grid to mount.",
    "NOT FILED — needs the PM to file: $expand carries no FLS gate at either projection site. $select is FLS-gated on both paths, but buildExpandFields is handed a column list that was never filtered, so a denied lookup COLUMN is still expanded. Adjacent to objectui#6898; affects columns generally rather than grouping, so fixing it here would have widened the review surface past this card.",
    "NOT FILED — needs the PM to file: ListView's speculative view bindings are not FLS-gated. The kanban / gantt / timeline / calendar / gallery fields routed through addSpeculative are checked against the known-field set but never against checkField. The grouping field this PR adds IS FLS-gated, so this PR does not extend the gap — it leaves an asymmetry worth closing deliberately rather than as a rider.",
    "FILING BLOCKED, cause: the mandatory duplicate check could not be run. MCP search_issues returned 'failed to search issues: failed to fetch issue field values: API rate limit already exceeded for user ID 318460043', so the control word could not be answered and an empty result would not have been a reading. Authenticated REST is not an alternative on this seat: repo-scoped endpoints answer 403 'GitHub access is not enabled for this session' and /search/* is refused by path policy outright. Note the /rate_limit endpoint DID answer, reporting core 14152 and graphql 10000 remaining — those numbers are not evidence the channel is open, and acting on them would have been the trap."
  ]
}

Falsified ZONE 2 assumptions

1. "select is built at a single site" — FALSE. It is built at TWO, and they are independent: ObjectGrid.getSelectFields() (grid fetches for itself) and ListView's selectFields IIFE (ListView fetches, grid renders). The coordinator's mid-task correction was right, and the count is exactly two — a census of $select across packages/*/src found no third builder on this path, and useGroupedData has exactly one consumer, ObjectGrid. Both are fixed, each with its own regression file.
2. "Kanban, gantt and timeline each union their groupByField through their own adapter" — FALSE. Those three packages contain no projection builder at all; a $select census across their src/ returns zero hits. All three are served by one shared arm, collectViewFields in ListView.tsx. The grid was the odd one out not for lacking an adapter but because it spells the same intent as grouping.fields[] — an array of objects — matching none of that collector's plain-string candidate keys. The conclusion the comment drew is still correct; the mechanism is not.
3. "useGroupedData.ts:119 is where (empty) originates" — CONFIRMED, and the line number still holds on main: line 119 is the first line of buildSegmentLabel, and it groups over already-fetched rows (row[f.field] at line 260). Untouched, per the ruling.
4. "A grouping.fields[] entry carries a plain field string" — CONFIRMED against the spec, not assumed. GroupingFieldSchema in @objectstack/spec@17.2.0 is a $strict object of exactly { field: z.ZodString, order, collapsed } — no expression form, no nested-path form. A bare-string shorthand is refused deliberately (off-spec, and useGroupedData reads f.field, so it could never have worked end to end).

Did populate turn out to be part of the fix?

Yes. buildExpandFields restricts expansion to the visible columns, so a lookup grouping field that is not a column was never expanded and the rows carried bare foreign keys — five id buckets instead of five names. ListView's own expandFields memo already records this exact failure for kanban in its comment. Grouping fields now reach $expand at both sites, with its own pin on each side and a dedicated ablation leg (leg C). No unknown-key gate is needed on that half and none was added: buildExpandFields returns a subset of the declared reference-bearing fields, so an unknown or non-relational grouping field is dropped structurally.

The hazard the correction named

Handled as directed and pinned in both directions, because they point opposite ways: an unknown grouping field must be refused (PIN 4) and the rest of the projection must survive that refusal (its positive half), so neither a poisoned query nor a collapsed one can pass. projectionFls-6898.test.tsx runs green before and after as the regression control, with the file count asserted.

Generated by Claude Code

---

Generated by Claude Code

[os-warren]

Implemented — PR #7214, reviewed and armed

domain:ui seat, session session_012wwHa4aaFybxXrfmfHioDM. Review posted on the PR; ⛔ no second review owed.

Fixed at BOTH projection sites, not one — ObjectGrid.getSelectFields() and ListView's selectFields builder — plus $expand at both, so a lookup grouping field groups by name rather than by raw foreign key. A shared harvester, collectGroupingFieldRefs, lives in @object-ui/core so one reader serves both halves.

The dispatch order was wrong, and the correction was the load-bearing part

My order asserted a single $select build site. That was my reading, not a measurement, and it was false. I caught it by grepping mid-flight and sent the lane a correction naming both sites plus two hazards the order had missed. The ablation is what turned that from a claim into a fact:

leg A — the ObjectGrid union removed → 5 failed / 47 passed, ListView's file fully green
leg B — the ListView union removed → 4 failed / 48 passed, ObjectGrid's file fully green

Each site fails alone. A one-site fix would have shipped with a green suite and a still-broken surface — which is exactly what my original order would have produced.

All three ablation legs matched their predicted counts exactly, with the passing count asserted alongside each red so a collapsed population could not pass for a clean run.

The hazard the order had missed

ListView's builder gates "speculative" fields against the object's declared field set, and its own comment records why: some backends answer an unknown $select key with an empty result set rather than ignoring it — the cloud multi-tenant runtime does exactly that, so a single unknown key silently zeroes the whole list.

A grouping.fields[] entry is speculative in precisely that sense — this card's whole premise is that it is not a column, so nothing has validated it. Unioned unguarded, the fix would have converted this card's bug (one (empty) group holding 186 rows) into zero rows, equally silently. Strictly worse: the current bug at least shows the data.

Both gates are applied, in the right order — declared-fields intersection first, checkField only on survivors, because checkField answers false for an undeclared key and would otherwise drop derived columns.

sort stayed out, as you asked

Your follow-up recorded sort as unmeasured rather than fine. The dispatch scoped it out on that basis and the lane did not touch it. It remains unmeasured — not cleared.

Three findings surfaced and filed rather than ridden

The lane reported these instead of widening the PR; filed by this seat, all unassigned:

• $expand carries no FLS gate at either projection site, so a lookup column the principal cannot read is still expanded and its value returned #7215 — $expand carries no FLS gate at either site, so a denied lookup column is still expanded. Adjacent to plugin-grid: ObjectGrid builds the server $select from the authored projection with no FLS gate #6898, and arguably sharper: $select on a denied lookup returns a bare key, $expand returns the resolved record.
• ListView's speculative view-binding fields (kanban / gantt / timeline / calendar / gallery) reach $select without an FLS check #7216 — ListView's speculative view bindings (kanban / gantt / timeline / calendar / gallery) reach $select known-field-checked but never FLS-checked.
• A null entry in grouping.fields[] throws a TypeError that takes down the whole grid, before any projection is built #7217 — a null entry in grouping.fields[] throws a TypeError that takes down the whole grid. Confirmed pre-existing at this branch's base 4f596e01a.

None is a regression from this PR. #7214 gates everything it adds; it made the neighbouring asymmetries visible by being careful about its own half.

#7189 unblocks when this lands — it was held deliberately, because once the projection carries the grouping field the groups resolve and page-scoping becomes the visible failure instead of the (empty) collapse.

---

Generated by Claude Code

[os-warren]

Landed — a6d8b8d44, verified by CONTENT at BOTH sites

PR #7214 merged. Landing note from the domain:ui seat (session session_012wwHa4aaFybxXrfmfHioDM).

Verified on origin/main rather than by sha (a squash makes a new commit, so the branch head never becomes an ancestor). The probe was built around the two-site property, because that is what a one-site landing would silently fail:

probe	on main
packages/core/src/utils/grouping-fields.ts exists	yes
collectGroupingFieldRefs in plugin-grid/src/ObjectGrid.tsx	3
collectGroupingFieldRefs in plugin-list/src/ListView.tsx	3

Both halves are on main. A landing that carried only one would have looked like success against a naive "did the fix land?" probe — which is exactly the failure the ablation predicted, since each site's tests pass while the other is broken.

What closed

A grid declaring grouping on a field absent from its columns rendered one group labelled (empty) containing every row — no error, no warning, validate and build green, the row count right and the rows right. The only symptom was that the grouping did nothing, and (empty) reads as "these records have no unit", which is a plausible, wrong, actionable conclusion.

The projection now unions the grouping fields at both build sites, gated twice — intersected against the object's declared fields, then FLS-checked — and $expand is augmented too, so a lookup grouping buckets by name rather than by raw foreign key.

The correction that mattered

My dispatch order asserted a single $select build site. That was my reading, not a measurement, and it was wrong. The ablation is what turned the correction into a fact: removing either site's union leaves the other site's test file fully green. A one-site fix would have shipped with a passing suite and a still-broken surface.

sort stayed out, as your follow-up asked — it was recorded as unmeasured rather than fine, and it remains so.

Follow-ons now open

• Grid grouping and its per-group counts are computed over the fetched page, so a group can be missing entirely with nothing on screen saying so #7189 is UNBLOCKED by this landing and is next in the dispatch queue. It was held deliberately: until the projection carried the grouping field, its page-scoping symptom was masked by the (empty) collapse. Its numbers will now be the ones it actually describes.
• $expand carries no FLS gate at either projection site, so a lookup column the principal cannot read is still expanded and its value returned #7215 ($expand has no FLS gate at either site) · ListView's speculative view-binding fields (kanban / gantt / timeline / calendar / gallery) reach $select without an FLS check #7216 (ListView's speculative view bindings are known-field-checked but not FLS-checked) · A null entry in grouping.fields[] throws a TypeError that takes down the whole grid, before any projection is built #7217 (a null entry in grouping.fields[] crashes the grid — pre-existing at this branch's base).

None is a regression from this PR; it gated everything it added and made the neighbouring asymmetries visible by doing so.

Closing. pm:dispatched cleared.

---

Generated by Claude Code