Skip to content

Commit e87070e

Browse files
fix(objectql): a find/findOne projection that names a formula field returns that projection, not every stored column (#22337)
Fixes #22300 Clause-②: no ## What changes `planFormulaProjection` (`packages/objectql/src/engine.ts`) widens a `fields` projection that names a formula field to every stored column plus `id`, so the formula's CEL `record.FIELD` lookups see the full row. `find` and `findOne` never cut that widening back off their results. A projection that named a formula field therefore returned every column of the record. - `planFormulaProjection` now also returns `widened`: the columns it added beyond what the caller named. `id` is one of them when the caller did not name it. - New `withoutFormulaWidening` / `rowsWithoutFormulaWidening` remove exactly those columns from the result. `find` and `findOne` call them at their `return`, after `executeWithMiddleware`. - The driver read is unchanged. The formula still sees the full row. ## Where the cut sits (M4), and what still reads the widened row Every internal reader of the widened rows runs before the cut, and they run in this order: 1. the formula pass (`resolveFormulaPermissions`, `applyFormulaPlan`), including the unevaluated-formula fault sink; 2. `expandRelatedRecords`, which reads only the foreign-key keys named in `expand`; 3. `resolveFileReferences`; 4. the `afterFind` hooks; 5. `maskSecretFields` / `omitInternalFields` and `stripSearchCompanionFromRead`; 6. the middleware post-phase. This includes the field-level-security result mask, and the audit redactions that add their judged columns to the projection and remove them afterwards. Cutting any earlier would starve one of these of a column it reads today. The cut removes the widened columns rather than keeping a list. A key an `afterFind` hook derives survives, and so does an expanded relation the caller named. Rows are copied, never mutated, in their own key order. Post-hoc sorting of formula fields is refused before planning (`assertOrderByIsMaterializable`), so it reads no row. ## Measurements ### M1: the widening, reproduced on `main` before the change - Engine: the new `engine-formula-projection-trim.test.ts`, at base `f2626c71db`: 8 failed and 4 passed of 12. The 4 that passed are the controls. Each failing case received 12 unrequested keys. - Door: a flow `get_record` on the real stack, `driver-sql` on better-sqlite3. That is `get-record-formula-projection.integration.test.ts`, at base `f2626c71db`: 8 failed of 8, covering both node branches and both `runAs`. Each served row carried 11 unrequested keys. - Also measured on the same base: the REST data read, `POST /api/v1/data/:object/query` with `fields`. It returns the same 13 keys for a formula projection. The same cut covers it. ### M2: field-level security against the widening The FLS result mask is `SecurityPlugin.maskOperationResult` → `FieldMasker.maskResults`. It is step 4 of the security middleware and runs in the middleware post-phase. That is after `planFormulaProjection` widened the driver read and after `applyFormulaPlan` ran, on the rows the read returns. Measured on base `f2626c71db` with the real `SecurityPlugin` over `driver-sql`. A member's permission set withheld one field (`readable: false`) that the projection did not name. The read was `POST /api/v1/data/:object/query` plus direct `find` / `findOne` with the member context: | read (member) | keys returned on base | withheld field present | |---|---|---| | `fields: [name, total_price]` | 13: name, total_price, quantity, unit_price, note, id, organization_id, owner_id, owning_business_unit_id, created_by, updated_by, created_at, updated_at | no | | `fields: [name]` | name | no | | no projection | the same 13 | no | **Answer: only columns the caller may read leave.** The widened read never carried the withheld field. It carried exactly the no-projection read's column set. That is the contract breach this card names, with no field-permission bypass. The seat may relay this to triage to drop `security`. A system-identity caller (for example a flow with `runAs: system`) has no field mask, so every column is readable to it. Its widened read is the same contract breach, not a permission bypass. ### M3: what a projection without a formula returns - `driver-sql`, measured through the door pin's control: `fields: [name, quantity]` returns exactly `name, quantity`. There is no `id` unless it is named. - The engine adds no key to a projection. `PLATFORM_PROVISIONED_COLUMNS` only admits names in the unknown-plain-field filter. - So the cut restores exactly the named columns plus the formula value. The pins assert this as row equality: the formula read equals the same read without the formula, plus the formula value, on `driver-sql` and on the driver-sql-shaped double. - `driver-memory` and `driver-mongodb` behave differently, and that is not engine behaviour: each adds `id` to every projection at the driver layer. See the measured reading in Acceptance notes. ### M5: other read paths `planFormulaProjection` has four callers: - `find` and `findOne` pass the caller's projection. - `hydrateWriteFormulas`, the write-result hydration, passes no projection. - `evaluateFormulaField` passes one field and evaluates on a copy. `count`, `aggregate` (its `driver.find` fallback included) and the write paths' pre-image reads never plan a formula projection. ## Base vs head: keys returned per case Engine table, driver-sql-shaped double, `find` and `findOne` alike. The stored row also carries `note`, `parent_id` and the seven provisioned columns: | projection | base `f2626c71db` | head | |---|---|---| | `[name, total_price]` | 14 keys (every stored column + `total_price`) | `name, total_price` | | `[id, owner_id, total_price]` | 14 keys | `id, owner_id, total_price` | | `[name, total_price]` + an `afterFind` hook adding a key | 15 keys | `name, total_price` + the hook's key | | `[name, quantity]` (control) | `name, quantity` | `name, quantity` | | none (control) | every declared column + `total_price` | unchanged | Door, flow `get_record` on `driver-sql`, both branches × both `runAs`: | `config.fields` | base `f2626c71db` | head | |---|---|---| | `[name, total_price]` | 13 keys | `name, total_price` | | `[name, quantity, total_price]` | 13 keys | `name, quantity, total_price` = control + `total_price` | | `[name, quantity]` (control) | `name, quantity` | `name, quantity` | ## Tests Runs are at head `99ffeb8fa5` unless noted. The one exception is the full objectql suite, which ran at `aaa5e4deb2`. Between that commit and head, the only change is the reshaped test double in the new test file, and that file was re-run at head. | what | command | result | |---|---|---| | objectql, full | `pnpm --filter @objectstack/objectql exec vitest run --project local --maxWorkers=2` (at `aaa5e4deb2`) | 386 files, 7600 tests passed | | objectql, repo project | `… --project repo` (at `aaa5e4deb2`) | 1 file, 5 tests passed | | the new engine pin | `… src/engine-formula-projection-trim.test.ts` | 12 passed | | objectql typecheck | `pnpm --filter @objectstack/objectql typecheck` | exit 0. Test-layer debt is unchanged (40 files, 234 errors, 65 signatures), and the new test file is in the `tsconfig.test.json` program (`--listFilesOnly`) | | service-automation, full | `pnpm --filter @objectstack/service-automation exec vitest run --maxWorkers=2` | 177 files, 2165 tests passed | | service-automation typecheck | `pnpm --filter @objectstack/service-automation typecheck` | exit 0. Test-layer debt is 0, and the door pin is in the program | | rest | `pnpm --filter @objectstack/rest exec vitest run --project local` / `--project repo` | 263 files, 4951 passed and 326 skipped / 5 files, 191 passed and 1 skipped | **Reverse verification**, at head, through `scripts/ablation-replace.mjs`: - **Mutation.** `withoutFormulaWidening` was changed to return every row untouched (anchor 1 → 0, blob `42aff651c70c` → `45700267e608`). - **Engine pin** (imports source): 8 failed, 4 passed. The 4 controls stay green. - **Built artifact.** The mutant JS was emitted with the marker in 4 built files (`ablation-dist-preflight` exit 0). The mutant's DTS step failed on the type narrowing the mutation removed (`TS18048`). The JS the suite consumes does not depend on that step. - **Door pin** (imports objectql `dist`): 8 failed of 8. - **Restore.** The blob equals HEAD, `git diff HEAD` is empty, the rebuild exits 0, the marker is absent from all 14 built files, and the tree is clean. Afterwards the engine pin passed 12 and the door pin passed 8. - **Direction:** red, as expected. **Gates.** `node scripts/pm/dispatch-gates.mjs --commands` at head derived 69 commands. All 69 ran and each exited 0. - `--ran` reconciliation: 69 derived, 69 run, 0 NOT-MEASURED, 0 UNRUN. The zero is derived from the recorded exit codes. - `check:objectql-double-limit` first failed on the new test double, because its probe could not drive it. The double was reshaped in `99ffeb8fa5`, and the gate now passes. **Lint, narrowed.** `npx eslint --no-inline-config --format json` on the three touched `.ts` files at head returned 3 files, 0 errors and 0 warnings, exit 0. 1. **Scope.** `eslint.config.mjs`'s `**/*.{ts,…}` and `packages/**/*.{ts,…}` blocks cover all three files, and all three come back in the JSON with results, so none was ignored. 2. **Count.** The JSON shows 3 files. 3. **Why other files cannot change.** `eslint.config.mjs` never enables type-aware linting: there is no `parserOptions.project` and no typed rule, as its own comment states. This diff therefore cannot change a verdict on a file it does not touch. Repo-wide `pnpm lint` is left to CI. **NOT MEASURED locally, left to CI:** - the Test Core shards; - Dogfood; - Temporal Conformance; - Build Core; - the workspace type-check lanes; - `driver-mongodb`'s projection, which was read from source only. ## Clause-② `Clause-②: no`, measured. The built entry declarations of `@objectstack/objectql` are byte-identical between the merge base `fe98cc63a4` and head `99ffeb8fa5`. The table gives each file's sha256 prefix, which is the same on both sides: | file | sha256 prefix | |---|---| | `dist/index.d.ts` | `edb995cf7c2baa9f` | | `dist/core.d.ts` | `3b5e05652e32b56e` | | `dist/util-BuqCJOyg.d.ts` | `357f6c243d69e450` | The new helpers are module-private. What changes is runtime behaviour: a projection that names a formula field now returns that projection, which is the direction the card asks for. ## Acceptance notes - **`id` on two drivers.** `driver-memory` and `driver-mongodb` add `id` to every projection at the driver layer (`InMemoryDriver.projectFields`, `MongoDBDriver.buildFindOptions`). The `driver-sql` family does not, and no driver contract declares either behaviour. - Measured at head with `driver-memory` through the built engine: `fields: [name]` returns `name, id`, while `fields: [name, total_price]` returns `name, total_price`. - So on those two drivers, a formula projection now omits the `id` that the same projection without a formula carries. On base it carried `id` together with every other column. - Carrier: none for the driver-level divergence (no contract declares either behaviour; the contract review 6066975867 answered A). - **Formula evaluation and field-level security.** The formula pass evaluates against the row before the field-level-security result mask runs, because the mask runs in the middleware post-phase. That holds on every read, with or without a projection. - Whether a formula over a withheld field should be evaluated for that caller is not measured here, and this PR does not change it. Carrier: none. - **What `afterFind` hooks see.** `afterFind` hooks still read the widened row, as before. If a hook re-assigns a key that the widening added, the cut removes it, because the caller never named it. ## Files - `packages/objectql/src/engine.ts`: the read path's formula projection region only, meaning `planFormulaProjection`, the new cut helpers and the two `return`s. - `packages/objectql/src/engine-formula-projection-trim.test.ts`: new. - `packages/services/service-automation/src/builtin/get-record-formula-projection.integration.test.ts`: new, the door pin. It is cross-lane (`domain:services`) and only adds a test. - `.changeset/22300-formula-projection-trim.md`: `@objectstack/objectql` patch. ## Patch round 1 Head `b6dbe39576` adds one commit on top of `99ffeb8fa5`. It changes one sentence in `.changeset/22300-formula-projection-trim.md` and nothing else. - **Why.** The old sentence said `id` is returned "when it is named, as for any other projection". That is false on `driver-memory` and `driver-mongodb`, which add `id` to every projection at the driver layer. - **New text:** "`id` is returned only when it is named. `driver-memory` and `driver-mongodb` add `id` to every projection at the driver layer, so on those two drivers a projection that names a formula field but not `id` no longer carries `id`, while the same projection without the formula still does: name `id` when you need it." - **Each clause, checked against the code:** - On every driver, the cut removes `id` whenever the widening added it, which is whenever the caller did not name it. - `InMemoryDriver.projectFields` and `MongoDBDriver.buildFindOptions` add `id` to any non-empty projection, on `find` and `findOne` alike. - Measured on `driver-memory`: `[name]` returns `name, id`, and `[name, total_price]` returns `name, total_price`. - **Unchanged.** No code or test changed, so the test runs and the `.d.ts` comparison above still hold: a changeset is not part of any build. - **Gates for the touched path, at `b6dbe39576`.** `dispatch-gates --commands .changeset/22300-formula-projection-trim.md` derived 20 commands, including `check-changeset-no-major --base origin/main`, `check-empty-changeset --base origin/main` and `check-adr-0087-registration --base origin/main`. All 20 exited 0. `--ran` reconciliation: 20 derived, 20 run, 0 NOT-MEASURED, 0 UNRUN. - **Merge check.** `git merge-tree --write-tree` against `origin/main` `3599fef123` is clean: exit 0, no conflicts. `main` has touched none of this PR's four files since the merge base, so nothing was merged. --- _Generated by [Claude Code](https://claude.ai/code/session_01EUBvqtauTDmHi2ZgY759p2)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 35afb15 commit e87070e

4 files changed

Lines changed: 443 additions & 4 deletions

File tree

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
'@objectstack/objectql': patch
3+
---
4+
5+
fix(objectql): a `find` / `findOne` whose `fields` names a formula field returns that projection, not every stored column
6+
7+
Clause-②: no
8+
9+
To evaluate a formula field, the engine widens the projection it hands the driver to every stored column of the object (CEL's `record.<field>` reads whatever the formula needs off the full row). The rows were never cut back, so a projection that named a formula field returned every column: the tenant, owner, owning unit and audit columns, and every field the caller did not name. A flow's `get_record`, whose `config.fields` is declared as "only these fields are read", passed all of them on to its later nodes.
10+
11+
Each row is now cut back to the caller's projection once the read is done: the columns the caller named, plus the formula's computed value. `id` is returned only when it is named. `driver-memory` and `driver-mongodb` add `id` to every projection at the driver layer, so on those two drivers a projection that names a formula field but not `id` no longer carries `id`, while the same projection without the formula still does: name `id` when you need it. The formula still sees the full row, and so do the `afterFind` hooks and the middlewares, as before. A key an `afterFind` hook derives is kept.
12+
13+
Unchanged: a projection that names no formula field, and a read with no projection (every declared column, with the formulas computed). Field-level security is unchanged as well: a field the caller may not read was already masked off the widened row, and still is.
Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [#22300] A `formula` field in a `fields` projection widens the DRIVER read to
5+
* every stored column (`planFormulaProjection`: CEL's `record.<field>` must see
6+
* the whole row), and the rows are cut back to the caller's projection once
7+
* the read is done. The formula still sees the full row; the caller does not.
8+
*
9+
* The table: {a projection naming a formula, one naming none, no projection}
10+
* × {`find`, `findOne`}, each asserting the EXACT key set a row leaves with.
11+
* The driver honours a projection the way driver-sql does — exactly the named
12+
* columns, nothing it was not asked for — so the projection WITHOUT a formula
13+
* is the reference: the row with a formula equals that row plus the formula's
14+
* value, key for key and value for value.
15+
*
16+
* The rows the driver stores carry the columns the registry provisions on
17+
* every business object (tenant, owner, owning unit, the audit actors and
18+
* timestamps); none of them is requested, so none may leave a projected read.
19+
*/
20+
21+
import { describe, it, expect } from 'vitest';
22+
import { ObjectQL } from './engine.js';
23+
24+
import '@objectstack/spec';
25+
import '@objectstack/formula';
26+
27+
type Row = Record<string, unknown>;
28+
29+
const OBJECT = 'fpt_line_item';
30+
31+
/** The stored row: two requested inputs, the formula's inputs, and the provisioned columns. */
32+
const STORED: Row = {
33+
id: 'li_1',
34+
name: 'Widget',
35+
quantity: 3,
36+
unit_price: 7,
37+
note: 'not requested',
38+
parent_id: 'opp_1',
39+
organization_id: 'org_1',
40+
owner_id: 'usr_1',
41+
owning_business_unit_id: 'bu_1',
42+
created_by: 'usr_1',
43+
updated_by: 'usr_1',
44+
created_at: '2026-01-01T00:00:00.000Z',
45+
updated_at: '2026-01-02T00:00:00.000Z',
46+
};
47+
48+
/**
49+
* A driver that answers a projection the way driver-sql does — exactly the
50+
* named columns, a stored NULL as `null` — and records every projection it was
51+
* handed, so a case can see what the formula was evaluated against. Shallow
52+
* copies only: the formula pass writes onto the rows a driver returns.
53+
*/
54+
function makeDriver() {
55+
const stores = new Map<string, Map<string, Row>>();
56+
const storeFor = (o: string) => {
57+
let s = stores.get(o);
58+
if (!s) { s = new Map(); stores.set(o, s); }
59+
return s;
60+
};
61+
storeFor(OBJECT).set(String(STORED.id), { ...STORED });
62+
const projections: Array<string[] | undefined> = [];
63+
const matches = (row: Row, where: unknown): boolean => {
64+
if (!where || typeof where !== 'object') return true;
65+
return Object.entries(where as Row).every(([k, v]) => {
66+
if (k === '$and') return (v as unknown[]).every((w) => matches(row, w));
67+
if (k === '$or') return (v as unknown[]).some((w) => matches(row, w));
68+
// Any other combinator is REFUSED, never read as a field name
69+
// (`check:where-matcher`).
70+
if (k.startsWith('$')) throw new Error(`test driver: unsupported combinator ${k}`);
71+
const cond = v as Row | null;
72+
if (cond && typeof cond === 'object' && !Array.isArray(cond)) {
73+
if ('$in' in cond) return Array.isArray(cond.$in) && cond.$in.includes(row[k]);
74+
if ('$eq' in cond) return row[k] === cond.$eq;
75+
throw new Error(`test driver: unsupported condition on ${k}`);
76+
}
77+
return row[k] === cond;
78+
});
79+
};
80+
const project = (row: Row, fields: unknown): Row => {
81+
if (!Array.isArray(fields) || fields.length === 0) return { ...row };
82+
return Object.fromEntries((fields as string[]).map((f) => [f, row[f] ?? null]));
83+
};
84+
const driver: any = {
85+
name: 'sql-shaped', version: '0.0.0', supports: {},
86+
async connect() {}, async disconnect() {}, async checkHealth() { return true; }, async execute() { return null; },
87+
async find(object: string, ast: any) {
88+
const matched = Array.from(storeFor(object).values()).filter((r) => matches(r, ast?.where));
89+
// Hold the caller's bound (`check:objectql-double-limit`).
90+
const bounded = typeof ast?.limit === 'number' ? matched.slice(0, ast.limit) : matched;
91+
projections.push(Array.isArray(ast?.fields) ? [...ast.fields] : undefined);
92+
return bounded.map((r) => project(r, ast?.fields));
93+
},
94+
async findOne(object: string, ast: any) {
95+
const hit = Array.from(storeFor(object).values()).find((r) => matches(r, ast?.where));
96+
projections.push(Array.isArray(ast?.fields) ? [...ast.fields] : undefined);
97+
return hit ? project(hit, ast?.fields) : null;
98+
},
99+
async create() { throw new Error('test driver: reads only'); },
100+
async update() { throw new Error('test driver: reads only'); },
101+
async delete() { throw new Error('test driver: reads only'); },
102+
async count(object: string) { return storeFor(object).size; },
103+
};
104+
return { driver, projections };
105+
}
106+
107+
async function boot() {
108+
const engine = new ObjectQL();
109+
const { driver, projections } = makeDriver();
110+
engine.registerDriver(driver, true);
111+
await engine.init();
112+
engine.registry.registerObject({
113+
name: OBJECT,
114+
label: 'Line Item',
115+
fields: {
116+
name: { name: 'name', type: 'text' },
117+
quantity: { name: 'quantity', type: 'number' },
118+
unit_price: { name: 'unit_price', type: 'number' },
119+
note: { name: 'note', type: 'text' },
120+
parent_id: { name: 'parent_id', type: 'text' },
121+
// Reads two columns the projections below never name.
122+
total_price: { name: 'total_price', type: 'formula', expression: { dialect: 'cel', source: 'record.quantity * record.unit_price' } },
123+
},
124+
} as never, 'test');
125+
return { engine, projections };
126+
}
127+
128+
type Verb = 'find' | 'findOne';
129+
const SYS = { isSystem: true } as const;
130+
131+
async function readOne(engine: ObjectQL, verb: Verb, fields?: string[]): Promise<Row> {
132+
const query = { where: { id: STORED.id }, ...(fields ? { fields } : {}), context: SYS };
133+
if (verb === 'findOne') {
134+
const row = await engine.findOne(OBJECT, query as never);
135+
expect(row, 'findOne found no row').not.toBeNull();
136+
return row as Row;
137+
}
138+
const rows = await engine.find(OBJECT, query as never);
139+
expect(rows).toHaveLength(1);
140+
return rows[0] as Row;
141+
}
142+
143+
const PROVISIONED = [
144+
'organization_id', 'owner_id', 'owning_business_unit_id',
145+
'created_by', 'updated_by', 'created_at', 'updated_at',
146+
];
147+
148+
describe.each<Verb>(['find', 'findOne'])('#22300 — %s: a formula in the projection widens the read, never the answer', (verb) => {
149+
it('a projection naming a formula field returns exactly the named fields, the formula computed from the full row', async () => {
150+
const { engine, projections } = await boot();
151+
const row = await readOne(engine, verb, ['name', 'total_price']);
152+
153+
expect(Object.keys(row).sort()).toEqual(['name', 'total_price']);
154+
// 3 × 7: the formula read `quantity` and `unit_price`, neither of them named.
155+
expect(row.total_price).toBe(21);
156+
for (const col of [...PROVISIONED, 'id', 'quantity', 'unit_price', 'note', 'parent_id']) {
157+
expect(row, `'${col}' was not requested and left the read`).not.toHaveProperty(col);
158+
}
159+
// …and the driver WAS asked for the full row: the widening is intact, only
160+
// the answer is cut back.
161+
const asked = projections.at(-1);
162+
expect(asked).toEqual(expect.arrayContaining(['name', 'quantity', 'unit_price', 'id', ...PROVISIONED]));
163+
expect(asked).not.toContain('total_price');
164+
});
165+
166+
it('the row equals the same projection without the formula, plus the formula value', async () => {
167+
const { engine } = await boot();
168+
const without = await readOne(engine, verb, ['name']);
169+
const withFormula = await readOne(engine, verb, ['name', 'total_price']);
170+
expect(withFormula).toStrictEqual({ ...without, total_price: 21 });
171+
expect(JSON.stringify(withFormula)).toBe(JSON.stringify({ ...without, total_price: 21 }));
172+
});
173+
174+
it('a column the caller names is kept — `id` and a provisioned column included', async () => {
175+
const { engine } = await boot();
176+
const row = await readOne(engine, verb, ['id', 'owner_id', 'total_price']);
177+
expect(row).toStrictEqual({ id: 'li_1', owner_id: 'usr_1', total_price: 21 });
178+
});
179+
180+
it('CONTROL: a projection naming no formula field is unchanged', async () => {
181+
const { engine, projections } = await boot();
182+
const row = await readOne(engine, verb, ['name', 'quantity']);
183+
expect(row).toStrictEqual({ name: 'Widget', quantity: 3 });
184+
expect(projections.at(-1)).toEqual(['name', 'quantity']);
185+
});
186+
187+
it('CONTROL: a read with no projection is unchanged — every declared column, the formula computed', async () => {
188+
const { engine, projections } = await boot();
189+
const row = await readOne(engine, verb);
190+
expect(row).toStrictEqual({ ...STORED, total_price: 21 });
191+
expect(projections.at(-1)).toBeUndefined();
192+
});
193+
194+
it('a key an afterFind hook derives is the hook\'s, and survives the cut', async () => {
195+
const { engine } = await boot();
196+
engine.registerHook('afterFind', (ctx: any) => {
197+
const list = Array.isArray(ctx.result) ? ctx.result : [ctx.result];
198+
for (const r of list) if (r) r.line_label = `${r.name} x${r.quantity}`;
199+
}, { object: OBJECT } as never);
200+
const row = await readOne(engine, verb, ['name', 'total_price']);
201+
// The hook ran on the widened row (it read `quantity`, unrequested); its
202+
// own key survives, the column it read does not.
203+
expect(row).toStrictEqual({ name: 'Widget', total_price: 21, line_label: 'Widget x3' });
204+
});
205+
});

‎packages/objectql/src/engine.ts‎

Lines changed: 80 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1534,10 +1534,22 @@ function declaredMultiValued(field: { type?: unknown; multiple?: unknown } | nul
15341534
});
15351535
}
15361536

1537+
/**
1538+
* Plan the formula pass of a read: which formula fields to evaluate, and —
1539+
* when the caller named a projection — the projection the DRIVER is asked for.
1540+
*
1541+
* A projection naming a formula field is widened to every stored column (plus
1542+
* `id`), because CEL's `record.<field>` reads whatever the formula needs off
1543+
* the full row. That widening is a means of EVALUATING the formula and never
1544+
* an answer: [#22300] `widened` lists the columns it added beyond what the
1545+
* caller named, and {@link withoutFormulaWidening} cuts them back off the rows
1546+
* once the read is done, so the caller gets the projection it declared plus
1547+
* the formula's value. `widened` is absent whenever nothing was widened.
1548+
*/
15371549
function planFormulaProjection(
15381550
schema: any,
15391551
requestedFields: string[] | undefined
1540-
): { plan: FormulaPlanEntry[]; projected?: string[] } {
1552+
): { plan: FormulaPlanEntry[]; projected?: string[]; widened?: string[] } {
15411553
if (!schema?.fields) return { plan: [] };
15421554
const allFieldNames = Object.keys(schema.fields);
15431555
// When no explicit projection, evaluate every formula field on the schema —
@@ -1578,13 +1590,69 @@ function planFormulaProjection(
15781590
if (fdef?.type === 'formula') continue;
15791591
projected.add(fname);
15801592
}
1581-
return { plan, projected: Array.from(projected) };
1593+
// [#22300] What the widening added — `id` included when the caller did not
1594+
// name it — is what the read cuts back off once the formulas are computed.
1595+
const named = new Set(requestedFields);
1596+
const widened = Array.from(projected).filter((f) => !named.has(f));
1597+
return { plan, projected: Array.from(projected), ...(widened.length > 0 ? { widened } : {}) };
15821598
}
15831599
// Implicit/full projection — leave projected undefined so the driver
15841600
// returns its default columns (typically *).
15851601
return { plan };
15861602
}
15871603

1604+
/**
1605+
* [#22300] Cut the columns {@link planFormulaProjection} widened a projection
1606+
* by back off what a read RETURNS — the formula saw the full row, the caller
1607+
* does not. Without it, a projection naming a formula field answered with every
1608+
* stored column (tenant, owner, owning unit, audit actors and timestamps, every
1609+
* unnamed field): a flow's `get_record`, whose `config.fields` is declared as
1610+
* "only these fields are read", passed all of them on to whatever its later
1611+
* nodes sent out.
1612+
*
1613+
* ## The answer it restores
1614+
*
1615+
* The same projection without the formula, plus the formula's value: measured
1616+
* on driver-sql (a projection is exactly the named columns — no `id` unless it
1617+
* is named) and pinned against it in `engine-formula-projection-trim.test.ts`.
1618+
* `id` is cut like any other widened column when the caller did not name it.
1619+
*
1620+
* ## Where it runs, and why there
1621+
*
1622+
* On the result `find` / `findOne` hand back, AFTER the middleware chain —
1623+
* the last internal consumer. Everything before it keeps reading the widened
1624+
* row exactly as before: the formula pass, `expand`, file-reference
1625+
* resolution, the `afterFind` hooks, the secret mask and the `__search`
1626+
* strip, and the middlewares' post-phase (field-level security's result mask,
1627+
* the audit redactions that judge a row by columns the caller did not name).
1628+
* Cutting earlier would starve those of a column they read today.
1629+
*
1630+
* It REMOVES the widened columns rather than keeping a list: a key an
1631+
* `afterFind` hook derives, or an expanded relation the caller named, is not
1632+
* the widening's and survives. A widened column a hook re-assigns is cut — the
1633+
* caller never named it.
1634+
*
1635+
* Never mutates a row: a row carrying a widened column is replaced by a copy
1636+
* without it, in the row's own key order; a row carrying none is returned as
1637+
* is.
1638+
*/
1639+
function withoutFormulaWidening<T>(row: T, widened: ReadonlySet<string> | undefined): T {
1640+
if (!widened || !row || typeof row !== 'object' || Array.isArray(row)) return row;
1641+
const source = row as unknown as Record<string, unknown>;
1642+
if (!Object.keys(source).some((key) => widened.has(key))) return row;
1643+
const cut: Record<string, unknown> = {};
1644+
for (const key of Object.keys(source)) {
1645+
if (!widened.has(key)) cut[key] = source[key];
1646+
}
1647+
return cut as unknown as T;
1648+
}
1649+
1650+
/** {@link withoutFormulaWidening} over a `find` result; anything but an array passes through. */
1651+
function rowsWithoutFormulaWidening<T>(rows: T, widened: ReadonlySet<string> | undefined): T {
1652+
if (!widened || !Array.isArray(rows)) return rows;
1653+
return rows.map((row) => withoutFormulaWidening(row, widened)) as unknown as T;
1654+
}
1655+
15881656
/**
15891657
* [#7095] ORDER BY a field whose value is computed on read — refused HERE, on
15901658
* the engine's own public boundary, and no longer only at the REST ingress.
@@ -12097,6 +12165,9 @@ export class ObjectQL implements IObjectQLEngine {
1209712165
assertProjectionHasNoDottedPaths(object, 'find', _findSchema, ast.fields);
1209812166
const _findFormula = planFormulaProjection(_findSchema, ast.fields);
1209912167
if (_findFormula.projected) ast.fields = _findFormula.projected;
12168+
// [#22300] The columns that widening added, cut back off the answer at the
12169+
// `return` — see `withoutFormulaWidening`.
12170+
const _findWidened = _findFormula.widened ? new Set(_findFormula.widened) : undefined;
1210012171

1210112172
// Drop any requested PLAIN field that doesn't exist on the schema.
1210212173
// Without this, drivers (notably SqlDriver) emit `SELECT unknown_col
@@ -12233,7 +12304,9 @@ export class ObjectQL implements IObjectQLEngine {
1223312304
}
1223412305
});
1223512306

12236-
return opCtx.result as any[];
12307+
// [#22300] The formula widening is cut back off the answer here, after the
12308+
// middleware chain — every internal consumer above read the full row.
12309+
return rowsWithoutFormulaWidening(opCtx.result as any[], _findWidened);
1223712310
}
1223812311

1223912312
/**
@@ -12400,6 +12473,8 @@ export class ObjectQL implements IObjectQLEngine {
1240012473
const _findOneRequestedFields = Array.isArray(ast.fields) ? [...ast.fields] : undefined;
1240112474
const _findOneFormula = planFormulaProjection(_findOneSchema, ast.fields);
1240212475
if (_findOneFormula.projected) ast.fields = _findOneFormula.projected;
12476+
// [#22300] Same as `find`: what the widening added is cut at the `return`.
12477+
const _findOneWidened = _findOneFormula.widened ? new Set(_findOneFormula.widened) : undefined;
1240312478

1240412479
// Drop unknown PLAIN fields — see the equivalent block in `find()` for
1240512480
// the rationale, and for why this tolerance is plain-columns-only ([#7589]
@@ -12502,7 +12577,8 @@ export class ObjectQL implements IObjectQLEngine {
1250212577
return hookContext.result;
1250312578
});
1250412579

12505-
return opCtx.result;
12580+
// [#22300] Same cut as `find`, same position: after the middleware chain.
12581+
return withoutFormulaWidening(opCtx.result, _findOneWidened);
1250612582
}
1250712583

1250812584
/**

0 commit comments

Comments
 (0)