Skip to content

Commit 0728cbf

Browse files
fix(objectql): a field-narrowed search no longer matches through the companion of a field outside the search-field set (#21930)
Fixes #21880 Clause-②: no ## What changed When the optional pinyin search companion is on, a field-narrowed search no longer matches through the companion of a field outside the search-field set. - **Where.** `expandSearchToFilter` in `packages/objectql/src/search-filter.ts`, the engine's search expansion. `searchAll` is not touched. - **The gate.** The `__search` companion clause is added only when every field the companion mirrors is inside `searchFields`. That is the effective set `resolveSearchFields` already computed, after the declared/auto-default precedence and any `$searchFields` narrowing. There is one gate and no second eligibility rule for the companion. - **Where the mirrored fields come from.** `resolveSearchCompanionSources`, the same function the registry provisions the companion from and plugin-pinyin-search fills it from. It is read over the same `fields` and the same display-field pointer the engine already passes to the expansion. No spec file changed, and no new export was added. - **What the companion is (measured).** One column per object, holding the normalized form of ONE source field: the resolved display/name field (`resolveSearchCompanionSources` returns that field, or `[]`). The gate is written as "every mirrored field is in the set", never "some", because a clause over one shared column matches through every field it mirrors. Today that means "the display/name field is in the set". - **What stays the same.** A search with no narrowing keeps the clause whenever the display/name field is in the object's searchable set, so pinyin recall there is unchanged. A CJK term still skips the clause. With the companion off, nothing changes. An empty mirror list passes vacuously. The registry never provisions a companion without a source, so that case is only an author-declared `__search` column, and it keeps today's answer. ## Tests **Unit** (`packages/objectql/src/search-companion.test.ts`, new describe, 10 cases): - (a) A field-narrowed search that leaves the mirrored field out gets no companion clause. Covered: a `$searchFields` override (array and comma-separated), the narrowing carried on the term (`{ query, fields }`), a declared `searchableFields` without the field, every term of a multi-term search, and an explicit `nameField` pointer, which moves what the companion mirrors. - (b) A search with no narrowing keeps it. Covered: the auto-default set, a narrowed set that still holds the mirrored field, and a request naming no allowed field, which falls back to the full set. - (c) A CJK term still skips it, with and without narrowing. **Dogfood** (`packages/qa/dogfood/test/search-companion-field-scope.dogfood.test.ts`, 7 cases). A real kernel boots with the real `SecurityPlugin` and `PinyinSearchPlugin` (`OS_SEARCH_PINYIN_ENABLED` on), over HTTP. Every row is named in CJK, so a pinyin term matches only through the companion. - **Row-scoped object** (`sharingModel: 'private'`). The member's search answers only the member's own matching row, through `/search` and through the data door. Control: the administrator gets both rows. - **A term present only in a field hidden from the member** (`readable: false` on `name`). The member gets no hit through `/search?objects=` and none through a `searchFields: ['code']` data-door query. Controls: the field really is hidden at the data door; the administrator hits the row through the companion (no narrowing keeps it); the member hits the same row through `code`, and that hit carries nothing of the hidden field. - `@objectstack/plugin-pinyin-search` is added to the private `@objectstack/dogfood` package's dependencies. `check:test-source-alias` asked for its anchored source alias in `packages/qa/dogfood/vitest.config.ts`, because the plugin's fill path is part of the pin's subject. ## Reverse verification (one-off, nothing left in the tree) The base clause was restored on committed HEAD, first at `e7b2a3c2e2` and again at the final head `40618fa8cc`, with the same readings both times. `node scripts/ablation-replace.mjs` replaced the gate with a constant-true guard carrying the marker `__ABLATED_21880`: anchor hits went 1 to 0, blob `5a2afee37090` to `ce8eb93139c8`. - `pnpm --filter @objectstack/objectql build`, then `ablation-dist-preflight` confirmed the marker is in 4 built files of `packages/objectql/dist`. - Unit, `src/search-companion.test.ts`: **5 failed / 34 passed**. The 5 failures are exactly the (a) cases; (b) and (c) stayed green. - Dogfood file: **2 failed / 5 passed**. The 2 failures are exactly the two hidden-field "no hit" cases. The row-scoped case and every control stayed green. - Direction observed: red, as expected. - Restore: blob equal to HEAD (`5a2afee37090`) and an empty `git diff HEAD`. After rebuilding objectql, `ablation-dist-preflight --absent` reported the marker absent from all 14 built files and a clean working tree. ## Local verification (at HEAD `40618fa8cc`, after merging `origin/main` at `faf8dce482`) - `pnpm --filter @objectstack/objectql exec vitest run --project local` (the package's `test` script): **375 files, 7479 tests passed**. - Dogfood, `search-companion-field-scope` plus the neighbouring `search-skip-unreadable`: **2 files, 11 tests passed**. - `pnpm --filter @objectstack/objectql run typecheck` exit 0, which includes `check:test-typecheck` over `tsconfig.test.json`. `pnpm --filter @objectstack/dogfood run typecheck` exit 0. `--listFilesOnly` shows both new test files are in their programs. - `node scripts/pm/dispatch-gates.mjs --commands` over this branch's change set derived 78 commands. All 78 were run at this head, and `--ran` reconciled them: **78 run, 0 NOT-MEASURED, 0 UNRUN**, every exit 0. A full `turbo run build` came first, so `check:dual-build-cjs-loads` measured instead of refusing. - Artifact-roster block, the 53 commands printed outside the total: 50 exit 0. Three are NOT WIRED locally because they need PR context: `check-closing-target-claim`, `check-partof-closing-keyword` and `check-single-claim-paths`. `check-partof-closing-keyword` was then run with `PR_BODY` set to this body: exit 0. The other two are declared to CI. - Symbol-anchor sweeps: `check:adr-symbol-anchors`, `check:scripts-symbol-anchors`, `check:spec-docblock-symbol-anchors` and `check:adr-anchors` all exit 0. - Lint, narrowed to the 4 changed `.ts` files: `eslint --no-inline-config --format json` reports 4 files, 0 errors, 0 warnings, and `eslint --print-config` resolves a config for each, so none is ignored. `eslint.config.mjs` enables no type-aware linting (no `parserOptions.project`), so this diff cannot change the verdict on any untouched file. `pnpm lint` over the whole tree is CI's run. ## Acceptance notes - **Recall change on un-narrowed searches.** One case of a search with no narrowing loses the clause: an object whose effective set omits its display/name field. Examples are a declared `searchableFields` without it, or a display field of a type the auto-default does not scan (`html`, `richtext`, which are title-eligible). That follows from the ruling: the declared set says the field is not searched. Measured over `examples/` at merge base `dcb11c2ec9`: one object declares `searchableFields` (`showcase_account`), and it includes `name`; one object sets an explicit `nameField` (`todo_task.subject`), a `text` field in the auto-default. So 0 example objects change. Derived display fields of type `html` or `richtext` were NOT MEASURED (that needs a registry boot of each example). - `packages/qa/dogfood/test/search-conformance.ledger.ts` is unchanged. Its rows describe the executor and the `$searchFields` override, and neither claim moved. --- _Generated by [Claude Code](https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent f2aa0c9 commit 0728cbf

7 files changed

Lines changed: 449 additions & 3 deletions

File tree

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
'@objectstack/objectql': patch
3+
---
4+
5+
A field-narrowed `$search` no longer matches through the pinyin search companion of a field outside the search-field set (#21880).
6+
7+
Clause-②: no
8+
9+
- **What changed.** When the optional pinyin search companion is on (`OS_SEARCH_PINYIN_ENABLED`), the engine's search expansion (`expandSearchToFilter`) adds the companion clause only when every field the companion mirrors is inside the effective search-field set: the set `resolveSearchFields` computes, after any `$searchFields` narrowing. The mirrored fields are read from `resolveSearchCompanionSources`, the same function the companion is provisioned and filled from.
10+
- **What stays the same.** A search with no narrowing keeps the clause whenever the display/name field is in the object's searchable set, so pinyin recall there is unchanged. A CJK term still skips the clause. Deployments with the companion off see no change.
11+
- **Who notices.** A search narrowed to fields that leave out the display/name field, by a `$searchFields` override, by the narrowing global search applies to the fields a caller may query, or by a declared `searchableFields` that omits it, no longer matches through that field's pinyin form.

‎packages/objectql/src/search-companion.test.ts‎

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -182,6 +182,123 @@ describe('expandSearchToFilter with companion column (query-time, additive)', ()
182182
});
183183
});
184184

185+
/**
186+
* [#21880] The companion clause follows the effective search-field set.
187+
*
188+
* The companion is a normalized copy of its source fields, so it may join a
189+
* search only when every field it mirrors is inside the set
190+
* `resolveSearchFields` computed. A field-narrowed search that leaves a
191+
* mirrored field out gets no companion clause; a search with no narrowing
192+
* keeps it, so recall is unchanged there.
193+
*/
194+
describe('[#21880] the companion clause follows the effective search-field set', () => {
195+
// `crm_contact`: the companion mirrors `name` (the derived display field).
196+
const fields = provisionSearchCompanion(contact()).fields as any;
197+
const companionClause = (term: string) => ({ [SEARCH_COMPANION_FIELD]: { $contains: term } });
198+
199+
it('premise: the companion is provisioned and mirrors exactly `name`', () => {
200+
expect(fields[SEARCH_COMPANION_FIELD]).toBeDefined();
201+
expect(resolveSearchCompanionSources({ name: 'crm_contact', fields })).toEqual(['name']);
202+
});
203+
204+
describe('(a) a field-narrowed search that leaves the mirrored field out gets no companion clause', () => {
205+
it('a `$searchFields` override without the mirrored field', () => {
206+
expect(expandSearchToFilter('zhangwei', { fields, requestedFields: ['email'] })).toEqual({
207+
$or: [{ email: { $icontains: 'zhangwei' } }],
208+
});
209+
// The comma-separated spelling a URL query parameter arrives as.
210+
expect(expandSearchToFilter('zhangwei', { fields, requestedFields: 'email,notes' })).toEqual({
211+
$or: [{ email: { $icontains: 'zhangwei' } }, { notes: { $icontains: 'zhangwei' } }],
212+
});
213+
});
214+
215+
it('the narrowing carried on the search term itself (`{ query, fields }`)', () => {
216+
expect(expandSearchToFilter({ query: 'zhangwei', fields: ['notes'] }, { fields })).toEqual({
217+
$or: [{ notes: { $icontains: 'zhangwei' } }],
218+
});
219+
});
220+
221+
it('a declared `searchableFields` set without the mirrored field', () => {
222+
expect(expandSearchToFilter('zhangwei', { fields, searchableFields: ['email'] })).toEqual({
223+
$or: [{ email: { $icontains: 'zhangwei' } }],
224+
});
225+
});
226+
227+
it('every term of a multi-term search', () => {
228+
const filter = expandSearchToFilter('zhang wei', { fields, requestedFields: ['email'] });
229+
expect(filter).toEqual({
230+
$and: [
231+
{ $or: [{ email: { $icontains: 'zhang' } }] },
232+
{ $or: [{ email: { $icontains: 'wei' } }] },
233+
],
234+
});
235+
});
236+
237+
it('reads the REAL source: an explicit `nameField` pointer moves what the companion mirrors', () => {
238+
// `crm_ticket` names `subject` as its title, so the companion mirrors
239+
// `subject` — not `name`, although a `name` field exists.
240+
const ticket = provisionSearchCompanion({
241+
name: 'crm_ticket',
242+
nameField: 'subject',
243+
fields: { subject: { type: 'text' }, name: { type: 'text' } },
244+
});
245+
const ticketFields = ticket.fields as any;
246+
expect(resolveSearchCompanionSources(ticket)).toEqual(['subject']);
247+
248+
const withoutSubject = expandSearchToFilter('zhangwei', {
249+
fields: ticketFields, displayField: 'subject', requestedFields: ['name'],
250+
});
251+
expect(withoutSubject).toEqual({ $or: [{ name: { $icontains: 'zhangwei' } }] });
252+
253+
const withSubject = expandSearchToFilter('zhangwei', {
254+
fields: ticketFields, displayField: 'subject', requestedFields: ['subject'],
255+
});
256+
expect(withSubject.$or).toContainEqual(companionClause('zhangwei'));
257+
});
258+
});
259+
260+
describe('(b) a search with no narrowing keeps the companion clause (recall unchanged)', () => {
261+
it('the auto-default set, which leads with the mirrored field', () => {
262+
expect(expandSearchToFilter('ZhangWei', { fields })).toEqual({
263+
$or: [
264+
{ name: { $icontains: 'ZhangWei' } },
265+
{ email: { $icontains: 'ZhangWei' } },
266+
{ notes: { $icontains: 'ZhangWei' } },
267+
companionClause('zhangwei'),
268+
],
269+
});
270+
});
271+
272+
it('a narrowed set that still holds the mirrored field', () => {
273+
expect(expandSearchToFilter('zw', { fields, requestedFields: ['name'] })).toEqual({
274+
$or: [{ name: { $icontains: 'zw' } }, companionClause('zw')],
275+
});
276+
});
277+
278+
it('the gate reads the RESOLVED set: a request naming no allowed field falls back to the full set', () => {
279+
// `resolveSearchFields` drops unknown names and falls back to the
280+
// allowed set when none survives — so the effective set holds `name`.
281+
const filter = expandSearchToFilter('zw', { fields, requestedFields: ['no_such_field'] });
282+
expect(filter.$or).toContainEqual(companionClause('zw'));
283+
});
284+
});
285+
286+
describe('(c) a CJK term still skips the companion clause, as before', () => {
287+
it('with and without narrowing', () => {
288+
expect(expandSearchToFilter('张伟', { fields })).toEqual({
289+
$or: [
290+
{ name: { $icontains: '张伟' } },
291+
{ email: { $icontains: '张伟' } },
292+
{ notes: { $icontains: '张伟' } },
293+
],
294+
});
295+
expect(expandSearchToFilter('张伟', { fields, requestedFields: ['name'] })).toEqual({
296+
$or: [{ name: { $icontains: '张伟' } }],
297+
});
298+
});
299+
});
300+
});
301+
185302
describe('containsCJK / isCompanionMatchableTerm', () => {
186303
it('detects Han characters', () => {
187304
expect(containsCJK('张伟')).toBe(true);

‎packages/objectql/src/search-filter.ts‎

Lines changed: 57 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,13 @@
4242
* `resolveSearchFields` still returns only source fields (the companion is
4343
* invisible to `$searchFields` overrides and to clients).
4444
*
45+
* [#21880] …and bounded by the same set. The companion is a normalized copy of
46+
* named source fields, so its clause is a match on THOSE fields. It joins a
47+
* search only when every field it mirrors is inside the effective search-field
48+
* set `resolveSearchFields` computed — after any `$searchFields` narrowing — and
49+
* a set that leaves a mirrored field out leaves the companion out with it. See
50+
* {@link companionWithinSearchFields}.
51+
*
4552
* [#21009] A field the object declares MULTI-VALUED (`isMultiValueField`: a
4653
* `tags` / `multiselect` / `checkboxes` field, or a `select` / `lookup` /
4754
* `user` / … declared `multiple: true`) is matched by MEMBERSHIP, `$contains`,
@@ -68,7 +75,12 @@ import {
6875
type SearchFieldMeta,
6976
type SearchFieldResolutionOptions,
7077
} from '@objectstack/spec/data';
71-
import { SEARCH_COMPANION_FIELD, isCompanionMatchableTerm } from './search-companion.js';
78+
import {
79+
SEARCH_COMPANION_FIELD,
80+
isCompanionMatchableTerm,
81+
resolveSearchCompanionSources,
82+
type CompanionObjectMeta,
83+
} from './search-companion.js';
7284

7385
export {
7486
resolveSearchFields,
@@ -147,6 +159,44 @@ function fieldClausesForTerm(field: string, term: string, meta: SearchFieldMeta)
147159
return [{ [field]: { $icontains: term } }];
148160
}
149161

162+
/**
163+
* [#21880] May the `__search` companion clause join a search over
164+
* `searchFields`? Only when every source field the companion mirrors is in
165+
* that set.
166+
*
167+
* The mirrored fields are read from {@link resolveSearchCompanionSources} —
168+
* the one function the registry's provisioning seam and plugin-pinyin-search's
169+
* populate hook already derive the companion from — over the same `fields` and
170+
* the same display-field pointer the engine handed in. So the answer is the
171+
* companion's real source, never a second guess at it.
172+
*
173+
* ⛔ The gate is `searchFields` and nothing else: the set `resolveSearchFields`
174+
* already computed, with the declared/auto-default precedence and any
175+
* `$searchFields` narrowing applied. No second eligibility rule for the
176+
* companion is consulted here — whatever a caller's narrowing removed from the
177+
* source columns, it removes from their normalized copy too.
178+
*
179+
* The companion is ONE column holding the normalized form of its sources, so
180+
* the test is "every source is in the set", never "some source is": a clause
181+
* over the shared column matches through every field it mirrors at once.
182+
* A search whose set holds every mirrored field — any search with no
183+
* narrowing, whenever the display/name field is in the object's searchable
184+
* set — keeps the clause, so recall there is unchanged.
185+
*
186+
* An empty source list passes vacuously. The registry never provisions a
187+
* companion without a source (`provisionSearchCompanion` returns early on an
188+
* empty list), so that case is only an author-declared `__search` column —
189+
* an ordinary field the platform does not fill — and it keeps today's answer.
190+
*/
191+
function companionWithinSearchFields(searchFields: readonly string[], opts: ExpandSearchOptions): boolean {
192+
const sources = resolveSearchCompanionSources({
193+
nameField: opts.displayField,
194+
fields: opts.fields as CompanionObjectMeta['fields'],
195+
});
196+
const inSet = new Set(searchFields);
197+
return sources.every((f) => inSet.has(f));
198+
}
199+
150200
/**
151201
* Expand a `$search` term into a `{ $or: [...] }` (single term) or
152202
* `{ $and: [{ $or: [...] }, ...] }` (multi-term) filter. Returns `null` when
@@ -177,10 +227,14 @@ export function expandSearchToFilter(raw: unknown, opts: ExpandSearchOptions): a
177227
// different mechanism from the source-column clauses in
178228
// `fieldClausesForTerm`, which compare against raw stored text and therefore
179229
// need `$icontains`. Do not "align" the two.
180-
const hasCompanion = !!opts.fields[SEARCH_COMPANION_FIELD];
230+
//
231+
// [#21880] …and only when the companion mirrors no field outside
232+
// `searchFields` — see `companionWithinSearchFields`.
233+
const withCompanion = !!opts.fields[SEARCH_COMPANION_FIELD]
234+
&& companionWithinSearchFields(searchFields, opts);
181235
const andClauses = terms.map((term) => {
182236
const clauses = searchFields.flatMap((f) => fieldClausesForTerm(f, term, opts.fields[f] || {}));
183-
if (hasCompanion && isCompanionMatchableTerm(term)) {
237+
if (withCompanion && isCompanionMatchableTerm(term)) {
184238
clauses.push({ [SEARCH_COMPANION_FIELD]: { $contains: term.toLowerCase() } });
185239
}
186240
return { $or: clauses };

‎packages/qa/dogfood/package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@
2626
"@objectstack/plugin-audit": "workspace:*",
2727
"@objectstack/plugin-auth": "workspace:*",
2828
"@objectstack/plugin-email": "workspace:*",
29+
"@objectstack/plugin-pinyin-search": "workspace:*",
2930
"@objectstack/plugin-security": "workspace:*",
3031
"@objectstack/plugin-sharing": "workspace:*",
3132
"@objectstack/plugin-webhooks": "workspace:*",

0 commit comments

Comments
 (0)