fix(services): settings, datasource, webhook and messaging plumbing passes the explicit system opt-in instead of no principal - #21940
Conversation
…stead of no principal
The services-lane producers that reached the data engine with no
principal and no isSystem now pass the explicit system opt-in
({ isSystem: true }) on every engine call:
- service-settings: SettingsService.loadRows, SettingsService.upsertRow
(existence probe and insert; the update already carried it) and the
sys_setting_audit writer.
- service-datasource: the sys_metadata helpers behind runtime
datasources (loadDatasourceRows, loadDatasourceRow,
persistDatasourceRow, deleteDatasourceRow) and the sys_secret binder
(bind, unbind, resolve).
- plugin-webhooks: AutoEnqueuer's subscription refresh and the
redeliver guard's subscription read.
- service-messaging: the dispatcher claim path (SqlNotificationOutbox
claim / claimDigest / reapExpired, SqlHttpOutbox claim / reapExpired)
and the emit fan-out (writeEvent, the inbox channel's send with its
locale read and delivered receipt, PreferenceResolver.loadRows,
RecipientResolver.resolveEmail).
None of the gates the security middleware runs before its
principal-less hand-off engages on these calls, so what each one reads
and writes today is unchanged; they no longer depend on that hand-off.
Claude-Session: https://claude.ai/code/session_011K3zqE8Pv1Evw5hc8tZCnN
Co-authored-by: Claude <noreply@anthropic.com>
…rgument IDataEngine reads take their execution context in the trailing options argument, the same position the writes take theirs; the query bag stays the query. Moves the opt-in there for every IDataEngine read this change touches (the SettingsEngine and sys_secret binder surfaces keep it in their single options bag). Claude-Session: https://claude.ai/code/session_011K3zqE8Pv1Evw5hc8tZCnN Co-authored-by: Claude <noreply@anthropic.com>
One pin per package: an engine double behind the package's real call path records the context every call carries, proves the population ran (claim UPDATEs and read-backs, both settings write branches, the inbox template path, the convergence read) and asserts isSystem on each. Claude-Session: https://claude.ai/code/session_011K3zqE8Pv1Evw5hc8tZCnN Co-authored-by: Claude <noreply@anthropic.com>
…the system opt-in Claude-Session: https://claude.ai/code/session_011K3zqE8Pv1Evw5hc8tZCnN Co-authored-by: Claude <noreply@anthropic.com>
…rvices Claude-Session: https://claude.ai/code/session_011K3zqE8Pv1Evw5hc8tZCnN Co-authored-by: Claude <noreply@anthropic.com>
…; tenant-audit census re-derived - The four pins' engine doubles open findOne / update / delete with the shared metadata-core dispatch asserts, hold a find's caller bound, and implement only the verbs the moved calls use; the engine-double ledger records the new pinned coverage. - The tenant-audit census is regenerated: the write sites that now thread the explicit system opt-in moved from carrying no context to threading one, and the page's hand-written figures follow the census. Claude-Session: https://claude.ai/code/session_011K3zqE8Pv1Evw5hc8tZCnN Co-authored-by: Claude <noreply@anthropic.com>
…rvices Claude-Session: https://claude.ai/code/session_011K3zqE8Pv1Evw5hc8tZCnN Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 4 package(s): 21 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: ⛔ 4 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails. What this run could not see
Coarse fallback — 12 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 61aa64583bff4474a390478d791b6dfcb015fece && git checkout 61aa64583bff4474a390478d791b6dfcb015fece
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin f2aa0c9fad592d11f8fac4b293f459bb5ddb792b 57f738dfb124f9f6a548c41dde81dff6e4d1f1ae && git checkout -B drift-repro f2aa0c9fad592d11f8fac4b293f459bb5ddb792b && git merge --no-ff 57f738dfb124f9f6a548c41dde81dff6e4d1f1ae
node scripts/docs-audit/affected-docs.mjs --json f2aa0c9fad592d11f8fac4b293f459bb5ddb792b
|
|
CI note from
Generated by Claude Code |
…er the system opt-in The engine skips its referential-integrity check for an isSystem write, so the fan-out writes (sys_notification and sys_inbox_message actor_id), the setting-audit write (sys_setting_audit actor_id) and SettingsService's user-scope insert (sys_setting user_id) now check the reference first: one sys_user read under the opt-in, refusing an unknown id with the engine's own answer (VALIDATION_FAILED, one reference_not_found finding, the same message) and failing open when the read cannot run, as the engine's probe does. A write that names no user is unchanged. Differential pins hold each producer's refusal equal to the engine's answer to the context-less write it made before the opt-in. The changeset moves to Clause-② yes (widening): SettingsEngine find/insert and SecretStoreEngineLike delete gained an optional context, so service-settings and service-datasource take a minor. Claude-Session: https://claude.ai/code/session_011K3zqE8Pv1Evw5hc8tZCnN Co-authored-by: Claude <noreply@anthropic.com>
…rvices Brings in the dogfood per-file cwd fix the check:pm-dispatch-gates self-test reads. No file of this branch is touched by the two incoming commits. Claude-Session: https://claude.ai/code/session_011K3zqE8Pv1Evw5hc8tZCnN Co-authored-by: Claude <noreply@anthropic.com>
…dationFailure constructor The producer-side refusal assigned its code in each package, which made both packages new stamp sites of a registered code with no provenance row. It is now built by `validationFailure` from `@objectstack/types` (already a runtime dependency of both), the constructor that already holds that provenance. The envelope is unchanged: same name, code, message and findings. The settings pin's find double applies the caller's bound before copying rows, and the package's vitest config aliases `platform-objects/identity` to source. Claude-Session: https://claude.ai/code/session_011K3zqE8Pv1Evw5hc8tZCnN Co-authored-by: Claude <noreply@anthropic.com>
Contract reviewServed-tier: ① Derived judgmentsJudged on the net diff against
Gate verdicts, from the check-runs on this head (latest run per check name): all 34 check names on this head have concluded, and every check that ran is ② Semver level
③ Boundary flagsRound 1 (the dev's first report and the seat's verdict on it):
Round 2 (the dev's patch-round report):
Carried, not this PR's, and recorded where they belong: producers in these packages the card does not name (census rows 24 onward on the closure card; Implemented-by: VERDICT: PASS |
Fixes #21913
Clause-②: yes (widening)
This is a slice of #21908: the services-lane producers. #21908 stays open, because it builds the deny itself, last.
What changes
Every engine call in the card's named functions now passes the explicit system opt-in that exists today:
{ isSystem: true }on the call's context. These calls used to reach the data engine with no context at all, so they had no principal and no opt-in. They got past the security middleware only through its principal-less hand-off (ADR-0096 E1), which #21908 retires. This PR adds no new elevation API, changes nothing any door authorizes, and does not build the deny.SettingsService.loadRowsfindonsys_settingSettingsService.upsertRowfindandinsertonsys_setting(itsupdatealready had the opt-in)buildSettingAuditWriterwriteinsertonsys_setting_auditloadDatasourceRows,loadDatasourceRowfind/findOneonsys_metadatapersistDatasourceRow,deleteDatasourceRowfindOne+insert/update/deleteonsys_metadatabind/unbind/resolveinsert/delete/findonsys_secretAutoEnqueuer.doRefreshfindonsys_webhookcreateWebhookRedeliverGuardfindOneonsys_webhookSqlNotificationOutbox.claim/claimDigest/reapExpiredfind, claimingupdate, read-backfind; the reapupdateSqlHttpOutbox.claim/reapExpiredfind, claimingupdate, read-backfind; the reapupdateMessagingService.writeEventinsertonsys_notificationsend+writeDeliveredReceiptinsertonsys_inbox_message, the recipient-localefindOneonsys_user(a helper onlysendcalls),insertonsys_notification_receiptPreferenceResolver.loadRowsfinds onsys_notification_preferenceRecipientResolver.resolveEmailfindOneonsys_userIDataEngine reads pass the opt-in in the trailing options argument, which is where the contract puts a read's context. Two package-local surfaces have a single options bag, and the opt-in goes there:
SettingsEngine, and thesys_secretbinder's engine slice.SettingsEngine.findand.insertandSecretStoreEngineLike.deletenow declare thecontextthey receive. No symbol is new on any package entry. The shared constants (FAN_OUT_SYSTEM_CONTEXT,DISPATCHER_SYSTEM_CONTEXT) live in package-internal modules.Rows 15 and 16 are not in this slice and wait for the maintainer.
Measurement
Instrument (H2). A local, uncommitted instrument sat at the security middleware. It recorded each principal-less, non-system context that reached the hand-off, with its stack. It recorded whether any of the six gates before the hand-off threw on such a call, and which of them matched the call's object and verb. It also recorded the outcome after
next(): the result type, row count, key set, a hash of the non-volatile values, or the error code. In the AFTER leg it recorded the same outcome for eachisSystemcall whose stack ran through these four packages. Both legs covered the whole dogfood suite (206 files, 1590 tests passed, 9 skipped, identical in both legs) and a booted showcase dev composition. The boot covered seed-admin, a settings read plus two writes, a runtime datasource create / patch / read / delete, and admin and anonymous requests, then sat idle for 65 seconds so the dispatchers and the webhook refresh ticked. The instrument was then reverted, and the file's blob equals HEAD (5b4ab28045af). The plugin-security dist was rebuilt clean:ablation-dist-preflight --absentpasses, and the marker had 3 hits in the instrumented dist.Before and after, per function. Columns: principal-less records BEFORE, principal-less records AFTER, and
isSystemrecords AFTER.SettingsService.loadRowsSettingsService.upsertRow(probe + insert)writeloadDatasourceRowspersistDatasourceRowdeleteDatasourceRowAutoEnqueuer.doRefreshSqlNotificationOutbox.claimSqlNotificationOutbox.claimDigestSqlNotificationOutbox.reapExpiredSqlHttpOutbox.claimSqlHttpOutbox.reapExpiredMessagingService.writeEventsend(row insert)writeDeliveredReceiptPreferenceResolver.loadRowsRecipientResolver.resolveEmailPrincipal-less totals moved 35245 → 33077 in dogfood (Δ 2168) and 422 → 183 at boot (Δ 239). Each delta is exactly the sum of the rows above. No principal-less record attributed to any moved function remains. The hand-off still sees row 15 and every other lane's producers.
No run reached these, so each is held by its unit pin instead:
loadDatasourceRow, the secret binder (this repo wires it into no composition), the redeliver guard, the inbox recipient-locale read (template path), and the claim path'supdateand read-back (no pending rows in any run).Gates before the hand-off (Zone 1). Across 35245 dogfood and 422 boot principal-less records, the "gate threw" record fired 0 times. The package-managed, system-row, curated-capability and audience-anchor gates never matched an object or verb these producers touch. Neither did the delegated-administration gate. The engine-owned guard matched the bucket on the writes to engine-owned objects. On a context with no user id, its own
isUserContextWritepredicate returns before it can refuse. No producer is held back.What each call answers is unchanged. Per function, call counts per object and verb are equal before and after. So are the outcome shapes (result type, row count, key set). There were 0 errors in either leg. Content hashes are equal for 11 of 14 functions in dogfood and 7 of 9 at boot. The rest differ only on values that change every run: the receipt's
attimestamp (all 8), and inbox and notification payloads that carry a per-run record id or date (2 of 8 and 3 of 8, from the approval and sweep tests). At boot, the probe's own per-phase file path sits in the stored datasource record. The plugin-audit rows these writes produce (sys_audit_log,sys_activity) are written in equal numbers before and after.H6,
loadRows. The call count is the same (2090 + 42), and the returned settings have equal hashes on every call. The opt-in adds one frozen context object. The middleware now exits at its system short-circuit instead of running the six gates and the hand-off. No wall-clock figure is quoted, because the container is shared.H7, reads on another principal's behalf. What these reads return (a user id for an address, a locale, preference rows) is consumed inside the fan-out.
emit()answers the notification id, counts and per-delivery outcomes. Its three in-repo callers (approvals, the flow notify node and comment mentions) relay counts and the id only. The opt-in changes none of this, because the principal-less read returned the same rows.One engine branch keyed on the flag stops running on these writes. It is row 23 of the
isSystemcensus page: the dangling-reference check is skipped for anisSystemwrite. Before the move, it ran 10 times nested under these producers (writeEvent2, inboxsend2, setting-auditwrite6), on theactor_idlookups, and resolved every time. After the move it does not run. A local probe (real ObjectQL and SQLite, deleted after the run) showed what that means for anactor_idthat names no user. With no context, today's path refuses withVALIDATION_FAILED("Actor: no sys_user record has id …"). UnderisSystemthe row is written. A real user is written both ways. Thatactor_idcomes fromemit()'sactorId, which a flow notify node can author. So the behaviour on measured traffic is unchanged, and a latent difference remains for anactorIdthat names no user. The Acceptance notes carry it.H4 pins and ablations. There is one pin per package. The engine double sits behind the package's real call path, proves the population ran, and asserts
isSystemon every call. Each pin was ablated by dropping the opt-in throughscripts/ablation-replace.mjs(the anchor must hit). Seven legs ran: settingsloadRows, the fan-out constant, the dispatcher constant, the datasourcesys_metadataconstant, the secret-binder constant, and the two webhook constants. Every leg went red under the mutation, and the failure names the call, for example "find on sys_setting: expected undefined to deeply equal { isSystem: true }". Every leg was restored with blob equal to HEAD and an emptygit diff HEAD, and went green again. The pins are package-local, imported fromsrcwith no dist in the path.Census pages (H3). The
isSystemcensus (check-system-context-census) is OK, and--fixchanged nothing: this change adds no elevation read site. The tenant-audit census did move, because the write sites now thread a context. It was regenerated withtenant-audit-census.mjs --write. On its page, the hand-written figures follow the census: the provable no-context, tenancy-enabled count went 9 → 2, unreadable 67 → 60, decidably elevated 114 → 121.Serial (H5).
origin/mainwas merged twice. It now includes #21906's squash, and the merge was clean. Agit merge-treeagainst #21877's head (5c405846, now closed as a draft) is clean. This PR edits neither PR's region:datasource-admin-plugin.tsanddatasource-secret-binder.tsonly, inservice-datasource.Tests
10e77fef6f, after mergingorigin/mainfaf8dce482. The next merge (9dce635337, which brings this PR to62960ffa1a) touches no file in these four packages. Typecheck of the four packages: exit 0.main.--no-inline-config --format json: 19 files, 0 errors, 0 warnings. Those files are inside the config's ownpackages/**/*.{ts,…}population, and the config enables no type-aware linting, so this diff cannot move a verdict on any untouched file. The fullpnpm lintrun belongs to CI.62960ffa1a, the head this PR opens with, every one of the 105 commandsdispatch-gates --commands --repo objectstack-ai/objectstackderives exited 0.dispatch-gates --ranreports: "105 derived famil(ies) accounted for — 105 run, 0 NOT-MEASURED". In an earlier pass, four of these went red on this branch, and they are now fixed.check:tenant-audit-censusneeded the census regenerated.check:engine-double-contractandcheck:objectql-double-limitneeded the pin doubles routed through the shared dispatch asserts and holding a find's bound, with the ledger recording the new pinned coverage.check:dual-build-cjs-loadsneeded eight unrelated packages built first.Acceptance notes
sys_secretstore the plugin builds (insert/get/update), andSettingsService.readStoredHandle.SqlNotificationOutboxandSqlHttpOutboxenqueue,ackandlist; the email and SMS channels' recipient reads;RecipientResolver.resolveRole/resolveTeam/resolveOwnerOf; the emit dedup lookup; the template renderer's read.resolveOwnerOfreads a business object, and its posture is not neutral. Today the sharing middleware answers a principal-less read of aprivateobject with a deny-all filter, so anowner_of:recipient on such an object resolves to nobody. Under the opt-in, that filter would be bypassed.unreadNotificationIds,upsertReadReceipt,notificationOrganization).actor_idofsys_notification,sys_inbox_messageandsys_setting_audit.Seat's append: patch round 1 at
57f738dfb1(written bydomain:servicesseat 1 from the dev's report6009307655; the dev does not edit this body)What changed in the patch round (seat verdict
6008259054). The sections above describe62960ffa1a; where they differ, this append is current.Clause-②: yes (widening). The exportedSettingsEngine(find,insert) andSecretStoreEngineLike(delete) gain an optionalcontext, so@objectstack/service-settingsand@objectstack/service-datasourcetake aminor.service-messagingandplugin-webhooksstaypatch. Line 2 above, the changeset and the claim (6003840075) moved together. Nothing accepted or refused at any door changes.isSystemwrite and has no option to keep it. So each producer that writes a user reference does one guardedsys_userread by id under the opt-in, then refuses an unknown id with the engine's own answer:VALIDATION_FAILED, onereference_not_foundfinding, and the same message.actor_idofsys_notification(writeEvent), ofsys_inbox_message(the inbox send) and ofsys_setting_audit(the setting-audit writer), and theuser_idof a user-scopesys_settingrow onSettingsService.upsertRow's insert. The last is the same difference, which this PR's opt-in introduced on that insert.validationFailurefrom@objectstack/types, already a runtime dependency of both packages, so neither package stamps the code itself andcheck:error-code-provenanceis green with no spec row and no waiver. It is shape-identical to the engine's refusal but notinstanceofobjectql'sValidationError; the callers on these paths read the message or the code, and every door maps the shape to400 VALIDATION_FAILED.sys_userread per write that names a user.a3c2209a68).check:pm-dispatch-gatesexits 0.57f738dfb1: 105 derived, 105 run, all exit 0. The 54 roster families: 51 exit 0, and 3 are NOT WIRED locally (they need a pull-request context; CI runs them).service-settings/vitest.config.tsgains one anchored alias (platform-objects/identity→src) for the new pin, whichcheck:test-source-aliasasks for.Carried, not filed here:
resolveOwnerOfis not neutral to move.Generated by Claude Code