fix(spec)!: credential-shaped datasource config values are refused at write and redacted on every read door for drivers with no shipped contract - #21877
Conversation
…ction Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-Authored-By: Claude <noreply@anthropic.com>
…epted Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-Authored-By: Claude <noreply@anthropic.com>
…edential exports Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-Authored-By: Claude <noreply@anthropic.com>
…/0062 on the write door Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-Authored-By: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 3 package(s): 1 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
What this run could not see
Coarse fallback — 139 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 1f565b01adcc7544dc38bc3e5b13499ba2f34e52 && git checkout 1f565b01adcc7544dc38bc3e5b13499ba2f34e52
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin cab639671528ef6f3a201e8995794378a4a28bfe 5c405846a58ec2edb6299e9cf60c9a2b277596b5 && git checkout -B drift-repro cab639671528ef6f3a201e8995794378a4a28bfe && git merge --no-ff 5c405846a58ec2edb6299e9cf60c9a2b277596b5
node scripts/docs-audit/affected-docs.mjs --json cab639671528ef6f3a201e8995794378a4a28bfe
|
Contract reviewServed-tier: Inputs read: card #21840 body and all four comments (triage 5990389094, claim 5990748393, os-dev-report 5995346311, claim correction 5995372304); PR #21877 body, its 11-file list, and the net diff ① Derived judgments
② Semver levelClause-②: yes (narrowing)
③ Boundary flags
Implemented-by: VERDICT: PASS |
Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-Authored-By: Claude <noreply@anthropic.com>
…airs, more spellings and connection-string forms Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-Authored-By: Claude <noreply@anthropic.com>
…dentials holding ; = : @, libpq ; values, header tuples; whole-word one-word keys Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-authored-by: Claude <noreply@anthropic.com>
…tasource-secret-keys
…y identity Round-3 review fixes for the contractless-driver credential walk: - the key judgment is linear (the capital-run split looks ahead instead of capturing the run) and bounded: a key, segment key, parameter or header name longer than 256 characters after NFKC, or holding a non-ASCII character, is judged credential-shaped unread; - libpq pairs are found leniently, one pass; - header shapes: Name: value lines, raw-headers lists, a tuple under a header-ish key, tuples longer than two, every pair label; - key names: ssl/tls key material, privkey, the data/content/hex/string/ str/raw/hash qualifiers, basicauth; - embedded shapes: token-shaped URL usernames, sig and X-Amz-Signature parameters, JSON-encoded strings, form-encoded strings, URL fragments; - a descriptor key exempts leaves only; bytes are one value; Map and Set are judged whole; a bare key needs a secret-looking value or a credential-shaped or header-ish holder; - restoreRedactedConfig and the /meta carry-forward resolve every array hop by identity (unchanged array, else a unique served projection) and drop a withheld value whose element changed, is gone or is ambiguous; the /meta path now follows id-less elements and nested arrays. Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-authored-by: Claude <noreply@anthropic.com>
…e read path still withholds it restoreRedactedConfig re-redacts the merged config and keeps only the grafts whose landing path is still withheld, repeating until nothing more drops (the /meta carry-forward's loop). An untouched Save skips the second walk: the merged config is the stored one. Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-authored-by: Claude <noreply@anthropic.com>
… keys by words; prose and SQL stay accepted - PEM private-key armour is secret material in any string (and in bytes); a bare `key` under an ssl / tls / cert* holder is key material; `pfx`, `pkcs12` and `p12` are credential words. - A non-ASCII key is credential-shaped only when its non-ASCII text sits inside or next to a credential word (confusable letters, format characters, stand-ins, adjacency, credential words of other scripts); otherwise a non-ASCII run is a word of its own. - A single-line `Name: value` string is read as a header line only under a header-ish key; SQL bind placeholders are not libpq or segment values; opaque URI schemes and times of day are not scheme-less userinfo. - A string or bytes longer than 64 KiB is judged credential material unread. - A bare `key` also counts hex and digit-free base64 key material. Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-authored-by: Claude <noreply@anthropic.com>
…und-4 exports Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-authored-by: Claude <noreply@anthropic.com>
… serves holds no finding A bare `key` under a header-ish holder is the header named `key` only in a header map, not in a list element (a pair's label). The contractless read projection is judged again until it holds no finding, so an untouched Save of what was served passes the write door. The dogfood edit-door case and the service tests use a plain-keyed `key:` pair. Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-authored-by: Claude <noreply@anthropic.com>
|
Closing without merging, per the maintainer's ruling on #21921 (comment 6007092527 and the correction after it). The platform does not guess which values in a plugin driver's Generated by Claude Code |
Fixes #21840
Clause-②: yes (narrowing)
What changed
Credential material in the
configof a datasource whose driver the platform ships no config contract for (a plugin-contributed driver such ascom.vendor.warehouse) is now refused at publish and withheld on every read door. A driver with a shipped contract (postgres,mysql,mongodb,turso,sqlite,sqlite-wasm,memoryand their aliases) is judged exactly as before, at both doors.packages/spec/src/data/driver/contractless-credentials.ts):findContractlessCredentialswalks a contractless driver'sconfigand reports every credential position, with itsconfig.<path>. The write door refuses each finding and the read door withholds each finding, so the two doors cannot disagree.isContractlessDriverdecides which drivers it applies to: those with no registered config schema.isCredentialShapedConfigKey): a key is NFKC-normalised, then judged on its whole name, split into words at separators and camel-case boundaries, case-insensitively. It matches whole words, never substrings. A key longer than 256 characters is credential-shaped without being read. A key whose last word is a locator, identifier or descriptor (credentialsRef,accessKeyId,tokenUrl,passwordFile,secretsManagerRegionand the like) is never credential-shaped, nor is a multi-word key that starts with a flag or count word (useDefaultCredentials,maxTokens). Otherwise it is credential-shaped when a word ispassword,passwd,passphrase,secretorcredential, when its last word is a token-type stem (token,pwd,jwt,cookie,auth,bearer,apikey,pfx,pkcs12,p12and others), or when it names key material (apiKey,privateKey,sslKey,serviceAccountKeyand others). A bareaccessKey,primaryKeyandpartitionKeystay accepted. A one-word key with no boundary left (APIKEY,dbpassword) is judged on its folded spelling by the same rules. The changeset has the full word lists.密码,パスワード,парольand others in the changeset); it reads as credential-shaped once invisible format characters are removed and Cyrillic and Greek look-alike letters are read as Latin; an ASCII credential word touches a non-ASCII character (password密码); or a credential word of four or more letters has at most one non-ASCII stand-in or insertion per four letters (tok€n). Otherwise a run of non-ASCII characters is a word of its own, so客户名称,Größeandcaféare accepted. The same rule judges query, form and connection-string segment parameter names.key(orkeys) in an object is credential material only in these places: inside a credential-shaped holder; directly in a header-ish holder's map (headers: { key }is the header namedkey), but not in an element of a list under it, wherekeyis a pair's label (headers: [{ key: 'Authorization', value }]); inside a TLS holder (a key with a wordssl,tls,mtls,x509,pfx,pkcs12, or a word starting withcert:ssl: { key, cert, ca }); or when its value looks like key material. Key material here means a secret-looking string (looksLikeSecretValue), 16 or more characters of hexadecimal holding a letter, digit-free base64 whose upper and lower case alternate like random text, or bytes. Camel-case names and paths stay names, so{ key: 'email' }and{ key: 'customerEmailAddress' }are accepted.packages/spec/src/data/datasource.zod.ts):DatasourceSchemaadds onecustomissue per finding, at the value's ownconfig.<path>(array elements included, such asconfig.servers.0.password), naming the remedy: remove the inline credential and bind it as the datasource's secret (the connection form's secret field, orexternal.credentialsRef). Every door that parses the schema is covered:defineStack({ datasources }),PUT /api/v1/meta/datasource/:name, and Setup → Datasources create and update. The connection test answersok: false. What is refused:credentials: {…},auth: {…}), except leaves whose last word is a descriptor or an identity (user,clientId,host,scopeand the like), socredentials: { type, clientId }is accepted whole. The exemption covers leaves only; an object below such a key is still judged as credential context;valueof a pair object whose label (name,key,headerorheaderName) is credential-shaped, and the value of a[name, value]tuple or a flat raw-headers list whose name is credential-shaped (anAuthorization,X-API-KeyorCookieheader);;key=valuetail property; the Oracle thin-driver userinfo; a credential segment of a semicolon-delimited connection string (quoted values honoured); a credential keyword of a libpq keyword/value string; a scheme-lessuser:password@hostuserinfo; a JSON-encoded object or array, walked by the same rules; PEM private-key armour (-----BEGIN … PRIVATE KEY-----, includingRSA,EC,DSA,ENCRYPTED,OPENSSHandPGP PRIVATE KEY BLOCK), in a string or in bytes; and aName: valueheader line with a credential-shaped name. A header line is read on any line of a multi-line string, but a single-line string is read as a header line only under a header-ish key, so a one-linedescription: 'Password: …'is prose;$1,?,:name), as inWHERE token = $1; the part after an opaque URI scheme (mailto:,sip:,sips:,tel:,urn:,xmpp:,news:,im:,pres:), which is read on its own instead of asuser:password@host; and a time of day before an@(12:30@);MAX_JUDGED_STRING_LENGTH), which is judged credential material without being read;CONTRACTLESS_CREDENTIAL_WALK_DEPTH), and aMaporSetanywhere, which cannot be judged and are not accepted unjudged.${API_KEY}), plain array data with no credential-shaped key, and every other key. The config shape itself stays unjudged.packages/spec/src/data/datasource-credential-redaction.ts): the one redactor,redactDatasourceConfig, now withholds every position the same walk reports for a contractless driver. A credential value is dropped (inside an array element too, without shifting its siblings), a credential embedded in a string is removed from it (redactEmbeddedCredentials), and a PEM private-key block is removed from its string. The served projection is then judged again, and redacted again, until it holds no finding, so an untouched Save of what was served passes the write door. A projection that has not settled after 8 passes is served empty. Every read exit already routes through this redactor:/api/v1/meta/datasource(item, list,/published,/layers, history),/api/v1/datasources(item and list), the data door oversys_metadata/sys_metadata_history, and the audit ledger's and activity feed's copies. No second redaction helper is added. Audit copies written before this release are projected through the same redactor byos migrate audit-metadata-bodies --apply.packages/services/service-datasource/src/datasource-config-redaction.ts):restoreRedactedConfigresolves each array hop by identity: the same index in an array left as served, otherwise the one element equal to the served projection, unique on both sides. A value whose element changed, is gone or is ambiguous is dropped, never carried onto another element. It then redacts the grafted config again and keeps a graft only where the read path would still withhold it, repeating until nothing more drops. So a pair's value beside a label renamed to a non-credential name, a deleted label, or a renamedkey:label is dropped. An untouched Save, where the patch equals the served projection, skips this second walk./metacarry-forward (packages/metadata-protocol/src/metadata-redaction.ts): thePUT /api/v1/meta/datasource/:namecarry-forward applies the same identity rule to id-less array elements and nested arrays, which it used to skip, so an untouched GET-then-PUT of a legacy row no longer drops values.packages/services/service-datasource/src/datasource-credential-migration.ts): for a contractless row, every top-level key that holds a finding of the same walk is reported as residue and refused with the remedy, instead ofnothing-to-migrate.@objectstack/spec/dataexports:isCredentialShapedConfigKey,embeddedCredentialOf,redactEmbeddedCredentials,connectionStringCredentialKeys,findContractlessCredentials(withContractlessCredentialFinding, whoseembeddedfindings under a header-ish key carryheaderish: true, andCONTRACTLESS_CREDENTIAL_WALK_DEPTH),withholdContractlessCredentials,looksLikeSecretValue,MAX_JUDGED_STRING_LENGTH, theEmbeddedCredentialOptionstype (itsheaderishoption is taken byembeddedCredentialOfandredactEmbeddedCredentials), andisContractlessDriver.api-surface/andexport-origins/regenerated.scripts/adr-anchors/packages__spec__src__data__datasource.zod.ts.json(ADR-0015, ADR-0062). The changeset.changeset/21840-contractless-datasource-credentials.mdstates the breaking change, the full refused and accepted lists, and the remedy.Why refuse at write rather than route into the secret store
ADR-0015 section 10: credentials never appear in metadata artefacts. ADR-0062 D3: the one route a credential has into a driver is the bound secret (
external.credentialsRef), resolved at connect and handed to the driver factory. For a contractless driver nothing says which key its factory reads its credential from, so silently movingconfig.apiKeyinto the single bound secret would hand the factory a secret it may never read and strip the key it does. That is the same reason the credential-migration planner already refuses to re-home such a row. Refusing is loud and names the working remedy; known drivers already refuse inline credentials the same way.Tests
Rounds 3 and 4 are described in the os-dev-reports on #21840. Round 3 (head
2320178c9d, comment 6002380150) added a linear key split with a 256-character cap on judged names, identity-based carry-forward, a lenient one-pass libpq scan, more header shapes and key spellings, embedded credentials in URL usernames, signed-URL signatures, JSON-, form- and fragment-encoded strings, leaf-only descriptor exemption, byte and Map/Set values, and a barekeyjudged only in context or by its value. Round 4 (head5c405846a5, comment 6004802884) added the re-judgment fixed point in the edit round trip, PEM private-key armour and TLS holders, the non-ASCII key rule, the header-line, SQL-placeholder, opaque-scheme and time-of-day readings, the 64 KiB unread cap, hex and base64 bare-key values, and a served projection judged again until clean. The figures below are from round 4.All runs are against the tree pushed as HEAD
5c405846a5. The tests import the subject fromsrc.packages/spec/src/data/datasource-contractless-credentials.test.ts: 434 tests, covering the key judgment in both directions, embedded credentials in strings, the write door and the read door per shape, and known-driver behaviour unchanged. Round-4 cases in both directions: PEM per armour type, TLS holders,pfx, the accepted and refused non-ASCII sets, parameter names, prose, SQL,mailto,sip,tel,urnand time-of-day values accepted with controls still refused, the 64 KiB cap with an at-cap control, hex and base64 bare-key values with name controls, bounded-time cases, and cases proving the served projection is accepted by the write door. Whole spec project: 619 files / 18884 tests passed (1 todo).packages/services/service-datasource/src/__tests__/datasource-contractless-credentials.test.ts: create and update refused per shape with nothing persisted and no secret minted;getDatasourceandlistDatasourceswithhold a legacy row's credentials; the untouched round trip carries them forward; the planner names the residue. Six new carry-forward tests: label renamed, label deleted,key:label renamed, renamed to another credential name (keeps the value), the untouched-Save control, and a re-read through the service. Whole package: 40 files / 762 tests passed.packages/qa/dogfood/test/datasource-contractless-credentials.dogfood.test.ts: the showcase composition with the admin routes and the audit writer mounted asos servemounts them, across a cold boot on one database file. Both write doors refuse and nothing is stored; the clean datasource saves; a row seeded at rest with the pre-refusal body is served by every read door without its credentials, with a positive-control marker present; the audit ledger's copy carries none. New case 4 drives the admin PATCH edit door on a seeded legacy row with renamed labels: the stored row and two read doors carry no withheld value, and the positive control (the edit reached the stored row) holds. Whole dogfood package in 4 batches: 204 files passed, 1 skipped (477 + 401 + 384 + 317 tests passed).metadata-protocol217 files passed, 3 skipped / 27925 tests passed, 19 skipped.plugin-audit39 files / 630 tests passed.scripts/ablation-replace.mjs, each anchor verified 1 to 0 on disk, each restore proven by blob equal to HEAD and an emptygit diff HEAD: (1) making the edit round trip keep every graft turned 4 service tests red (label renamed, label deleted,key:renamed, through the service), while the control and the renamed-to-credential case stayed green; (2) dropping the TLS-holder clause turned 1 test red (a PEM underssl.keyis still caught by the armour reading); (3) removing PEM detection turned 7 tests red. The restore runs re-ran 434/434 and 30/30 green. Two earlier attempts that never measured anything (one refused by the tool, one mangled by shell quoting) were discarded.@objectstack/spec(including scripts and the test layer),@objectstack/service-datasource,metadata-protocol,plugin-audit, dogfood.node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --ran: 99 derived, 98 run with exit 0, 1 NOT MEASURED.check:generatedgreen after regeneratingapi-surface/andexport-origins/.check:type-check-debttimed out at the 300 s per-gate budget and passed on a rerun (79/80 packages, debt unchanged).NOT MEASURED:
check:dual-build-cjs-loads, reason: PREREQUISITE NOT MET (exit 3; it needs a full repo build). Declared to CI.The branch was at least 3 commits behind
origin/mainwhen the gates ran; main was not merged in this round. This diff touches no fake engine (scripts/engine-double-contract.pinned.jsonchanged on main).Acceptance notes
configcarries the withheld value forward and is then refused; sending that key as an empty string clears it. A metadata-door save of the served body is not affected (its gate runs before the carry-forward).${API_KEY}); a lower-case${…}is judged as written. The platform's own environment names are upper-case.user:x@host-shaped value that is not a credential is refused; a key such assecretsManagerholding a provider name is judged credential-shaped; and credential words of other scripts match as substrings inside a key.:wordis read as a SQL bind placeholder, not a credential.keylabel, because therekeyis read as the header namedkey.Generated by Claude Code