Skip to content

Research: pressure-test released Decision Engine v1.0.0 across domains - #71

Draft
camerontjs-dot wants to merge 25 commits into
mainfrom
research/v1-pressure-test-rc0-20260911
Draft

camerontjs-dot wants to merge 25 commits into
mainfrom
research/v1-pressure-test-rc0-20260911

Conversation

@camerontjs-dot

Copy link
Copy Markdown
Owner

Classification

Draft Research Infrastructure / post-release pressure campaign. Keep Draft regardless of outcome.

No maintained Decision Engine runtime, release metadata, Contract C/D authority, Authorization, execution, merge, tag, release, or production-default change is authorized.

Exact subject

The test subject is the published Decision Engine v1.0.0 release artifact, not the research branch checkout.

  • release commit: 7be709b2141c767c5da89b8b94cf90233c4238fe
  • release tree: 4e92703e781024de62c77f2f14e9164aceb16815
  • annotated tag: v1.0.0
  • release source archive SHA-256: c440d5de2a3853db9f4ccc2ba37d273e29ce8d9e71e0d646fc0c20eee0cd5902

The workflow downloads that exact published archive, verifies its digest, extracts it, and executes the pressure harness against the extracted release object. It separately proves that this research branch does not modify maintained src/**, scripts/**, or tests/**.

Real producer control

The campaign also checks out exact Claim Audit Lab v0.5.0:

  • release commit: 5533bbcf27a3ee3a7d901f7dfc44c241bc558e2c
  • tag object: 193c07178a8d891231746fca97c8c1bef0e9dc3c

CAL v0.5.0 regenerates a fresh Contract C 1.0.0 object from its released locked Contract-B fixture. This is the real released-producer control.

Pressure surface

RC0 preregisters:

  • both maintained v1 policies across CLEAR / HOLD / FAILED and invalid-policy/authority/binding surfaces;
  • MainFrame knowledge-synthesis-shaped cases;
  • SOP / controlled-requirement-shaped cases;
  • source-literature supported and contradicted cases;
  • regulated-quality unsupported/overstated cases;
  • missing-source credential and ambiguous/not-checkable cases;
  • causal-basis versus residual citation discrimination;
  • cross-case claim/citation replay;
  • stale same-ID immutable-content replay;
  • caller evaluator/registry substitution;
  • exact repeat determinism and context-key-order invariance;
  • CLI/library parity;
  • exact downstream Contract D applicability and Authorization firewall;
  • an explicit capability-gap map for desired decisions v1.0.0 does not maintain.

Consumer-constructed domain cases must first pass exact released Contract C validation. They are explicitly not claimed as current CAL producer reachability or real-world semantic gold.

Capability questions under test

Rather than forcing existing policies to impersonate broader product behavior, RC0 records absence as a capability gap for questions such as:

  • MainFrame synthesis-note lifecycle promotion;
  • SOP/controlled-document approval;
  • regulated publication/use approval;
  • typed human-review requirements;
  • evidence remediation requests;
  • task dispatch;
  • release qualification approval;
  • aggregate multi-claim approval;
  • conditional approval;
  • explicit reject/remove/retract effects.

Prior evidence deliberately not rediscovered

Earlier Decision Engine research already established real CAL supported/contradicted/not-checkable behavior, mixed evidence handling, execution-state precedence, producer-identity scope, the Contract-B internal-reference aperture, B-index strict-ingress prototype, maintained mutation testing, heterogeneous research domains, evaluator-substitution falsification, and immediate bound materialization.

RC0 uses those as known boundaries and concentrates on the released artifact, broader domain-shaped consumption, cross-object composition pressure, and capability coverage.

Falsifier

Any cross-case/stale/misbound object acquiring CLEAR, any HOLD/FAILED result becoming candidate_for_authorization, unknown-policy fallthrough, caller implementation substitution, nondeterminism, CLI/library disagreement, invalid authority acceptance, or unexplained metamorphic sensitivity/invariance weakens or falsifies the released surface.

Evidence record

Preregistration:

research/v1-pressure-test-rc0/README.md

No interpretation is final until the hosted run and resulting artifact are inspected.

Copy link
Copy Markdown
Owner Author

RC0 first hosted cut preserved as an evaluator/harness deviation.

Run 34649785243 completed successfully on research head 40c387f5cdfd10c0eeca82a47eb2ddf619c16c5b; artifact 10283680332, digest sha256:ea9eaf35378e3dfb81919d430f195a49d976529fc8bc8ad0f5d07ee56fae8d94.

Observed substantive gates passed: exact published v1.0.0 archive verification, unchanged released runtime proof, exact C/D/CAL authority checks, fresh CAL v0.5.0 Contract C generation, released-artifact domain/adversarial matrix, downstream Contract D consumer step, source mutation guard, artifact upload.

However artifact inspection found a harness coverage defect: downstream-contract-d-receipt.json asserted failed_does_not_escalate=true while its observed outcome counts were candidate_for_authorization=13, hold=7, evaluation_failed=0. The Node pressure harness did exercise cross-case FAILED Decisions, but those FAILED Decisions were canonicalized only and were not added to the downstream decision index. Therefore that specific downstream FAILED claim was not evidenced by run 34649785243.

Classification: EVALUATOR_OR_HARNESS_DEFECT, not a Decision Engine runtime defect.

Disposition: preserve this run; correct only the missing FAILED-to-downstream coverage and rerun. No maintained runtime change, Contract mutation, Authorization, execution, promotion, merge, tag, or release.

Copy link
Copy Markdown
Owner Author

RC0 second hosted cut preserved as a second evaluator expectation defect.

Run 34650058540 on research head 6add8fd92d6224b52d260f3d8ff7b895a0b1604a reached and passed all upstream/released-artifact pressure steps, including:

  • published v1.0.0 archive verification;
  • exact runtime immutability proof;
  • fresh CAL v0.5.0 -> Contract C generation;
  • nine-domain / 20-Decision adversarial matrix;
  • two independent FAILED Decision controls;
  • exact replay of the known Contract-B internal-reference aperture against the published v1.0.0 artifact.

It then failed in downstream_contract_d.py because the harness asserted that a mismatched requested operation must always return not_applicable.

Exact released Contract D consumer authority at 298a1a0f7b7b6d7712e11200d04faec3e1ca169b, blob 8b4ad5c9d6fc1145cf334d1416b5d52b9ed93c68, establishes different precedence:

  1. validate Decision/expectation;
  2. compare input authority, policy, target;
  3. if evaluation.state == failed, return evaluation_failed;
  4. only for completed Decisions validate/compare effect and requested operation.

Therefore a FAILED Decision with exact authority/policy/target correctly remains evaluation_failed even when the caller supplies a different requested operation. It has no effect to compare. Wrong authority or target still returns not_applicable before the failed-state branch.

Classification: EVALUATOR_OR_HARNESS_DEFECT, not a Decision Engine or Contract D runtime defect.

Also observed in this run: the known Contract-B aperture reproduced exactly on the published v1.0.0 artifact. Five internal proposition/evidence substitutions were rejected by exact indexed Contract-C validation but both released V1 policies returned CLEAR under the documented bundle-level-only ingress. This is a reproduced known release limitation, not a newly discovered regression.

Next action: correct only the FAILED-operation expectation and rerun. No maintained runtime change, Contract mutation, Authorization, execution, promotion, merge, tag, or release.

Copy link
Copy Markdown
Owner Author

RC0 third hosted cut is behaviorally PASS but preserves one remaining receipt-label defect before terminalization.

Hosted run 34650242995 on branch head 36151537b8674cd29dc22a3074fe4a7c382fe3ce completed SUCCESS. Every substantive step passed, including released-v1.0.0 artifact digest verification, fresh CAL v0.5.0 production, 9-domain pressure matrix, independent FAILED controls, known Contract-B aperture replay, exact Contract D downstream consumption, mutation guard, and evidence upload.

Artifact:

  • ID 10283586119
  • name decision-engine-v1-pressure-rc0-2da5a08f54b6d7f65c3f66bdad21f5b7901b5340
  • GitHub digest sha256:cd71d95beacb70b1d0e102578d4fd44e1f68abab24bfc89649020910ec0720b5
  • API-bound branch head: 36151537b8674cd29dc22a3074fe4a7c382fe3ce

Observed downstream outcomes across 22 exact Contract D objects:

  • candidate_for_authorization: 13
  • hold: 7
  • evaluation_failed: 2

HOLD and FAILED did not escalate. Wrong target/authority remained non-applicable. Wrong requested operation was non-applicable for completed Decisions; exact FAILED Decisions correctly remained evaluation_failed before effect comparison.

The combined artifact receipt has a record-label defect: it writes research_head = $GITHUB_SHA. Under a pull_request run, that value is GitHub's synthetic merge-ref SHA (2da5a08f54b6d7f65c3f66bdad21f5b7901b5340 here), not the branch head. GitHub run/artifact metadata correctly records actual head 36151537....

Classification: EVALUATOR_OR_HARNESS_RECORD_DEFECT. Behavioral evidence remains valid, but the terminal receipt should distinguish branch head from workflow merge SHA.

Next action: change only receipt labeling to record both exact values, then run one final exact-head cycle. No Decision Engine runtime, Contract, Authorization, execution, promotion, merge, tag, or release change.

Copy link
Copy Markdown
Owner Author

RC0 terminal disposition

DISPOSITION: CONFORMS WITH KNOWN AUTHORITY APERTURE AND CAPABILITY GAPS. NO PROMOTION IMPLIED. KEEP DRAFT.

Exact research branch head:
ad3b5ac132e918a3e8f0418525899af85b233215

Exact released subject:
Decision Engine v1.0.0 -> 7be709b2141c767c5da89b8b94cf90233c4238fe

Final hosted evidence:

  • ordinary CI 34650424468 — SUCCESS on exact branch head
  • pressure run 34650424995 — SUCCESS on exact branch head
  • pressure artifact ID 10283771333
  • artifact name decision-engine-v1-pressure-rc0-ad3b5ac132e918a3e8f0418525899af85b233215
  • GitHub artifact digest sha256:d7f6326d06182e334dd7dcdd744db815eccf374012d0547ec8a3109edc1f10a1
  • independently downloaded artifact ZIP SHA-256 matched that GitHub digest exactly

The terminal receipt correctly distinguishes research branch head ad3b5ac... from pull-request workflow subject / synthetic merge SHA 18e139eac01df03ce77926054009ee0f7c554e70.

Observed pressure result

The published v1.0.0 source artifact itself was downloaded and verified against release SHA-256 c440d5de2a3853db9f4ccc2ba37d273e29ce8d9e71e0d646fc0c20eee0cd5902. Maintained src/**, scripts/**, and tests/** were proved unchanged by this research branch.

RC0 exercised both maintained V1 policies across nine domain-shaped Contract-C-valid cases spanning six domain families:

  • MainFrame knowledge synthesis
  • SOP / controlled requirements
  • source literature
  • regulated quality
  • regulated credentials
  • ambiguous / not-checkable material

A fresh real producer control was also generated from exact CAL v0.5.0, yielding Contract C SHA-256 sha256:a6b03b66d77916b62a8cf668a4c683529a45370687a09c3d2adb64c2c2bb53cc. Released V1 consumed it successfully.

Across the ordinary domain/real-producer matrix, 20 exact Contract D Decisions were emitted. Two additional independent FAILED controls brought the downstream total to 22. Exact Contract D consumption observed:

  • candidate_for_authorization: 13
  • hold: 7
  • evaluation_failed: 2

No HOLD or FAILED Decision escalated. Wrong authority and wrong target were non-applicable. Wrong requested operation was non-applicable for completed Decisions; FAILED Decisions correctly remained evaluation_failed before effect comparison. Effect substitution was non-applicable. Metadata remained non-authoritative.

Additional controls passed for wrong Contract-C whole-object identity, wrong expected Contract-B top-level binding, wrong Contract-C authority root, unknown policy identity, cross-case claim replay, cross-case citation replay, stale same-ID claim target, stale same-ID citation target, caller evaluator/registry substitution, exact-repeat determinism, context-key-order invariance, policy-A verdict sensitivity, policy-B basis-membership sensitivity, documented assessment-stage invariance, and CLI/library parity.

Important semantic observations

The two maintained policies remained independent in useful ways:

  • contradicted/unsupported claims HOLD under claim-verification while an exact causal counterevidence contribution can still CLEAR for citation;
  • a supported claim can CLEAR while a retained residual/non-deciding contribution HOLDs for citation.

Therefore a positive claim-tag Decision does not imply that every attached source is a deciding citation, and a negative/held claim Decision does not suppress exact counterevidence citation authority.

Known authority aperture reproduced on the published release

RC0 replayed the prior Contract-B internal-reference experiment from exact science head 570104216ef104ea8d3ba00f9383402b9c2e3154 against the published v1.0.0 artifact.

Five substitutions were tested:

  1. proposition text hash;
  2. proposition ID;
  3. evidence passage hash;
  4. evidence source ID;
  5. evidence passage ID.

For every substitution, exact Contract-C validation with the independently supplied Contract-B index rejected the object, while released V1's documented bundle-level ingress accepted the recomputed exact Contract-C object and both maintained policies reached CLEAR.

Classification: KNOWN_AUTHORITY_APERTURE_REPRODUCED_ON_RELEASE_ARTIFACT, not a newly discovered regression. The release correctly propagates the changed Contract-C immutable identity into Contract D, but it does not independently re-establish proposition/evidence references against the complete Contract-B index.

This remains the most material trust-boundary limitation if V1 is ever asked to consume Contract C from an untrusted or independently assembled producer path.

Capability gaps surfaced

RC0 records ten desired decision requirements that V1 does not maintain and does not emulate through approximate policy reuse:

  • MainFrame synthesis-note lifecycle promotion;
  • SOP / controlled-document approval or rejection;
  • regulated publication/use approval;
  • typed human-review requirement;
  • evidence remediation/retrieval request;
  • task dispatch;
  • release/qualification approval;
  • aggregate multi-claim approval;
  • conditional / approve-with-caveats disposition;
  • explicit reject/remove/retract candidate effects.

These are CAPABILITY_GAP, not V1 defects. V1's public promise remains deliberately limited to exact supported-claim verified-tag candidacy and exact causal-basis citation candidacy.

Evaluator record

Three harness defects were preserved rather than erased:

  1. first cut failed to route generated FAILED Decisions into the downstream consumer;
  2. second cut incorrectly expected wrong-operation not_applicable to outrank Contract D's FAILED-state precedence;
  3. third cut mislabeled pull-request GITHUB_SHA as the research branch head.

Each was classified as evaluator/harness defect, corrected narrowly, and rerun. The final exact-head run passed with unambiguous receipts.

Boundary

No maintained Decision Engine runtime change, Contract mutation, Authorization, execution, merge, promotion, tag, or release occurred in RC0.

No new Decision Engine runtime defect was observed within the tested scope. RC0 does not establish universal correctness, semantic/world truth of the synthetic domain cases, arbitrary CAL successor compatibility, or suitability for whole-document regulated approval.

Copy link
Copy Markdown
Owner Author

Terminal RC0 disposition

SUPPORTED_WITH_KNOWN_BOUNDS — keep Draft.

Exact released subject

  • Decision Engine release commit: 7be709b2141c767c5da89b8b94cf90233c4238fe
  • published v1.0.0 source archive SHA-256: c440d5de2a3853db9f4ccc2ba37d273e29ce8d9e71e0d646fc0c20eee0cd5902

Decisive science execution

  • science head: 71793bb3b933c7f491824ec61c8b2b663dbf6f8f
  • hosted pressure run: 34673772055 — PASS
  • job: 103499852916 — PASS
  • ordinary CI: 34673772043 — PASS
  • artifact ID: 10291328421
  • artifact digest: sha256:1e158cb0223a0499d718ad65d33052e5ceac84e01810a2a6fce234f790aca89c

Terminal record-only reproduction

  • terminal record head: ab56705d65f5f05e02e8f6cc12969fa82ba079b8
  • hosted pressure run: 34673946002 — PASS
  • job: 103500311865 — PASS
  • ordinary CI: 34673945927 — PASS
  • artifact ID: 10291752625
  • artifact digest: sha256:c26e054bb0f67e7022ad576ee9403d9e9eba3b2346f04dcb00ffddbc35b98a3b

The terminal commit changes only the research evidence record and removes a duplicate research workflow/setup-marker clutter. No maintained src/**, scripts/**, tests/**, release metadata, Contract authority, Authorization, or execution surface changed.

Observed result

The published v1.0.0 artifact passed the preregistered multi-angle cohort across:

  • both maintained policies;
  • fresh exact CAL v0.5.0 Contract C production;
  • 9 exact-validator-valid domain-shaped cases across 6 domain families;
  • CLEAR / HOLD / FAILED paths;
  • stale and cross-case claim/citation replay;
  • wrong C identity, wrong expected B binding, wrong C authority, and unknown policy rejection;
  • caller implementation-substitution controls;
  • exact-repeat determinism and context-key-order invariance;
  • CLI/library parity;
  • exact Contract D validation and independent consumption.

Twenty-two canonical Contract D Decisions were independently consumed:

  • candidate_for_authorization: 13
  • hold: 7
  • evaluation_failed: 2

No HOLD or FAILED Decision escalated and no Authorization or execution occurred.

Fresh CAL v0.5.0 Contract C SHA-256:

sha256:a6b03b66d77916b62a8cf668a4c683529a45370687a09c3d2adb64c2c2bb53cc

Known authority boundary reproduced

The already-documented Contract-B internal-reference aperture was reproduced on the published artifact. With top-level B bundle identity held fixed, five coherent C-internal substitutions were rejected by exact indexed Contract-C validation but remained acceptable to maintained bundle-level V1 ingress when freshly canonicalized and freshly targeted:

  1. proposition text hash;
  2. proposition ID;
  3. evidence passage hash;
  4. evidence source ID;
  5. evidence passage ID.

Classification: KNOWN_AUTHORITY_APERTURE_REPRODUCED_ON_RELEASE_ARTIFACT.

This is preserved as a known release limitation, not relabelled a newly discovered V1 defect.

Capability gaps

RC0 records 10 explicit capability gaps rather than overloading the two maintained policies: MainFrame note lifecycle promotion, controlled-document/SOP approval, regulated publication/use approval, typed human review, evidence remediation, task dispatch, release qualification approval, aggregate multi-claim approval, conditional approval, and explicit negative actions such as reject/remove/retract.

Preserved harness failure

Accidental duplicate-workflow run 34673772056 / job 103499856502 failed at its branch-scope guard before any Decision semantics executed. It is preserved as EVALUATOR_OR_HARNESS_DEFECT; the duplicate evaluator was removed in the terminal record commit.

Next boundary

The highest-value successor is not another generic mutation sweep. First close or explicitly externalize the C→B exact-reference authority boundary using the already-supported strict indexed-ingress shape with an independently established exact B-index identity. Then test one explicit new Decision domain, preferably MainFrame audited-note promotion, without changing the meaning of the two v1.0.0 policies.

Full terminal synthesis: research/v1-pressure-test-rc0/RESULT.md.

No merge or production promotion is authorized by this research record.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant