Research: mutation-audit maintained Decision machinery

[camerontjs-dot]

Classification

Draft Research Infrastructure / evaluator-and-regression apparatus audit. No maintained source change, merge, release, tag, promotion, Authorization, or execution.

Exact base

Decision Engine main@a4425f8eb47449ff6c683222921bbea9483742e2.

Result

APPARATUS SUBSTANTIALLY ACTIVE; ONE CHEAP MAINTAINED FALSIFIER MISSING.

Science head: b02d175b4566047c05c11a461008b08fcbaf032f.

Hosted evidence:

• push research run 34254493624: SUCCESS;
• job 102156745415: SUCCESS;
• normal repository CI run 34254522194: SUCCESS;
• push artifact de-machinery-mutation-audit-34254493624, ID 10067318223, ZIP SHA-256 22107f20541b6d81fcdde776ab6c72ae71757554ce2e7b2f3b7bf6d8b7d478ad.

Question

Do the maintained Decision Engine tests actually kill plausible broken implementations of the CAL Contract C -> Decision -> Contract D machinery, or are we relying on historical research volume without active regression power?

Method

The workflow first required the exact unmodified baseline to pass:

• supported-claim integration;
• causal-basis citation integration;
• exact-authority runtime/CLI integration;
• one supplemental wrong-Contract-C-authority control.

It then mutated the checked-out src/** only inside the GitHub runner, one mutant at a time, restored exact main between mutants, and reran the relevant existing maintained tests. Branch bytes never modify maintained source.

Bounded mutation score

10 load-bearing mutants were tested.

Maintained integration tests killed 9 / 10 (0.90) on this bounded sample.

Killed by tests already maintained on main:

1. skip released Contract C validator;
2. skip top-level Contract B binding;
3. Policy A ignores result execution;
4. Policy A ignores supported-verdict predicate;
5. Policy A ignores exact target-content binding;
6. Policy B ignores causal-basis membership / lets residual citation CLEAR;
7. Policy B skips exact claim-evidence-link target binding;
8. runtime lets unknown policy fall through;
9. Contract D canonical-output path skips exact D authority-root verification.

Survivor

M01_skip_contract_c_authority_root_verification survived all three maintained integration surfaces:

• tests/contractCToContractD.integration.mjs -> PASS;
• tests/contractCBasisCitation.integration.mjs -> PASS;
• tests/decisionEvaluateCli.integration.mjs -> PASS.

This does not mean live DE lacks the check. src/contractCIngress.js currently verifies exact Contract C checkout/tag/validator identity, and conformance workflows pin the exact authority checkout.

It means removing that runtime check is not caught by the maintained integration tests exercised here.

Supplemental recent-pressure control

A research-only negative control supplied a non-Git lookalike Contract C root containing copied validator/fixture files.

• correct maintained runtime -> rejects authority_identity_mismatch;
• M01 -> produces completed / clear.

The supplemental control kills the survivor immediately. So this is a cheap missing regression falsifier, not a difficult unresolved architecture question.

Interpretation

The current CAL Decision apparatus is more than a pile of historical test records: most tested load-bearing implementation faults are actively rejected by maintained regressions.

But historical/research evidence is not automatically maintained protection. Exact wrong/lookalike Contract C authority-root behavior is one demonstrated example where a useful negative control has not yet been promoted into the maintained integration suite.

Research-only requirements from PRs #46-#50 are deliberately not scored as mutants when current maintained semantics do not claim them. In particular:

• Policy A assessment-stage policy choice;
• producer semantic/policy identity allowlisting;
• exact C->B indexed reference authority.

Smallest justified follow-up

The smallest regression-only hardening supported by this audit is one maintained wrong/lookalike Contract C authority-root negative control on the shared ingress surface.

PR #50 separately establishes the strict B-index ingress shape, but its independent index-identity source remains an integration boundary and should not be conflated with this cheap test gap.

Full result: research/de-machinery-mutation-audit/RESULT.md.

Keep Draft.