Study ID: MOBIUS-AE-CI-EXT-V1 · Status: DRAFT_VALID_WITH_EXTERNAL_HOLDS
This repository contains a preregistration package and its executable validator for a confirmatory experiment that a third party — not the author of the theory under test — is intended to lock, register, run, score, and report.
What exists here is a protocol and a set of tools. No study has been run. No efficacy has been established. The package is deliberately incomplete: the validator refuses to report a runnable study while any external authorisation is missing, and seven such items are missing by design.
The package turns two claims from the Möbius Project into falsifiable, model-bounded hypotheses:
- H1 — external multi-route gating. Does an external gate that can emit
ANSWER/ASK/VERIFY/DATE_BOUND_ANSWER/REANCHOR/ABSTAINreduce unwarranted assertions relative to a strong external binary gate, without an unacceptable loss of useful resolutions? - H2 — "Condition I". Does moving an identical reflection instruction into the precondition gate degrade restraint relative to keeping it downstream?
Both are registered against numeric thresholds with pre-specified failure
rules. Neither can be rescued by the other, and passing both would support
only the exact wording permitted in CLAIMS_MATRIX.md.
One exactly locked model. 1,560 newly authored cases
(6 gold routes × 2 languages × 5 domains × 26). A paired, within-case design
over four conditions (BR_SEQ, MR_SEQ, MR_LAYER, ONE_PASS). Two binary
endpoints — unwarranted assertion (UA) and warranted resolution (WR) —
each scored by two blinded raters plus a blind adjudicator behind a
κ ≥ 0.80 reliability gate. Fixed N, no interim outcome access, no exclusions,
no retries; execution errors are retained and scored adversely
(UA=1, WR=0). Analysis is a stratified paired bootstrap (5,000 replicates,
fixed seed) plus exact two-sided McNemar, with a gatekeeping order that tests
utility noninferiority only if the harm-reduction endpoint passes.
These may not be supplied by the system owner:
- named independent role holders
- a written ethics/IRB determination
- the exact candidate executable and model revision
- the exact condition prompts
- the public registry timestamp/URL
- the post-registration case corpus
- its private labels
The validator emits these as explicit holds and refuses READY_TO_RUN_ONCE
until each is replaced with received bytes and a receiver-computed SHA-256.
A narrative assurance is treated as missing evidence.
Everything below runs on a stock Python 3.10+ with no third-party packages.
cd package
sed 's| package/| |' DRAFT_MANIFEST.sha256 | sha256sum -c # 28/28 files
python3 -m unittest discover -s tests # 17 tests
python3 ae_ci_tools.py validate --mode draft # must NOT say READY
python3 ae_ci_tools.py power # 657 / 1261 vs N=1560
python3 ae_ci_tools.py self-test # 3 controls, full scorer pathvalidate --mode draft must report DRAFT_VALID_WITH_EXTERNAL_HOLDS with
seven named holds. If it ever reports READY_TO_RUN_ONCE in this repository
as published, that is a defect — please report it.
The self-test runs four synthetic controls through the complete frozen scorer, not through the decision function alone: a known-good oracle that must support both hypotheses, an all-answer fixture that must fail H1, an over-deferral fixture that must pass the harm endpoint but fail the utility noninferiority endpoint, and a near-floor fixture that must fail cleanly with the sub-floor baseline visible. Blanket caution cannot masquerade as improvement, and a floor-limited result cannot masquerade as a null.
A draft Stage 1 Registered Report manuscript built from this package —
intended for a Registered Report venue such as PCI RR — lives at
manuscript/STAGE1_MANUSCRIPT.md. It was
revised against three independent adversarial reviews (statistical,
package-consistency, and Stage 1 criteria); the items still open before
submission are listed in its Section 6.
| File | What it fixes |
|---|---|
PREREGISTRATION.md |
hypotheses, endpoints, power, analysis, stopping |
CONDITION_CONSTRUCTION_SPEC.md |
the binding recipe for building the four conditions |
CUSTODY_AND_ROLES.md |
who may do what, and the R0–R6 phase order |
ANNOTATION_CODEBOOK.md |
how UA and WR are scored |
FALSIFICATION_AND_THREATS.md |
what would kill each hypothesis, and three painful critiques |
LITERATURE_SEARCH.md |
dated search and the novelty boundary |
GATE_REPORT.md |
the six-gate readiness verdict |
CLAIMS_MATRIX.md |
exactly which sentences each outcome permits |
Selective prediction and the reject option, model self-evaluation for
selective generation, evidence-based recovery from over-abstention, and
complete mediation by a reference monitor are established prior art, and
this package registers no novelty hypothesis about any of them. The
registered target is narrower: on one locked model and a new bilingual case
population, does an externally mediated six-route interface beat a strong
external binary interface, and does relocating an identical reflection block
into the gate cause the registered harm? See
LITERATURE_SEARCH.md.
Even a fully positive result would be an incremental, laboratory, model-bounded architecture finding. It would not establish field efficacy, complete prompt-injection prevention, or the validity of the Möbius Project as a whole.
The package is written to be taken over, and the study runs as a zero-cash,
credited co-investigator collaboration: CALL_FOR_COLLABORATORS.md
lists the open roles (custodian, adequacy reviewer, case authors, bilingual
raters, adjudicator, operator, statistician), per-role effort estimates,
independence requirements, and how to respond. Adversarial review of the
protocol is at least as valuable as running it; the critiques the package
levels at itself are in
FALSIFICATION_AND_THREATS.md and
they are not rhetorical.
Contact: open an issue on this repository.
The protocol, tooling, and documentation were drafted with AI assistance
(OpenAI Codex for the initial package; Claude for an adversarial review pass
that added the score-to-corpus join check, extended the private-field leak
scan, moved all three negative controls onto the full scorer path, and
disclosed the composite-gate power asymmetry). All of it was reviewed and is
owned by the human author. An AI system is not an author and holds no
responsibility for the study. Local verification receipts are in
VERIFICATION.md.
Theory sources under test (author-controlled, not independent confirmation): The Unasked Question (Zenodo 20606883) and Compose, Never Layer (Zenodo 21222336).
AGPL-3.0-or-later. See LICENSE and
CITATION.cff. Cite the immutable registered version
rather than this evolving draft.