Skip to content

Commit 1db32e3

Browse files
hyperpolymathhyperpolymath
authored
test(contractile): conformance corpus and manifest (24 invalid, 6 valid)
Spec-mandated `<verb>/examples/valid.a2ml` and `<verb>/examples/invalid/{missing_id,wrong_status,empty_array}.a2ml` for all six verbs, plus `conformance/manifest.a2ml` registering expect/reason per fixture and the measured declaration-vs-runner gaps as `known_gaps`. Provisional by design: fixtures use the deployed declaration vocabulary and assume RFC-0001 O5 option (a) (CLI normalisation). If O5 resolves to a migration, this corpus is rewritten in the same PR that migrates the six declarations. Nothing here is executed: probes are strings in data files. Consumed by `contractile self-test` at phase P1.
1 parent e41bc56 commit 1db32e3

26 files changed

Lines changed: 591 additions & 0 deletions

‎.machine_readable/contractiles/adjust/examples/invalid/empty_array.a2ml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# SPDX-License-Identifier: MPL-2.0
2+
# Fixture: adjust/examples/invalid/empty_array.a2ml — contractile conformance corpus (INVALID)
3+
# Expected: `contractile adjust typecheck adjust/examples/invalid/empty_array.a2ml` → exit 3
4+
# Valid only under RFC-0001 O5 option (a); see conformance/manifest.a2ml
5+
# Reason: the primary array (`requirements`) is empty
6+
7+
## Requirements
8+
9+
### (no items declared)

‎.machine_readable/contractiles/adjust/examples/invalid/missing_id.a2ml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# SPDX-License-Identifier: MPL-2.0
2+
# Fixture: adjust/examples/invalid/missing_id.a2ml — contractile conformance corpus (INVALID)
3+
# Expected: `contractile adjust typecheck adjust/examples/invalid/missing_id.a2ml` → exit 3
4+
# Valid only under RFC-0001 O5 option (a); see conformance/manifest.a2ml
5+
# Reason: the requirement has no `### <id>` heading
6+
7+
## Requirements
8+
9+
- description: requirement with no id heading
10+
- tolerance: zero violations
11+
- corrective: Fix by hand
12+
- severity: advisory

‎.machine_readable/contractiles/adjust/examples/invalid/wrong_status.a2ml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# SPDX-License-Identifier: MPL-2.0
2+
# Fixture: adjust/examples/invalid/wrong_status.a2ml — contractile conformance corpus (INVALID)
3+
# Expected: `contractile adjust typecheck adjust/examples/invalid/wrong_status.a2ml` → exit 3
4+
# Valid only under RFC-0001 O5 option (a); see conformance/manifest.a2ml
5+
# Reason: `status: mostly` is outside {declared, partial, verified, failing}
6+
7+
## Requirements
8+
9+
### doc-language
10+
- description: Documentation uses inclusive language
11+
- tolerance: no new violations
12+
- corrective: Review and rewrite flagged passages
13+
- severity: advisory
14+
- status: mostly

‎.machine_readable/contractiles/adjust/examples/valid.a2ml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# SPDX-License-Identifier: MPL-2.0
2+
# Fixture: adjust/examples/valid.a2ml — contractile conformance corpus (VALID)
3+
# Expected: `contractile adjust typecheck adjust/examples/valid.a2ml` → exit 0
4+
# Valid only under RFC-0001 O5 option (a); see conformance/manifest.a2ml
5+
6+
## Requirements
7+
8+
### doc-language
9+
- description: Documentation uses inclusive language
10+
- tolerance: no new violations in changed passages
11+
- corrective: Review and rewrite flagged passages
12+
- severity: advisory

‎.machine_readable/contractiles/bust/examples/invalid/empty_array.a2ml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# SPDX-License-Identifier: MPL-2.0
2+
# Fixture: bust/examples/invalid/empty_array.a2ml — contractile conformance corpus (INVALID)
3+
# Expected: `contractile bust typecheck bust/examples/invalid/empty_array.a2ml` → exit 3
4+
# Valid only under RFC-0001 O5 option (a); see conformance/manifest.a2ml
5+
# Reason: the primary array (`failure_modes`) is empty
6+
7+
## Failure Modes
8+
9+
### (no items declared)

‎.machine_readable/contractiles/bust/examples/invalid/missing_id.a2ml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# SPDX-License-Identifier: MPL-2.0
2+
# Fixture: bust/examples/invalid/missing_id.a2ml — contractile conformance corpus (INVALID)
3+
# Expected: `contractile bust typecheck bust/examples/invalid/missing_id.a2ml` → exit 3
4+
# Valid only under RFC-0001 O5 option (a); see conformance/manifest.a2ml
5+
# Reason: the failure mode has no `### <id>` heading
6+
7+
## Failure Modes
8+
9+
- class: timeout
10+
- description: failure mode with no id heading
11+
- injection_probe: sleep 1
12+
- recovery_probe: test -f /dev/null
13+
- expected_recovery_time_seconds: 5
14+
- status: declared

‎.machine_readable/contractiles/bust/examples/invalid/wrong_status.a2ml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# SPDX-License-Identifier: MPL-2.0
2+
# Fixture: bust/examples/invalid/wrong_status.a2ml — contractile conformance corpus (INVALID)
3+
# Expected: `contractile bust typecheck bust/examples/invalid/wrong_status.a2ml` → exit 3
4+
# Valid only under RFC-0001 O5 option (a); see conformance/manifest.a2ml
5+
# Reason: `status: broken` is outside {declared, drilled, verified, failing}
6+
7+
## Failure Modes
8+
9+
### timeout-handled
10+
- class: timeout
11+
- description: Timeouts are bounded
12+
- injection_probe: sleep 1
13+
- recovery_probe: test -f /dev/null
14+
- expected_recovery_time_seconds: 5
15+
- status: broken

‎.machine_readable/contractiles/bust/examples/valid.a2ml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# SPDX-License-Identifier: MPL-2.0
2+
# Fixture: bust/examples/valid.a2ml — contractile conformance corpus (VALID)
3+
# Expected: `contractile bust typecheck bust/examples/valid.a2ml` → exit 0
4+
# Valid only under RFC-0001 O5 option (a); see conformance/manifest.a2ml
5+
6+
## Failure Modes
7+
8+
### disk-full-handled
9+
- class: disk_full
10+
- description: A full disk is detected and reported without corrupting state
11+
- injection_probe: dd if=/dev/zero of=/tmp/fill bs=1M count=1 status=none
12+
- recovery_probe: test -f /tmp/fill && rm -f /tmp/fill
13+
- expected_recovery_time_seconds: 30
14+
- status: declared

‎.machine_readable/contractiles/conformance/README.adoc‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
// SPDX-License-Identifier: CC-BY-SA-4.0
2+
// Copyright (c) 2026 Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk>
3+
= Contractile Conformance Corpus
4+
5+
Fixtures and manifest for `contractile self-test`
6+
(`RFC-0001-contractile-cli.adoc` §Conformance tests, phase P1).
7+
8+
== Layout
9+
10+
[source]
11+
----
12+
.machine_readable/contractiles/
13+
├── conformance/
14+
│ ├── README.adoc ← this file
15+
│ └── manifest.a2ml ← registered fixtures: expect + reason per file
16+
├── must/examples/
17+
│ ├── valid.a2ml ← MUST typecheck → exit 0
18+
│ └── invalid/
19+
│ ├── missing_id.a2ml ← MUST fail → exit 3
20+
│ ├── wrong_status.a2ml ← MUST fail → exit 3
21+
│ └── empty_array.a2ml ← MUST fail → exit 3
22+
└── … (same shape for trust, adjust, dust, bust, intend)
23+
----
24+
25+
Fixtures live in the verb directories because that is where
26+
`standards/docs/CONTRACTILE-SPEC.adoc` §Test Fixtures puts them. The manifest
27+
lives one level up so a single file registers the whole corpus, following
28+
`hyperpolymath/k9-ecosystem/conformance/manifest.a2ml`.
29+
30+
NOTE: `must/`, `trust/`, `adjust/` and `intend/` now exist as verb directories
31+
holding *only* `examples/`. Their declarations are still at the pre-spec flat
32+
paths. That is deliberate: it makes the corpus spec-located without pretending
33+
the layout migration (spec Ruling 2 and this repository's own drift) is done.
34+
35+
== Status: provisional, and why that is honest
36+
37+
Every fixture uses the *deployed* declaration vocabulary (`- run:`, `- probe:`,
38+
`- tolerance:`/`- corrective:`, `- injection_probe:`), not the vocabulary the
39+
Nickel runner schemas expect (`probe`, `target`, `reason`, …). That is a
40+
deliberate bet recorded in the manifest:
41+
42+
[quote]
43+
____
44+
assumes = "RFC-0001 O5 option (a): the CLI normalises deployed declaration
45+
fields onto the runner schemas. If O5 resolves to (b) or (c), this corpus must
46+
be rewritten (24 invalid + 6 valid fixtures)."
47+
____
48+
49+
If O5 resolves to "migrate the declarations" (option b), these fixtures become
50+
wrong on purpose and are rewritten in the same PR that migrates the six
51+
declarations. Until then they pin the *actual* corpus, which is the only corpus
52+
whose pass/fail the estate can act on.
53+
54+
== What `self-test` must do with this corpus (P1 acceptance)
55+
56+
. Read `conformance/manifest.a2ml`; an unregistered fixture fails the run.
57+
. `[[valid]]` entries: exit `0` and a per-item verdict line.
58+
. `[[invalid]]` entries: exit `3` (input error), for the recorded `reason`.
59+
Asserting only "non-zero" is not conformance — the reason must match.
60+
. `[[known_gaps]]` entries: the CLI must *report* the gap (warning, or error
61+
under `--strict`) rather than silently pass or crash.
62+
. Two consecutive runs produce byte-identical reports (RFC-0001 §Determinism).
63+
64+
== Known gaps (measured, not invented)
65+
66+
See `manifest.a2ml` `[[known_gaps]]`. Summary: `adjust` has no discharge field
67+
at all; `dust` declarations lack the schema-required `target`/`reason`; `bust`
68+
`class` values fall outside the runner enum; `adjust` uses
69+
`severity: advisory`, outside `severity_core`; `intend` wishes are depth-4
70+
headings under a depth-3 horizon group, so a flat heading scan mis-parses them;
71+
and every schema's `id` is carried by the heading, never a field.
72+
73+
== Not yet here (P1 additions)
74+
75+
Per RFC-0001 §Conformance tests, phase P1 adds: `unknown_field`, `no_discharge`,
76+
`malformed_utf8`, `escaped_path` (import escaping the repo root), permission
77+
refusal cases (exit `4`), timeout cases (exit `6`), manifest-tamper cases
78+
(exit `7`), and golden outputs for `text|json|a2ml`.
79+
80+
== Guardrails
81+
82+
* Nothing in this corpus is executed today: probes are strings in data files.
83+
The fixtures use inert commands (`test -f`, `ls`) so that a future accidental
84+
execution is harmless — except `bust/examples/valid.a2ml`, whose
85+
`injection_probe` writes a 1 MiB file in `/tmp` by design; it is listed here
86+
so that is not a surprise at P4.
87+
* `.github/hooks/validate-a2ml.sh` validates these files in CI (structure, SPDX,
88+
headings). Contractile-shaped files are exempt from the manifest identity
89+
checks by design.

0 commit comments

Comments
 (0)