From d06e5595c2af8ddff3d29f9844b3b49b30b138a9 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Mon, 14 Sep 2026 22:14:42 +0100 Subject: [PATCH 1/3] fix(validator): see .deed at all, and accept every ruled identity form MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit deed-validate-action was a THIRD editable upstream of this validator and had never been fixed. Measured on a fresh clone of main, 2026-09-14: grep -c '\.deed' validate-a2ml.sh -> 0 discovery glob -> find ... -name '*.a2ml' So the action published under the DEED name scanned zero files in a converted repository and exited 0, reporting success. Task #72 recorded "BOTH upstreams landed" (deed-ecosystem#52, deed-core#3); that was an undercount — there are three, and this is the one with the widest blast radius: 131 workflows across the estate reference it by `uses:`. Five changes, deliberately in ONE commit, because landing the glob alone is strictly worse than landing nothing: 1. Discovery globs `.deed` as well as `.a2ml`. 2. Identity accepts the DEED keyword form `:canonical-name` (also `:estate-authority`, `:agent-id`). The existing patterns all require `=` or `:` as a SEPARATOR, so a leading-colon space-separated keyword matched none of them. 3. Version accepts `:schema-version`. Matched narrowly on purpose: `:registry-version` is a distinct optional field and must NOT satisfy the required-version check. 4. Identity accepts the DEED HEAD FORM. Not every deed carries `:canonical-name` — ATLAS.deed identifies itself by its head alone, exactly as the six-file set identifies itself by a `[metadata]` section. The four ruled heads are ENUMERATED (estate-deed, estate-atlas-deed, repo-deed, praxis-deed) rather than matched as `*-deed`, so an invented head is not silently accepted as a fifth. Only the FIRST s-expression in the file is consulted: a recognised head appearing after some other form is a misplaced head, not identity, and must still fail. 5. Identity and version accept the s-expression dialect's NESTED forms, `(metadata (name "…") (version "…"))`, which match neither the TOML `key =` nor the `[metadata]` bracket patterns. This is the sanctioned fourth surface, and the conformance corpus lists valid/s-expression-state.a2ml as expect="pass" while it was failing. Without 2-5, adding the glob would discover every deed and then fail it — and report_issue() promotes warnings to errors under strict mode, so those 131 consumers would have gone from a silent no-op straight to a hard red gate. Fixing one of two disagreeing layers is itself the defect. `AI.deed` is added beside `AI.a2ml` in the identity exemption so that renaming a file does not silently TIGHTEN the gate on it. VERIFIED against deed-ecosystem's conformance corpus — an INDEPENDENTLY authored expectation table (conformance/manifest.a2ml), not fixtures written to match this patch. All 15 declared expectations now hold: 8 positive fixtures (expect="pass") -> all clean incl. one per ruled head, and the s-expression fixture that was failing 7 negative fixtures (expect="error" or -> all still correctly flagged "strict-error") Two further on-disk negatives, not listed in the manifest, discriminate the new head logic specifically: deed-head-not-first.deed -> still warns (head is misplaced) deed-registry-version-only.deed -> warns on VERSION ONLY, proving the narrow `:schema-version` match holds and that the head grants identity Plus a synthetic negative control: `(invented-deed` is rejected while `(estate-deed` is accepted, proving the enumeration is doing the work. Assertions are on the discovery COUNT, never the exit code — before this commit the same tree discovered 0 files and exited 0. Note for future testing: this script takes its path from INPUT_PATH, NOT from $1, so a positional argument is silently ignored and it scans `.` instead. Also retitles the action to "Validate DEED Manifests" with DEED-accurate input and output descriptions, which is prerequisite to any Marketplace listing: the listing identity is taken from action.yml at the published tag. The script FILENAME stays validate-a2ml.sh. It is internal to the action, and renaming it would break the 120 stamped copies and any direct caller; that belongs in the #72 sweep, not in a release commit. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01QNjWX2B4FffG7zqMBMui6v --- action.yml | 21 +++++++++--------- validate-a2ml.sh | 58 ++++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 62 insertions(+), 17 deletions(-) diff --git a/action.yml b/action.yml index d0a0310..3458e02 100644 --- a/action.yml +++ b/action.yml @@ -1,15 +1,16 @@ # SPDX-License-Identifier: MPL-2.0 # Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) # -# action.yml — Validate A2ML Manifests GitHub Action -# Scans repository for .a2ml files and validates structure, required fields, -# SPDX headers, and attestation blocks. +# action.yml — Validate DEED Manifests GitHub Action +# Scans repository for .deed (and superseded .a2ml) files and validates +# structure, required fields, SPDX headers, and attestation blocks. -name: 'Validate A2ML Manifests' +name: 'Validate DEED Manifests' description: >- - Scan and validate .a2ml manifest files in your repository. - Checks for required fields (agent-id/pedigree name, version), - SPDX headers, and attestation block structure. + Scan and validate .deed manifest files against the DEED grammar. + Checks identity (:canonical-name), version (:schema-version), + SPDX headers, and attestation block structure. Also reads the + superseded .a2ml extension. author: 'Jonathan D.A. Jewell' branding: @@ -19,7 +20,7 @@ branding: inputs: path: description: >- - Directory path to scan for .a2ml files. + Directory path to scan for .deed files (and superseded .a2ml). Defaults to the repository root. required: false default: '.' @@ -32,7 +33,7 @@ inputs: outputs: files-scanned: - description: 'Number of .a2ml files scanned' + description: 'Number of DEED-family files scanned' value: ${{ steps.validate.outputs.files_scanned }} errors: description: 'Number of validation errors found' @@ -44,7 +45,7 @@ outputs: runs: using: 'composite' steps: - - name: Validate A2ML manifests + - name: Validate DEED manifests id: validate shell: bash env: diff --git a/validate-a2ml.sh b/validate-a2ml.sh index fbdd88c..e7510f4 100755 --- a/validate-a2ml.sh +++ b/validate-a2ml.sh @@ -136,22 +136,61 @@ validate_a2ml() { || [[ "$line" =~ ^@abstract ]]; then has_identity=true fi + # DEED keyword identity form: `:canonical-name "..."`. Leading colon, + # hyphenated, SPACE-separated — so it matches none of the forms above, + # which all require `=` or `:` as a separator. Adding the `.deed` glob + # without this would discover every deed and then fail it. + if [[ "$line" =~ ^[[:space:]]*:(canonical-name|estate-authority|agent-id)[[:space:]] ]]; then + has_identity=true + fi # Check for version field (either separator) if [[ "$line" =~ ^[[:space:]]*(version|schema_version)[[:space:]]*[=:] ]]; then has_version=true fi + # DEED keyword version form: `:schema-version "0.1.0"`. Matched + # deliberately narrowly: `:registry-version` is a DISTINCT optional + # field and must NOT satisfy the required-version check. + if [[ "$line" =~ ^[[:space:]]*:schema-version[[:space:]] ]]; then + has_version=true + fi + # S-expression dialect (the sanctioned fourth surface): identity and + # version are NESTED FORMS — `(metadata (name "…") (version "…"))` — + # so they match neither the TOML `key =` nor the `[metadata]` bracket + # patterns above. The conformance corpus lists + # valid/s-expression-state.a2ml as expect="pass", and it did not. + if [[ "$line" =~ ^[[:space:]]*\((metadata|scorecard)([[:space:]]|$) ]]; then + has_identity=true + fi + if [[ "$line" =~ ^[[:space:]]*\(version[[:space:]] ]]; then + has_version=true + fi # Template placeholder marker ({{PROJECT_NAME}}, {{VERSION}}, …) if [[ "$line" == *"{{"*"}}"* ]]; then has_placeholders=true fi done < "$file" + # DEED head form as identity. Not every deed carries `:canonical-name` — + # `ATLAS.deed` identifies itself by its head alone, exactly as the six-file + # set identifies itself by a `[metadata]` section. Only the FIRST + # s-expression in the file is consulted: a recognised head appearing after + # some other form is a MISPLACED head, not identity, and must still warn. + # The four heads are enumerated rather than matched as `*-deed` so that an + # invented head is not silently accepted as a fifth. + local first_form + first_form="$(grep -m1 '^(' "$file" || true)" + if [[ "$first_form" =~ ^\((estate-deed|estate-atlas-deed|repo-deed|praxis-deed)([[:space:]]|$) ]]; then + has_identity=true + fi + # Classes that are identity-free by design (see header): local basename basename="$(basename "$file")" local identity_exempt=false # AI manifests: markdown prose (0-AI-MANIFEST.a2ml, AI.a2ml, …) - if [[ "$basename" == *"AI-MANIFEST"* || "$basename" == "AI.a2ml" ]]; then + # `AI.deed` is listed alongside `AI.a2ml` so that renaming a file does not + # silently TIGHTEN the gate on it. A rename must be behaviour-preserving. + if [[ "$basename" == *"AI-MANIFEST"* || "$basename" == "AI.a2ml" || "$basename" == "AI.deed" ]]; then identity_exempt=true fi # Templates/scaffolds @@ -234,14 +273,19 @@ validate_a2ml() { # --------------------------------------------------------------------------- echo "::group::A2ML Manifest Validation" -echo "Scanning ${SCAN_PATH} for .a2ml files..." +echo "Scanning ${SCAN_PATH} for .deed and .a2ml files..." echo "" # Find all .a2ml files, excluding .git directory -mapfile -t a2ml_files < <(find "$SCAN_PATH" -name '*.a2ml' -not -path '*/.git/*' -type f | sort) +# Discovery globs BOTH extensions. `.deed` is the current format name; `.a2ml` +# is the superseded one and is still present in the wild, so both are scanned. +# NOTE: globbing `.a2ml` alone meant this action scanned nothing in a converted +# repository and exited 0 — a total gate bypass that reports success. Assert on +# the discovery COUNT, never on the exit code. +mapfile -t a2ml_files < <(find "$SCAN_PATH" \( -name '*.a2ml' -o -name '*.deed' \) -not -path '*/.git/*' -type f | sort) if [[ ${#a2ml_files[@]} -eq 0 ]]; then - echo "::notice::No .a2ml files found in ${SCAN_PATH}" + echo "::notice::No .deed or .a2ml files found in ${SCAN_PATH}" echo "files_scanned=0" >> "$GITHUB_OUTPUT" 2>/dev/null || true echo "errors=0" >> "$GITHUB_OUTPUT" 2>/dev/null || true echo "warnings=0" >> "$GITHUB_OUTPUT" 2>/dev/null || true @@ -249,7 +293,7 @@ if [[ ${#a2ml_files[@]} -eq 0 ]]; then exit 0 fi -echo "Found ${#a2ml_files[@]} .a2ml file(s)" +echo "Found ${#a2ml_files[@]} DEED-family file(s)" echo "" for file in "${a2ml_files[@]}"; do @@ -276,9 +320,9 @@ echo "::endgroup::" # Exit with failure if errors were found if [[ $ERRORS -gt 0 ]]; then - echo "::error::A2ML validation failed with ${ERRORS} error(s)" + echo "::error::DEED validation failed with ${ERRORS} error(s)" exit 1 fi -echo "A2ML validation passed." +echo "DEED validation passed." exit 0 From 212b257c1131b7926232ca8d064c486974762b29 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 15 Sep 2026 03:19:39 +0100 Subject: [PATCH 2/3] fix(validator): detect nested identity/version at any position, not line start CodeRabbit flagged `(version` on validate-a2ml.sh:164 as unreachable for the sanctioned nested form. Verified: the `^[[:space:]]*` anchor required `(version` to open the line, so an INLINED (state (metadata (name "x") (version "1.0.0"))) set has_version=false and, in strict mode, was REJECTED as a valid manifest. The conformance corpus did not catch this because valid/s-expression-state.a2ml writes the form MULTI-LINE, putting `(version` at the start of its own line. A corpus that passes is not proof the regex is right -- it is proof the corpus does not exercise the shape. Both layers are fixed, not one: the identity check on the line above carried the same anchor, so `(state (metadata ...))` inlined also warned "No identity found". It survived only when the head happened to be one of the four ruled heads and the head-form check rescued it. Fixing `version` alone would have left the two checks disagreeing. Token boundaries are preserved -- the `(` prefix and the trailing whitespace/EOL mean `:registry-version` and `(versioning` still do not match. Measured against conformance/ (deed-ecosystem), before and after: valid/ 8 files, 0 errors, 0 warnings (unchanged) invalid/ 10 files, 10 warnings non-strict / 10 errors strict (unchanged) The invalid/ set is the positive control: it proves the run could have reported non-zero. shellcheck -S warning clean. README.adoc (CodeRabbit's second finding): the action is "Validate DEED Manifests" in action.yml but the README still said "Validate A2ML" and documented `.a2ml`-only discovery. Also corrected while here, because this README is the Marketplace landing page: - the spec link pointed at standards/tree/main/a2ml, which is a 404; the live path is standards/tree/main/deed - the usage snippet named hyperpolymath/standards/a2ml/actions/validate@main, which is not this action at all - `link:../../README.adoc` was a relative link left over from when this action lived inside the standards monorepo, and resolves nowhere here - "Attested Markup Language" -> "Attestation Markup Language" (ruled) Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01QNjWX2B4FffG7zqMBMui6v --- CHANGELOG.adoc | 45 +++++++++++++++++++++++++++++++++++++++++++++ README.adoc | 33 +++++++++++++++++---------------- validate-a2ml.sh | 11 +++++++++-- 3 files changed, 71 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.adoc b/CHANGELOG.adoc index ca1c652..f888390 100644 --- a/CHANGELOG.adoc +++ b/CHANGELOG.adoc @@ -7,3 +7,48 @@ Changelog], and this project adheres to https://semver.org/spec/v2.0.0.html[Semantic Versioning]. === [Unreleased] + +=== [1.0.0] -- 2026-09-15 + +First tagged release, and the first release in which the action can actually +see a `.deed` file. + +==== Added + +* `.deed` discovery. The file-discovery glob now matches `*.deed` alongside the + superseded `*.a2ml`. Before this release the glob named `*.a2ml` only, so a + repository containing nothing but `.deed` manifests was scanned, found zero + files, reported zero errors and exited `0` -- a silent total bypass rather + than a visible failure. +* Recognition of the `:canonical-name`, `:estate-authority` and `:agent-id` + identity keys, and of `:schema-version` as a version key. +* Recognition of identity by head form alone for the four ruled DEED heads + (`estate-deed`, `estate-atlas-deed`, `repo-deed`, `praxis-deed`), which is how + `ATLAS.deed` identifies itself -- it carries no `:canonical-name`. + +==== Fixed + +* Nested `(metadata ...)` and `(version ...)` forms written *inline* were not + detected. Both checks were anchored with `^[[:space:]]*`, so a manifest + written as `(state (metadata (name "x") (version "1.0.0")))` reported + "No identity found" and "Missing version" -- and under `strict: true` that + rejected a valid manifest. The anchors are gone; the `(` prefix and the + trailing whitespace/EOL boundary are retained, so `(versioning` and + `:registry-version` still correctly fail the version check. +* `README.adoc`, which is the GitHub Marketplace landing page, carried three + dead or wrong references: the DEED specification link pointed at + `standards/tree/main/a2ml` (a 404), the usage snippet told users to write + `uses: hyperpolymath/standards/a2ml/actions/validate@main` (a different + repository entirely), and the ecosystem section used a monorepo-relative + `link:../../README.adoc` that resolves nowhere from a standalone repo. + +==== Changed + +* Prose and examples throughout `README.adoc` and `action.yml` now speak of DEED + and `.deed`, with `.a2ml` referred to as superseded rather than as the primary + format. A2ML is expanded as "Attestation Markup Language". + +==== Known limitations + +* The action has no self-test workflow of its own, so this fix is not yet + exercised by the repository's own CI. Tracked as a follow-up. diff --git a/README.adoc b/README.adoc index 298f8c4..fb44c2c 100644 --- a/README.adoc +++ b/README.adoc @@ -1,18 +1,18 @@ // SPDX-License-Identifier: CC-BY-SA-4.0 // Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) -= Validate A2ML Manifests -- GitHub Action += Validate DEED Manifests -- GitHub Action :author: Jonathan D.A. Jewell :toc: preamble :icons: font == Overview -GitHub Action that scans a repository for `.a2ml` files and validates their structure, -required fields, and compliance with the -https://github.com/hyperpolymath/standards/tree/main/a2ml[A2ML specification]. +GitHub Action that scans a repository for `.deed` files -- and the superseded `.a2ml` +extension -- and validates their structure, required fields, and compliance with the +https://github.com/hyperpolymath/standards/tree/main/deed[DEED specification]. -A2ML (Attested Markup Language) is the manifest format used across +DEED -- which supersedes A2ML (Attestation Markup Language) -- is the manifest format used across https://github.com/hyperpolymath/standards[RSR (Rhodium Standard Repository)] projects to declare machine-readable metadata, AI agent instructions, attestation provenance, and project state. @@ -47,18 +47,18 @@ Add this step to any GitHub Actions workflow: [source,yaml] ---- -name: Validate A2ML +name: Validate DEED on: [push, pull_request] permissions: contents: read jobs: - validate-a2ml: + validate-deed: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - - uses: hyperpolymath/standards/a2ml/actions/validate@main + - uses: hyperpolymath/deed-validate-action@v1 with: path: '.' # Directory to scan (default: repo root) strict: 'false' # Promote warnings to errors (default: false) @@ -72,12 +72,12 @@ jobs: | `path` | `.` -| Directory path to scan for `.a2ml` files. The scan is recursive, excluding `.git/`. +| Directory path to scan for `.deed` files (and superseded `.a2ml`). The scan is recursive, excluding `.git/`. | `strict` | `false` | When `true`, all warnings are promoted to errors and the action fails on any validation - issue. Recommended for repositories that require full A2ML compliance. + issue. Recommended for repositories that require full DEED compliance. |=== === Outputs @@ -87,7 +87,7 @@ jobs: | Output | Description | `files-scanned` -| Number of `.a2ml` files discovered and processed. +| Number of DEED-family files (`.deed` and superseded `.a2ml`) discovered and processed. | `errors` | Count of validation errors. The action exits with code 1 if this is non-zero. @@ -103,7 +103,7 @@ jobs: | Code | Meaning | `0` -| All files valid (or only warnings in non-strict mode). Also returned when no `.a2ml` +| All files valid (or only warnings in non-strict mode). Also returned when no DEED-family files are found (with a `::notice::` annotation). | `1` @@ -132,8 +132,9 @@ SPDX-License-Identifier: MPL-2.0 See link:LICENSE[LICENSE] for the full text. -== Part of the A2ML Ecosystem +== Part of the DEED Ecosystem -This action is part of the link:../../README.adoc[A2ML specification and tooling] in the -https://github.com/hyperpolymath/standards[standards monorepo]. See the parent directory -for language bindings, Pandoc support, editor integrations, and the CLI. +This action is part of the https://github.com/hyperpolymath/deed-ecosystem[DEED specification +and tooling]. The normative grammar lives in the +https://github.com/hyperpolymath/standards/tree/main/deed[standards monorepo], with language +bindings, Pandoc support, editor integrations, and the CLI. diff --git a/validate-a2ml.sh b/validate-a2ml.sh index e7510f4..43f2ae5 100755 --- a/validate-a2ml.sh +++ b/validate-a2ml.sh @@ -158,10 +158,17 @@ validate_a2ml() { # so they match neither the TOML `key =` nor the `[metadata]` bracket # patterns above. The conformance corpus lists # valid/s-expression-state.a2ml as expect="pass", and it did not. - if [[ "$line" =~ ^[[:space:]]*\((metadata|scorecard)([[:space:]]|$) ]]; then + # ⛔ NOT anchored to line start: the nested form is frequently INLINED + # as `(state (metadata (name "…") (version "…")))`, putting both + # `(metadata` and `(version` mid-line. A `^[[:space:]]*` anchor here + # silently failed every inlined deed -- and in strict mode that + # rejected a VALID manifest. The `(` prefix and the trailing + # whitespace/EOL keep the token boundary, so `:registry-version` + # and `(versioning` still do not match. + if [[ "$line" =~ \((metadata|scorecard)([[:space:]]|$) ]]; then has_identity=true fi - if [[ "$line" =~ ^[[:space:]]*\(version[[:space:]] ]]; then + if [[ "$line" =~ \(version[[:space:]] ]]; then has_version=true fi # Template placeholder marker ({{PROJECT_NAME}}, {{VERSION}}, …) From dd833dbe823e5f5350e18a40efca0df1d9effa30 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 15 Sep 2026 03:40:07 +0100 Subject: [PATCH 3/3] test(self-test): run the action against its own tree for the first time The action had no self-test workflow, and its own repository contained not one .deed file -- so nothing it did was ever exercised against the format it is named for. That is why a validator which could not see .deed at all shipped and stayed shipped. Adds .github/workflows/self-test.yml and a fixture corpus under test/fixtures/. Two rules are load-bearing in the workflow, both learned from this defect: 1. Assert on discovery COUNT, never on exit code. A validator that finds nothing exits 0. Run the pre-fix validator against test/fixtures/deed-only and it emits `::notice::No .a2ml files found` -- a notice, not a warning -- sets no outputs and exits 0. An exit-code assertion calls that a pass. 2. Always run a positive control. The invalid-corpus job must report a non-zero error count; if it ever reports zero, the run has lost the ability to see failure and every other green result is worthless. Both regression tests were proven to FAIL against the versions carrying the defects they target, rather than merely passing against the fixed one: - validator at d06e559^ (no .deed glob) vs deed-only/ -> 0 discovered, exit 0 - validator at d06e559 (glob fixed, ^ anchors intact) vs valid/ -> 2 errors, on inline-nested.deed and legacy-superseded.a2ml, both inlined forms - validator at HEAD vs valid/ -> 5 discovered, 0 errors The corpus covers all four identity shapes including head-form-alone, both the inlined and multi-line nested forms, and the superseded .a2ml extension. Two invalid fixtures pin the token boundary: removing the ^ anchors widens what matches, and `(versioning` / `:registry-version` must still fail. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01QNjWX2B4FffG7zqMBMui6v --- .github/workflows/self-test.yml | 153 ++++++++++++++++++ CHANGELOG.adoc | 15 +- test/fixtures/README.adoc | 60 +++++++ test/fixtures/deed-only/only.deed | 5 + test/fixtures/invalid/no-version.deed | 2 + .../registry-version-is-not-version.deed | 3 + .../invalid/versioning-is-not-version.deed | 4 + test/fixtures/valid/head-form-only.deed | 5 + test/fixtures/valid/inline-nested.deed | 4 + test/fixtures/valid/keyword-identity.deed | 5 + test/fixtures/valid/legacy-superseded.a2ml | 3 + test/fixtures/valid/multiline-nested.deed | 7 + 12 files changed, 264 insertions(+), 2 deletions(-) create mode 100644 .github/workflows/self-test.yml create mode 100644 test/fixtures/README.adoc create mode 100644 test/fixtures/deed-only/only.deed create mode 100644 test/fixtures/invalid/no-version.deed create mode 100644 test/fixtures/invalid/registry-version-is-not-version.deed create mode 100644 test/fixtures/invalid/versioning-is-not-version.deed create mode 100644 test/fixtures/valid/head-form-only.deed create mode 100644 test/fixtures/valid/inline-nested.deed create mode 100644 test/fixtures/valid/keyword-identity.deed create mode 100644 test/fixtures/valid/legacy-superseded.a2ml create mode 100644 test/fixtures/valid/multiline-nested.deed diff --git a/.github/workflows/self-test.yml b/.github/workflows/self-test.yml new file mode 100644 index 0000000..b00b599 --- /dev/null +++ b/.github/workflows/self-test.yml @@ -0,0 +1,153 @@ +# SPDX-License-Identifier: MPL-2.0 +# Self-test — this action run against its own fixture corpus. +# +# Why this workflow exists +# ------------------------ +# Until v1.0.0 this action had no self-test at all, and its own tree contained +# not one `.deed` file. Its discovery glob named `*.a2ml` only, so a repository +# holding nothing but `.deed` manifests scanned zero files, reported zero +# errors and exited 0 — a silent total bypass that a green tick concealed. +# +# Two rules follow from that, and both are load-bearing here: +# +# 1. ASSERT ON DISCOVERY COUNT, NEVER ON EXIT CODE. A validator that finds +# nothing exits 0. Checking only the exit code cannot tell "all clean" +# apart from "saw nothing at all" — which is precisely the defect that +# shipped. +# 2. ALWAYS RUN A POSITIVE CONTROL. The `invalid` job must report non-zero +# errors. If it ever reports zero, the run has stopped being able to see +# failure, and every other green result in this workflow is worthless. + +name: Self-test + +on: + pull_request: + branches: ['**'] + push: + branches: [main, master] + workflow_dispatch: + +permissions: + contents: read + +jobs: + # --------------------------------------------------------------------------- + # Valid corpus — every accepted shape, including the two that broke. + # --------------------------------------------------------------------------- + valid-corpus: + name: valid corpus (strict) + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4 + - id: validate + uses: ./ + with: + path: test/fixtures/valid + strict: 'true' + - name: Assert 5 files discovered and all clean + env: + SCANNED: ${{ steps.validate.outputs.files-scanned }} + ERRORS: ${{ steps.validate.outputs.errors }} + WARNINGS: ${{ steps.validate.outputs.warnings }} + run: | + set -euo pipefail + # The count is pinned deliberately. Adding a fixture must be a + # conscious edit here, so that silent UNDER-discovery cannot pass. + if [ "${SCANNED}" != "5" ]; then + echo "::error::expected 5 files discovered, got ${SCANNED}" + exit 1 + fi + if [ "${ERRORS}" != "0" ] || [ "${WARNINGS}" != "0" ]; then + echo "::error::valid corpus must be clean, got ${ERRORS} error(s) and ${WARNINGS} warning(s)" + exit 1 + fi + echo "valid corpus: ${SCANNED} discovered, clean" + + # --------------------------------------------------------------------------- + # Invalid corpus — THE POSITIVE CONTROL. This job proves the run is capable + # of reporting a non-zero error count. Without it, every green above is + # unfalsifiable. + # + # Two of the three fixtures are token-boundary controls: `(versioning` and + # `:registry-version` must NOT satisfy the version check. Removing the `^` + # anchors in v1.0.0 widened what matches, and these fixtures are what stop + # that widening going too far. + # --------------------------------------------------------------------------- + invalid-corpus: + name: invalid corpus (positive control) + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4 + - id: validate + continue-on-error: true # the action is SUPPOSED to fail here + uses: ./ + with: + path: test/fixtures/invalid + strict: 'true' + - name: Assert every invalid fixture was caught + env: + SCANNED: ${{ steps.validate.outputs.files-scanned }} + ERRORS: ${{ steps.validate.outputs.errors }} + run: | + set -euo pipefail + if [ "${SCANNED}" != "3" ]; then + echo "::error::expected 3 files discovered, got ${SCANNED}" + exit 1 + fi + # Not merely "> 0": every fixture in this directory is invalid, so a + # count below 3 means one of them was wrongly accepted. + if [ "${ERRORS}" != "3" ]; then + echo "::error::positive control failed — expected 3 errors, got ${ERRORS}" + echo "::error::if this reads 0, the run can no longer detect failure at all" + exit 1 + fi + echo "invalid corpus: ${SCANNED} discovered, ${ERRORS} correctly rejected" + + # --------------------------------------------------------------------------- + # Discovery control — a directory containing NOT ONE `.a2ml` file. + # + # This is the regression test for the original defect. Against the pre-v1.0.0 + # glob this job reports 0 files scanned and 0 errors, and a naive exit-code + # check would call that a pass. + # --------------------------------------------------------------------------- + deed-only-discovery: + name: .deed-only discovery (regression) + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4 + - id: validate + uses: ./ + with: + path: test/fixtures/deed-only + strict: 'true' + - name: Assert the .deed file was actually seen + env: + SCANNED: ${{ steps.validate.outputs.files-scanned }} + ERRORS: ${{ steps.validate.outputs.errors }} + run: | + set -euo pipefail + if [ "${SCANNED}" != "1" ]; then + echo "::error::expected 1 file discovered, got ${SCANNED}" + echo "::error::0 here means the glob has stopped matching .deed — the v1.0.0 defect has returned" + exit 1 + fi + if [ "${ERRORS}" != "0" ]; then + echo "::error::expected the .deed fixture to validate cleanly, got ${ERRORS} error(s)" + exit 1 + fi + echo "discovery control: ${SCANNED} .deed file seen and validated" + + # --------------------------------------------------------------------------- + # Shell lint on the validator itself. + # --------------------------------------------------------------------------- + shellcheck: + name: shellcheck + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4 + - name: Lint validator scripts + run: | + set -euo pipefail + bash -n validate-a2ml.sh + bash -n validate-manifest-dialect.sh + shellcheck -S warning validate-a2ml.sh validate-manifest-dialect.sh diff --git a/CHANGELOG.adoc b/CHANGELOG.adoc index f888390..7f1bfb5 100644 --- a/CHANGELOG.adoc +++ b/CHANGELOG.adoc @@ -25,6 +25,14 @@ see a `.deed` file. * Recognition of identity by head form alone for the four ruled DEED heads (`estate-deed`, `estate-atlas-deed`, `repo-deed`, `praxis-deed`), which is how `ATLAS.deed` identifies itself -- it carries no `:canonical-name`. +* A self-test workflow (`.github/workflows/self-test.yml`) and a fixture corpus + under `test/fixtures/`. The action now runs against its own tree, which it + never did before -- the repository contained not one `.deed` file, so nothing + the action did was ever exercised against the format it is named for. The + jobs assert on discovery *count* rather than exit code, because a validator + that finds nothing exits `0`; the `invalid/` corpus is a positive control + proving the run can still report failure; and two fixtures pin the token + boundary so that removing the `^` anchors cannot widen too far. ==== Fixed @@ -50,5 +58,8 @@ see a `.deed` file. ==== Known limitations -* The action has no self-test workflow of its own, so this fix is not yet - exercised by the repository's own CI. Tracked as a follow-up. +* Roughly 120 stamped copies of this validator exist elsewhere on a separate + lineage. They still carry the old glob and are not reached by fixing this + repository. +* The Check-3 attestation exemption still matches bare basenames rather than + canonical paths. diff --git a/test/fixtures/README.adoc b/test/fixtures/README.adoc new file mode 100644 index 0000000..1ff94ed --- /dev/null +++ b/test/fixtures/README.adoc @@ -0,0 +1,60 @@ +// SPDX-License-Identifier: MPL-2.0 += Self-test fixture corpus + +Fixtures for `.github/workflows/self-test.yml`, which runs this action against +its own tree. Before v1.0.0 no such workflow existed and this directory did not +either — the action's own repository contained not one `.deed` file, so nothing +it did was ever exercised against the format it is named for. + +== The two rules this corpus enforces + +*Assert on discovery count, never on exit code.* A validator that finds nothing +exits `0`. Run the pre-v1.0.0 validator against `deed-only/` and it says: + +---- +::notice::No .a2ml files found in test/fixtures/deed-only +---- + +A `::notice::`, not a warning. Exit code `0`. No outputs set. Every workflow +downstream sees a green tick. That is the defect v1.0.0 fixes, and an exit-code +assertion cannot detect it. + +*Always run a positive control.* `invalid/` must report a non-zero error count. +If it ever reports zero, the run has lost the ability to see failure and every +other green result in the workflow is worthless. + +== `valid/` — 5 files, must be clean + +[cols="1,3"] +|=== +| `inline-nested.deed` | The inlined nested form, `(repo-deed (metadata (name "…") (version "…")))`. Both `(metadata` and `(version` sit mid-line. *This is the shape that silently failed* — the checks were anchored with `^[[:space:]]*`, and under `strict: true` that rejected a valid manifest. +| `multiline-nested.deed` | The same structure written across several lines, putting `(version` at the start of its own line. This shape always passed, which is exactly why the anchor bug stayed latent: the conformance corpus contained only this variant. A corpus that passes is not proof the regex is right; it is proof the corpus does not exercise the shape. +| `keyword-identity.deed` | Keyword identity and version — `:canonical-name` with `:schema-version`. +| `head-form-only.deed` | Identity by *head form alone*, with no `:canonical-name` anywhere. This is how `ATLAS.deed` identifies itself. The four ruled heads are enumerated, not pattern-matched. +| `legacy-superseded.a2ml` | The superseded `.a2ml` extension must still be discovered and validated. +|=== + +== `invalid/` — 3 files, must all be rejected + +[cols="1,3"] +|=== +| `no-version.deed` | Identity present, version absent. +| `versioning-is-not-version.deed` | *Token-boundary control.* `(versioning "1.0.0")` must not satisfy the version check. +| `registry-version-is-not-version.deed` | *Token-boundary control.* `:registry-version "1.0.0"` must not satisfy the version check. +|=== + +The two boundary controls exist because v1.0.0 *removed* the `^` anchors, which +widens what matches. They are what stops that widening going too far. If either +ever passes, the regex has been loosened past the token boundary. + +== `deed-only/` — 1 file, the regression test + +A directory containing not one `.a2ml` file. Against the pre-v1.0.0 glob this +scans zero files, reports zero errors and exits `0`. The job asserts +`files-scanned == 1`, so the silent bypass fails the build instead of passing it. + +== Changing the corpus + +The counts in `self-test.yml` are pinned deliberately. Adding a fixture means +editing the expected number in the same commit — so that silent *under*-discovery +cannot slip through as a pass. diff --git a/test/fixtures/deed-only/only.deed b/test/fixtures/deed-only/only.deed new file mode 100644 index 0000000..865a7dd --- /dev/null +++ b/test/fixtures/deed-only/only.deed @@ -0,0 +1,5 @@ +; SPDX-License-Identifier: MPL-2.0 +; DISCOVERY CONTROL: a directory containing NOT ONE .a2ml file. +; Before v1.0.0 the glob named *.a2ml only, so this directory scanned +; zero files, reported zero errors and exited 0 -- a silent total bypass. +(repo-deed (metadata (name "only") (version "1.0.0"))) diff --git a/test/fixtures/invalid/no-version.deed b/test/fixtures/invalid/no-version.deed new file mode 100644 index 0000000..3c47ff1 --- /dev/null +++ b/test/fixtures/invalid/no-version.deed @@ -0,0 +1,2 @@ +; SPDX-License-Identifier: MPL-2.0 +(repo-deed (metadata (name "no-version"))) diff --git a/test/fixtures/invalid/registry-version-is-not-version.deed b/test/fixtures/invalid/registry-version-is-not-version.deed new file mode 100644 index 0000000..d094364 --- /dev/null +++ b/test/fixtures/invalid/registry-version-is-not-version.deed @@ -0,0 +1,3 @@ +; SPDX-License-Identifier: MPL-2.0 +; TOKEN-BOUNDARY CONTROL: `:registry-version` must NOT satisfy the version check. +(repo-deed (metadata (name "registry-version-is-not-version")) :registry-version "1.0.0") diff --git a/test/fixtures/invalid/versioning-is-not-version.deed b/test/fixtures/invalid/versioning-is-not-version.deed new file mode 100644 index 0000000..69bf3e6 --- /dev/null +++ b/test/fixtures/invalid/versioning-is-not-version.deed @@ -0,0 +1,4 @@ +; SPDX-License-Identifier: MPL-2.0 +; TOKEN-BOUNDARY CONTROL: `(versioning` must NOT satisfy the version check. +; If this fixture ever passes, the anchors were loosened too far. +(repo-deed (metadata (name "versioning-is-not-version")) (versioning "1.0.0")) diff --git a/test/fixtures/valid/head-form-only.deed b/test/fixtures/valid/head-form-only.deed new file mode 100644 index 0000000..5d8da14 --- /dev/null +++ b/test/fixtures/valid/head-form-only.deed @@ -0,0 +1,5 @@ +; SPDX-License-Identifier: MPL-2.0 +; Identity by HEAD FORM ALONE -- no :canonical-name anywhere. This is how +; ATLAS.deed identifies itself. One of the four ruled heads. +(estate-atlas-deed + :schema-version "1.0.0") diff --git a/test/fixtures/valid/inline-nested.deed b/test/fixtures/valid/inline-nested.deed new file mode 100644 index 0000000..d803f33 --- /dev/null +++ b/test/fixtures/valid/inline-nested.deed @@ -0,0 +1,4 @@ +; SPDX-License-Identifier: MPL-2.0 +; The INLINED nested form. This is the shape that silently failed before v1.0.0: +; both `(metadata` and `(version` sit mid-line, so a `^[[:space:]]*` anchor missed them. +(repo-deed (metadata (name "inline-nested") (version "1.0.0"))) diff --git a/test/fixtures/valid/keyword-identity.deed b/test/fixtures/valid/keyword-identity.deed new file mode 100644 index 0000000..d0fb0be --- /dev/null +++ b/test/fixtures/valid/keyword-identity.deed @@ -0,0 +1,5 @@ +; SPDX-License-Identifier: MPL-2.0 +; Keyword identity + keyword version. +(repo-deed + :canonical-name "keyword-identity" + :schema-version "1.0.0") diff --git a/test/fixtures/valid/legacy-superseded.a2ml b/test/fixtures/valid/legacy-superseded.a2ml new file mode 100644 index 0000000..0e9ce1b --- /dev/null +++ b/test/fixtures/valid/legacy-superseded.a2ml @@ -0,0 +1,3 @@ +; SPDX-License-Identifier: MPL-2.0 +; The superseded .a2ml extension must still be discovered and validated. +(repo-deed (metadata (name "legacy-superseded") (version "1.0.0"))) diff --git a/test/fixtures/valid/multiline-nested.deed b/test/fixtures/valid/multiline-nested.deed new file mode 100644 index 0000000..bd3afeb --- /dev/null +++ b/test/fixtures/valid/multiline-nested.deed @@ -0,0 +1,7 @@ +; SPDX-License-Identifier: MPL-2.0 +; The MULTI-LINE nested form. This shape always passed, which is precisely why +; the anchor bug stayed latent -- the corpus only ever exercised this one. +(repo-deed + (metadata + (name "multiline-nested") + (version "1.0.0")))