From 95759ec614ec16592f3883228972578fd127fb3f Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Thu, 17 Sep 2026 22:53:57 +0000 Subject: [PATCH] =?UTF-8?q?feat(deed):=20conformance=20lane=20=E2=80=94=20?= =?UTF-8?q?grammar-faithful=20validator=20+=20spec-driven=20translator=20(?= =?UTF-8?q?campaign=20#837)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit tools/deed_lint.py: stdlib-only validator for deed.anbf v1.0.0 — header, single form with full-consumption, head whitelist, :schema-version exactly once, #t/#f only, #u5 name bodies, four-escape strings, SP/line-end/comment separators only (tabs invalid), estate-first filename dispatch with stem side conditions. Embedded 14-case corpus + fixtures/valid+invalid dirs, --self-test and --fixtures modes. tools/a2ml_to_deed.py: implements the mappings in 1-formats/deed/ mappings/ and ONLY those (P-2a table-closed fail on untabled fields). P-1 uuid re-derive-and-compare, refuse-to-emit without --beholding-chora, booleans → #t/#f, enums → symbols against closed sets, SPDX passthrough, and a lint post-condition: an invalid deed cannot be written. Modes: full/clade/agentic/neurosym/playbook translation, state-scan (family-3 ruling paperwork classifier), scorecard-scan (family-5 census + leak TSV). Reference: full six-instance translation of rsr-template-repo passes the lane end-to-end (uuid oracle: uuid5(URL, github.com/hyperpolymath/ rsr-template-repo) == a5ea1382-…, verified). Live estate recon extended the tables exactly as designed via PRs #846/#847/#848. CI: deed-conformance.yml gates self-test → fixtures → every committed *.deed on pushes/PRs touching the grammar, tools, or deeds. Refs #837 --- .github/workflows/deed-conformance.yml | 40 ++ 1-formats/deed/tools/README.adoc | 73 +++ 1-formats/deed/tools/a2ml_to_deed.py | 551 ++++++++++++++++++ 1-formats/deed/tools/deed_lint.py | 437 ++++++++++++++ .../fixtures/invalid/inequals_chora.deed | 2 + .../fixtures/invalid/inescape-u_chora.deed | 2 + .../tools/fixtures/invalid/inhead_chora.deed | 2 + .../invalid/inmissing-schema_chora.deed | 2 + .../fixtures/invalid/inno-header_chora.deed | 1 + .../fixtures/invalid/insection_chora.deed | 4 + .../tools/fixtures/invalid/intab_chora.deed | 2 + .../fixtures/invalid/intrailing_chora.deed | 2 + .../invalid/intrue-literal_chora.deed | 2 + .../fixtures/invalid/inunbalanced_chora.deed | 2 + .../fixtures/valid/booleans-uuid_chora.deed | 2 + .../tools/fixtures/valid/minimal_chora.deed | 2 + .../tools/fixtures/valid/nested_chora.deed | 5 + .../valid/quoted-list-symbols-007_chora.deed | 2 + .../valid/rsr-template-repo_chora.deed | 119 ++++ 19 files changed, 1252 insertions(+) create mode 100644 .github/workflows/deed-conformance.yml create mode 100644 1-formats/deed/tools/README.adoc create mode 100644 1-formats/deed/tools/a2ml_to_deed.py create mode 100644 1-formats/deed/tools/deed_lint.py create mode 100644 1-formats/deed/tools/fixtures/invalid/inequals_chora.deed create mode 100644 1-formats/deed/tools/fixtures/invalid/inescape-u_chora.deed create mode 100644 1-formats/deed/tools/fixtures/invalid/inhead_chora.deed create mode 100644 1-formats/deed/tools/fixtures/invalid/inmissing-schema_chora.deed create mode 100644 1-formats/deed/tools/fixtures/invalid/inno-header_chora.deed create mode 100644 1-formats/deed/tools/fixtures/invalid/insection_chora.deed create mode 100644 1-formats/deed/tools/fixtures/invalid/intab_chora.deed create mode 100644 1-formats/deed/tools/fixtures/invalid/intrailing_chora.deed create mode 100644 1-formats/deed/tools/fixtures/invalid/intrue-literal_chora.deed create mode 100644 1-formats/deed/tools/fixtures/invalid/inunbalanced_chora.deed create mode 100644 1-formats/deed/tools/fixtures/valid/booleans-uuid_chora.deed create mode 100644 1-formats/deed/tools/fixtures/valid/minimal_chora.deed create mode 100644 1-formats/deed/tools/fixtures/valid/nested_chora.deed create mode 100644 1-formats/deed/tools/fixtures/valid/quoted-list-symbols-007_chora.deed create mode 100644 1-formats/deed/tools/fixtures/valid/rsr-template-repo_chora.deed diff --git a/.github/workflows/deed-conformance.yml b/.github/workflows/deed-conformance.yml new file mode 100644 index 000000000..42a24c3f9 --- /dev/null +++ b/.github/workflows/deed-conformance.yml @@ -0,0 +1,40 @@ +# SPDX-License-Identifier: MPL-2.0 +name: deed-conformance + +# Conformance lane for the DEED grammar (standards#837). Three gates: +# validator self-test → fixture corpus → every committed *.deed file. + +on: + push: + paths: + - '**.deed' + - '1-formats/deed/**' + - '.github/workflows/deed-conformance.yml' + pull_request: + paths: + - '**.deed' + - '1-formats/deed/**' + - '.github/workflows/deed-conformance.yml' + +permissions: {} + +jobs: + lint: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 + with: + persist-credentials: false + - name: validator self-test + run: python3 1-formats/deed/tools/deed_lint.py --self-test + - name: fixture corpus (valid must pass, invalid must fail) + run: python3 1-formats/deed/tools/deed_lint.py --fixtures 1-formats/deed/tools/fixtures + - name: lint every committed deed + run: | + set -e + FILES=$(find . -name '*.deed' -not -path './1-formats/deed/tools/fixtures/invalid/*' -print) + if [ -z "$FILES" ]; then + echo "no committed .deed files yet — fixtures carry the gate" + exit 0 + fi + echo "$FILES" | xargs python3 1-formats/deed/tools/deed_lint.py diff --git a/1-formats/deed/tools/README.adoc b/1-formats/deed/tools/README.adoc new file mode 100644 index 000000000..27277c84c --- /dev/null +++ b/1-formats/deed/tools/README.adoc @@ -0,0 +1,73 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 += DEED conformance lane — tools +Campaign: standards#837 · Grammar: link:../spec/abnf/deed.anbf[spec/abnf/deed.anbf +(v1.0.0, current until the canonical-file ruling)] + +== The lane's one property + +*Nothing invalid ever lands.* Every emitted deed passes the validator before +it may be written; every `.deed` committed in this repo is linted in CI. + +== `deed_lint.py` — the validator + +Stdlib-only implementation of the normative grammar. Enforces: + +* SPDX header required; one form; full input consumed +* doc-head whitelist (`estate-deed / repo-deed / estate-atlas-deed / praxis-deed`) +* `:schema-version` STRING exactly once (validator-level semantic, per grammar) +* filename ↔ doc-head dispatch with the exact-stem-first side condition + (`estate_chora.deed` can never be a repo-deed; stems may contain dots — + split on the final suffix only) +* booleans ONLY `#t`/`#f`; bare `true/false/yes/no` values rejected +* uuid5 literals `#u5"name"` — body is the RFC 4122 §4.3 name input, never hex +* strings: exactly four legal escapes (`\" \\ \n \t`); `\r`/`\uXXXX` are + parse errors; raw control bytes rejected; spaces INSIDE symbols impossible +* separators are SP / line-end / comment ONLY — tabs are invalid everywhere + (K9-consistent); bare CR rejected (CRLF or LF line-ends only) +* quoted values restricted to symbols and lists + +Usage: + + python3 deed_lint.py FILE.deed... + python3 deed_lint.py --self-test # 14-case embedded corpus + python3 deed_lint.py --fixtures fixtures/ # valid/ must pass, invalid/ must fail + +== `a2ml_to_deed.py` — the translator + +Implements ONE authoritative mapping: the specs in `../mappings/`. It is +deliberately unable to invent mappings — untabled fields fail closed: + +* P-1 uuid: re-derived (RFC 4122 §4.3, URL namespace) and compared against + the instance; mismatch → no output. Never copied. +* `--beholding-chora NAME` required: refuse-to-emit otherwise (no invented + identifiers). +* Booleans `true/false → #t/#f`; enums → symbols against closed sets; + empty slots kept as greppable `""` / `()` per slot doctrine. +* SPDX headers pass through (translation changes grammar, not licence). +* POST-CONDITION: emitted text is linted; failure writes nothing. + + a2ml_to_deed.py full --canonical-name NAME --beholding-chora C [--out F] + a2ml_to_deed.py clade|agentic|neurosym|playbook FILE.a2ml --canonical-name N --beholding-chora C + a2ml_to_deed.py state-scan FILE... # family-3 classification (no translation; ruling pending #843) + a2ml_to_deed.py scorecard-scan DIR # family-5 corpus census + absolute-path leak TSV + +Reference translation of the estate's canonical template passes the lane +(six instance files → one deed, uuid oracle verified). + +Working estate instances surfaced *live* during development and fixed by +table extension, exactly as the frame prescribes (fail → review → extend): +`[axis-3-audit-rules]`, `[methodology]`, `[rsr-repo-skeleton]`, plus one +genuine registry deficiency flagged, not patched (standards' own CLADE +lacks `primary-name` — CLADE-006 territory, reported on #837). + +== Fixtures + +`fixtures/valid/` and `fixtures/invalid/` mirror the embedded self-test +corpus plus one real translated deed (`valid/rsr-template-repo_chora.deed`). +Reload rule: fixtures are generated FROM the `_FIXTURES` table in +`deed_lint.py` — edit the table, regenerate the files, commit both. + +== CI + +`deed-conformance.yml` runs on any change touching `**.deed`, the grammar, +or these tools: self-test → fixtures → lint every committed `*.deed` file. diff --git a/1-formats/deed/tools/a2ml_to_deed.py b/1-formats/deed/tools/a2ml_to_deed.py new file mode 100644 index 000000000..01ed70488 --- /dev/null +++ b/1-formats/deed/tools/a2ml_to_deed.py @@ -0,0 +1,551 @@ +#!/usr/bin/env python3 +"""a2ml→deed translator engine (campaign #837, conformance-lane half). + +STDLIB-ONLY. Everything emitted is validated by deed_lint BEFORE it may be +written — an invalid deed can never leave the tool (the lane property). + +Rules enforced (from the mapping specs in 1-formats/deed/mappings/): + * `--beholding-chora NAME` is REQUIRED for real emission (refuse-to-emit + otherwise; grammar: a repo-deed MUST name the chora it reads, and the + body of #u5 literals is the RFC 4122 §4.3 NAME, never hex). + * CLADE uuid: re-derived via RFC 4122 §4.3 (URL namespace) and compared + against the instance — fail-closed on mismatch, never copied (P-1). + * Booleans: a2ml true/false → deed #t/#f (only legal booleans). + * Enums → symbols against closed sets; any untabled field FAILS the + translation (P-2a table-closed) — it never passes through on guesswork. + * Strings emit with exactly the four legal escapes; \\r or control bytes + fail closed. + * SPDX header lines of the a2ml source pass through as the deed's header + (translation changes grammar, not licence facts). + * STATE (family 3): classification report only — no translation pending + the owner's ruling (#843). + * Scorecards (family 5): scan/report only — no translation (#845), plus + the absolute-path leak list. + +Usage: + a2ml_to_deed.py clade|meta-ecosystem|agentic|neurosym|playbook + FILE.a2ml --canonical-name N --beholding-chora C [--out F] + a2ml_to_deed.py full DESCRITILES_DIR --canonical-name N --beholding-chora C [--out F] + a2ml_to_deed.py state-scan FILE... + a2ml_to_deed.py scorecard-scan DIR +""" +import argparse +import os +import re +import sys +import tomllib +import uuid + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +import deed_lint + +# ---------------------------------------------------------------- helpers + +SYMBOL_RE = re.compile(r"[A-Za-z][A-Za-z0-9.*/<>=!?+_-]*\Z") + + +class TranslateError(Exception): + pass + + +def esc(s): + """Emit one legal deed string literal; fail closed on un-representable bytes.""" + if not isinstance(s, str): + raise TranslateError(f"expected str, got {type(s).__name__}") + for ch in s: + if ch in "\r" or (ord(ch) < 0x20 and ch not in "\n\t"): + raise TranslateError(f"source string contains un-representable control byte U+{ord(ch):04X}") + body = s.replace("\\", "\\\\").replace('"', '\\"').replace("\n", "\\n").replace("\t", "\\t") + return f'"{body}"' + + +def sym(s, what="symbol", closed=None): + if not isinstance(s, str) or not SYMBOL_RE.fullmatch(s): + raise TranslateError(f"{what}: {s!r} is not a legal deed symbol") + if closed is not None and s not in closed: + raise TranslateError(f"{what}: {s!r} is not in the closed set {sorted(closed)} — table-closed fail") + return s + + +def val(v, what="value", closed=None): + if isinstance(v, bool): + return "#t" if v else "#f" + if isinstance(v, int): + return str(v) + if isinstance(v, str): + if closed is not None: + return sym(v, what, closed) + return esc(v) + if isinstance(v, list): + return "(" + " ".join(val(x, what) for x in v) + ")" + raise TranslateError(f"{what}: unsupported type {type(v).__name__}") + + +def spdx_of(path, profile=None): + headers = [] + with open(path, encoding="utf-8") as fh: + for line in fh: + m = re.match(r"^;;?\s*(SPDX-\S.*)$", line.rstrip("\n")) or re.match( + r"^#\s*(SPDX-\S.*)$", line.rstrip("\n") + ) + if m: + headers.append(";; " + m.group(1)) + elif line.strip() and not line.startswith("#"): + break + if not headers: + headers = [";; SPDX-License-Identifier: MPL-2.0"] + return headers + + +def parse_a2ml(path): + src = open(path, encoding="utf-8").read() + prof = re.findall(r"^\s*@profile\(\s*id\s*=\s*([^\s)]+)\s*\)", src, flags=re.M) + body = re.sub(r"^\s*@profile\([^\n]*\)\s*\n", "", src, flags=re.M) + try: + return tomllib.loads(body), (prof[0] if prof else None) + except tomllib.TOMLDecodeError as e: + raise TranslateError(f"source does not parse as a2ml/TOML ({e}) — P-2 fail-closed") + + +def require(data, section, keys, fam, required=None, defaults=None): + if section not in data: + raise TranslateError(f"{fam}: required section [{section}] missing") + unknown = set(data[section]) - set(keys) + if unknown: + raise TranslateError(f"{fam}: untabled field(s) {sorted(unknown)} in [{section}] — P-2a fail-closed") + for k in (required or []): + if k not in data[section]: + raise TranslateError(f"{fam}: required field {section}.{k} missing (registry gate territory — instance deficiency)") + merged = dict(defaults or {}) + merged.update(data[section]) + return merged + + +CLADES = { + "fv", "nl", "rm", "gv", "db", "ap", "ix", "dx", "pt", "ax", "gm", "sc", "UNASSIGNED" +} +LINEAGE = {"standalone", "monorepo", "monorepo-child", "inflated", "deflated", "hub", "satellite"} +PHASES = {"reserved", "incubating", "active", "dormant", "merged", "superseded", "archived", "extinct"} +META_TYPES = {"library", "binary", "service", "website", "monorepo"} +PKGMGR = {"guix", "cargo", "mix"} +RELATIONSHIPS = { + "standard-source", "build-tooling", "validation-tooling", "signing-tooling", "verification-tooling" +} +SCAN_DEPTH = {"quick", "standard", "deep"} + +ALLOWED_SECTIONS = { + "clade-family": {"identity", "clade", "forges", "lineage", "status"}, + "meta": {"metadata", "project-info", "development-practices", "maintenance-axes", + "scoping", "axis-2-maintenance-rules", "architecture-decisions", + "axis-3-audit-rules", "design-rationale"}, + "ecosystem": {"metadata", "position", "pipeline", "related-projects"}, +} + + +def emit_clade_clauses(path, data): + extra = set(data) - {"identity", "clade", "forges", "lineage", "status"} + if extra: + raise TranslateError(f"clade: unknown top-level section(s) {sorted(extra)} — fail-closed") + ident = require(data, "identity", + ["uuid", "primary-forge", "primary-owner", "canonical-name", "prefixed-name"], + "clade", required=["uuid", "primary-forge", "primary-owner", "canonical-name"], + defaults={"prefixed-name": ""}) + owner, name = ident["primary-owner"], ident["canonical-name"] + derived = uuid.uuid5(uuid.NAMESPACE_URL, f"github.com/{owner}/{name}") + if str(derived) != ident["uuid"]: + raise TranslateError( + f"P-1 fail-closed: re-derived uuid {derived} != stored {ident['uuid']} — never copied, never emitted" + ) + cl = [] + cl.append(" (identity :primary-forge " + sym(ident["primary-forge"], "identity.primary-forge")) + cl.append(" :owner " + esc(owner)) + cl.append(" :prefixed-name " + esc(ident["prefixed-name"]) + ")") + c = require(data, "clade", ["primary", "primary-name", "secondary", "assigned", "rationale"], + "clade", required=["primary", "primary-name"], + defaults={"secondary": [], "assigned": "", "rationale": ""}) + cl.append("") + cl.append(" (clade :primary " + sym(c["primary"], "clade.primary", CLADES)) + cl.append(" :primary-name " + esc(c["primary-name"])) + cl.append(" :secondary (" + " ".join(sym(x, "clade.secondary", CLADES) for x in c["secondary"]) + ")") + cl.append(" :assigned " + esc(c["assigned"])) + cl.append(" :rationale " + esc(c["rationale"]) + ")") + fg = require(data, "forges", ["github", "gitlab", "bitbucket"], "clade", + required=["github"], defaults={"gitlab": "", "bitbucket": ""}) + cl.append("") + cl.append(" (forges :github " + esc(fg["github"])) + cl.append(" :gitlab " + esc(fg["gitlab"])) + cl.append(" :bitbucket " + esc(fg["bitbucket"]) + ")") + ln = require(data, "lineage", ["type", "parent", "born", "previous-names", "instantiated-from"], + "clade", required=["type"], defaults={"parent": "", "born": "", "previous-names": [], "instantiated-from": ""}) + cl.append("") + cl.append(" (lineage :type " + sym(ln["type"], "lineage.type", LINEAGE)) + cl.append(" :parent " + esc(ln["parent"])) + cl.append(" :born " + esc(ln["born"])) + cl.append(" :previous-names " + val(ln["previous-names"])) + cl.append(" :instantiated-from " + esc(ln["instantiated-from"]) + ")") + st = require(data, "status", + ["phase", "since", "present", "aliases", "merged-into", "superseded-by", "successors", "ended", "history"], + "clade", required=["phase", "present"], + defaults={"since": "", "aliases": [], "merged-into": "", "superseded-by": "", "successors": [], "ended": "", "history": []}) + cl.append("") + cl.append(" (status :phase " + sym(st["phase"], "status.phase", PHASES)) + cl.append(" :since " + esc(st["since"])) + cl.append(" :present " + val(st["present"])) + cl.append(" :aliases " + val(st["aliases"])) + cl.append(" :merged-into " + esc(st["merged-into"])) + cl.append(" :superseded-by " + esc(st["superseded-by"])) + cl.append(" :successors " + val(st["successors"])) + cl.append(" :ended " + esc(st["ended"])) + hist = st.get("history") or [] + if hist: + cl.append(" (history") + for h in hist: + unknown = set(h) - {"phase", "since", "note"} + if unknown: + raise TranslateError(f"clade: history entry has untabled key(s) {sorted(unknown)}") + missing = set(("phase", "since", "note")) - set(h) + if missing: + raise TranslateError(f"clade: history entry missing required key(s) {sorted(missing)}") + cl.append(" (entry :phase " + sym(h["phase"], "history.phase", PHASES)) + cl.append(" :since " + esc(h["since"])) + cl.append(" :note " + esc(h["note"]) + ")") + cl.append(" ))") + else: + cl[-1] = cl[-1] + ")" + return ident["canonical-name"], "\n".join(cl) + + + + +def emit_meta_clause(data): + extra = set(data) - ALLOWED_SECTIONS["meta"] + if extra: + raise TranslateError(f"meta: unknown section(s) {sorted(extra)} — fail-closed") + L = [" (meta"] + md = require(data, "metadata", ["version", "last-updated"], "meta") + L.append(f" :version {esc(md['version'])}") + L.append(f" :last-updated {esc(md['last-updated'])}") + pi = require(data, "project-info", ["type", "languages", "license", "author"], "meta") + L.append(f" :type {sym(pi['type'], 'project-info.type', META_TYPES)}") + L.append(f" :languages ({' '.join(sym(x, 'languages') for x in pi['languages'])})") + L.append(f" :license {sym(pi['license'], 'project-info.license')}") + L.append(f" :author {esc(pi['author'])}") + dp = require(data, "development-practices", ["build-tool", "container-runtime", "ci-platform", "package-manager"], "meta") + L.append(f" :build-tool {sym(dp['build-tool'])}") + L.append(f" :container-runtime {sym(dp['container-runtime'])}") + L.append(f" :ci-platform {sym(dp['ci-platform'])}") + L.append(f" :package-manager {sym(dp['package-manager'], 'package-manager', PKGMGR)}") + ma = require(data, "maintenance-axes", ["scoping-first", "execution-order", "axis-1", "axis-2", "axis-3"], "meta") + L.append(f" :scoping-first {val(ma['scoping-first'])}") + for k in ("execution-order", "axis-1", "axis-2", "axis-3"): + L.append(f" :{k} {esc(ma[k])}" if k != "axis-1" else f" :{k} {esc(ma[k])}") + sc = require(data, "scoping", ["sources", "marker-scan", "idris-unsound-scan"], "meta") + L.append(" (scoping") + L.append(f" :sources {esc(sc['sources'])}") + L.append(f" :marker-scan {esc(sc['marker-scan'])}") + L.append(f" :idris-unsound-scan {esc(sc['idris-unsound-scan'])})") + a2 = require(data, "axis-2-maintenance-rules", + ["corrective-first", "adaptive-second", "adaptive-focus", "perfective-third", "perfective-source"], "meta") + L.append(f" :corrective-first {val(a2['corrective-first'])}") + L.append(f" :adaptive-second {val(a2['adaptive-second'])}") + L.append(f" :adaptive-focus {esc(a2['adaptive-focus'])}") + L.append(f" :perfective-third {val(a2['perfective-third'])}") + L.append(f" :perfective-source {esc(a2['perfective-source'])}") + a3 = data.get("axis-3-audit-rules") or {} + if a3: + merged3 = require(data, "axis-3-audit-rules", + ["audit-focus", "compliance-focus", "drift-risk-example", "effects-evidence"], "meta", + defaults={"audit-focus": "", "compliance-focus": "", "drift-risk-example": "", "effects-evidence": ""}) + L.append(" (axis-3") + L.append(f" :audit-focus {esc(merged3['audit-focus'])}") + L.append(f" :compliance-focus {esc(merged3['compliance-focus'])}") + L.append(f" :drift-risk-example {esc(merged3['drift-risk-example'])}") + L.append(f" :effects-evidence {esc(merged3['effects-evidence'])})") + adrs = (data.get("architecture-decisions") or {}).get("adr") or [] + ADR_STATUSES = {"proposed", "accepted", "deprecated", "superseded", "rejected"} + for adr in adrs: + unknown = set(adr) - {"id", "title", "status", "date"} + if unknown: + raise TranslateError(f"meta: ADR entry untabled key(s) {sorted(unknown)}") + L.append(f" (adr :id {esc(adr['id'])} :title {esc(adr['title'])} :status {sym(adr['status'], 'adr.status', ADR_STATUSES)} :date {esc(adr['date'])})") + if any((data.get("architecture-decisions") or {}).keys() - {"adr"}): + raise TranslateError("meta: [architecture-decisions] has non-adr keys — fail-closed (family-2 §3)") + if data.get("design-rationale"): + raise TranslateError("meta: [design-rationale] populated — fail-closed pending table rows (family-2 §3)") + return "\n".join(L) + ")" + + +def emit_ecosystem_clause(data, canonical_name): + extra = set(data) - ALLOWED_SECTIONS["ecosystem"] + if extra: + raise TranslateError(f"ecosystem: unknown section(s) {sorted(extra)} — fail-closed") + md = require(data, "metadata", ["project", "ecosystem"], "ecosystem") + if md["project"] != canonical_name: + raise TranslateError( + f"P-6 fail-closed: ecosystem project {md['project']!r} != deed canonical-name {canonical_name!r}" + ) + pos = require(data, "position", ["type", "purpose", "what-this-is-not"], "ecosystem") + pipe = require(data, "pipeline", ["position", "chain", "notes", "coordination"], "ecosystem") + rp = require(data, "related-projects", ["projects"], "ecosystem") + L = [" (ecosystem"] + L.append(f" :project {esc(md['project'])}") + L.append(f" :ecosystem {esc(md['ecosystem'])}") + L.append(f" :position-type {esc(pos['type'])}") + L.append(f" :purpose {esc(pos['purpose'])}") + L.append(" :not (" + " ".join(esc(x) for x in pos["what-this-is-not"]) + ")") + L.append(f" :pipeline-position {esc(pipe['position'])}") + L.append(f" :chain {esc(pipe['chain'])}") + L.append(f" :pipeline-notes {esc(pipe['notes'])}") + coord = pipe["coordination"] + L.append(" :coordination " + (sym(coord, "coordination") if coord else '""')) + for r in rp["projects"]: + unknown = set(r) - {"name", "relationship", "notes"} + if unknown: + raise TranslateError(f"ecosystem: related-projects entry untabled key(s) {sorted(unknown)}") + L.append(f" (related :name {esc(r['name'])} :relationship {sym(r['relationship'], 'relationship', RELATIONSHIPS)} :notes {esc(r['notes'])})") + if rp["projects"]: + L[-1] = L[-1] + ")" + else: + L[-1] = L[-1] + ")" + return "\n".join(L) + + +def emit_profiled_clause(fam, data, profile): + if fam == "agentic": + extra = set(data) - {"metadata", "agent-permissions", "agent-constraints", "maintenance-integrity", "automation-hooks", "methodology"} + if extra: + raise TranslateError(f"agentic: unknown section(s) {sorted(extra)}") + md = require(data, "metadata", ["version", "last-updated"], "agentic") + ap = require(data, "agent-permissions", + ["can-edit-source", "can-edit-tests", "can-edit-docs", "can-edit-config", "can-create-files"], "agentic") + mi = require(data, "maintenance-integrity", + ["fail-closed", "require-evidence-per-step", "allow-silent-skip", + "require-rerun-after-fix", "release-claim-requires-hard-pass"], "agentic") + for sec in ("agent-constraints", "automation-hooks"): + if data.get(sec): + raise TranslateError( + f"agentic: [{sec}] carries FIELDS — fail-closed (D4-1: comment-canon prose must stay prose; divergent field content earns its own table rows)") + L = [f" (agentic (profile :id {esc(profile)})" if profile else " (agentic", + f" :version {esc(md['version'])} :last-updated {esc(md['last-updated'])}", + " (permissions :source " + val(ap["can-edit-source"]), + " :tests " + val(ap["can-edit-tests"]), + " :docs " + val(ap["can-edit-docs"]), + " :config " + val(ap["can-edit-config"]), + " :create-files " + val(ap["can-create-files"]) + ")", + " (integrity :fail-closed " + val(mi["fail-closed"]), + " :require-evidence-per-step " + val(mi["require-evidence-per-step"]), + " :allow-silent-skip " + val(mi["allow-silent-skip"]), + " :require-rerun-after-fix " + val(mi["require-rerun-after-fix"]), + " :release-claim-requires-hard-pass " + val(mi["release-claim-requires-hard-pass"]) + ")"] + mo = data.get("methodology") + if mo is not None: + mo = require(data, "methodology", ["instructions-dir", "default-mode"], "agentic", + defaults={"instructions-dir": "", "default-mode": ""}) + line = " (methodology" + if mo["instructions-dir"]: + line += f" :instructions-dir {esc(mo['instructions-dir'])}" + if mo["default-mode"]: + line += f" :default-mode {sym(mo['default-mode'], 'methodology.default-mode')}" + L.append(line + ")") + L[-1] = L[-1] + ")" + else: + L[-1] = L[-1] + ")" + return "\n".join(L) + if fam == "neurosym": + extra = set(data) - {"metadata", "hypatia-config", "symbolic-rules", "neural-config"} + if extra: + raise TranslateError(f"neurosym: unknown section(s) {sorted(extra)}") + md = require(data, "metadata", ["version", "last-updated"], "neurosym") + hc = require(data, "hypatia-config", ["scan-enabled", "scan-depth", "report-format"], "neurosym") + for sec in ("symbolic-rules", "neural-config"): + if data.get(sec): + raise TranslateError(f"neurosym: [{sec}] populated — fail-closed pending table rows") + L = [f" (neurosym (profile :id {esc(profile)})" if profile else " (neurosym", + f" :version {esc(md['version'])} :last-updated {esc(md['last-updated'])}", + " (hypatia :scan-enabled " + val(hc["scan-enabled"]), + " :scan-depth " + sym(hc["scan-depth"], "scan-depth", SCAN_DEPTH), + f" :report-format {esc(hc['report-format'])}))"] + return "\n".join(L) + if fam == "playbook": + extra = set(data) - {"metadata", "deployment", "incident-response", "release-process", + "docs-format", "maintenance-operations", "rsr-repo-skeleton"} + if extra: + raise TranslateError(f"playbook: unknown section(s) {sorted(extra)}") + md = require(data, "metadata", ["version", "last-updated"], "playbook") + for sec in ("deployment", "incident-response", "release-process", "docs-format", "maintenance-operations"): + if data.get(sec): + raise TranslateError(f"playbook: [{sec}] populated — fail-closed pending table rows") + skel = "" + if "rsr-repo-skeleton" in data: + sk = require(data, "rsr-repo-skeleton", + ["skeleton-version", "last-updated", "authority-allowlist", "enforcement-workflow"], + "playbook", required=["skeleton-version"], + defaults={"last-updated": "", "authority-allowlist": "", "enforcement-workflow": ""}) + skel = (" (skeleton :version " + esc(sk["skeleton-version"]) + + "\n :last-updated " + esc(sk["last-updated"]) + + "\n :authority-allowlist " + esc(sk["authority-allowlist"]) + + "\n :enforcement-workflow " + esc(sk["enforcement-workflow"]) + ")") + L = [f" (playbook (profile :id {esc(profile)})" if profile else " (playbook", + f" :version {esc(md['version'])} :last-updated {esc(md['last-updated'])}"] + close = " (deployment) (incident-response) (release-process) (docs-format) (maintenance-operations))" + if skel: + L.append(skel) + L.append(close) + return "\n".join(L) + raise TranslateError(f"unknown family {fam!r}") + + +def compose_deed(headers, canonical_name, beholding, clauses, out_name=None): + body = "\n\n".join(c for c in clauses if c) + text = "\n".join(headers) + "\n" + \ + "(repo-deed\n" + \ + ' :schema-version "1.0.0"\n' + \ + f" :canonical-name {esc(canonical_name)}\n" + \ + f" :beholding-chora #u5{esc(beholding)}\n\n" + \ + body + "\n)\n" + fname = out_name or f"{canonical_name}_chora.deed" + try: + deed_lint.validate(text, filename=fname) + except deed_lint.LintError as e: + raise TranslateError(f"POST-CONDITION FAILED — emitted deed does not conform ({e}); nothing written") + return text, fname + + +def state_report(path, data): + statal, journal, derivable, collision = [], [], [], [] + md = data.get("metadata", {}) + pc = data.get("project-context", {}) + pos = data.get("position", {}) + if "status" in md: + collision.append(f'[metadata] status={md["status"]!r} — duplicates CLADE status.phase (CLADE wins, ruled vocabulary)') + if "phase" in pos: + statal.append(f'[position] phase={pos["phase"]!r} → (status …) clause') + if "maturity" in pos: + statal.append(f'[position] maturity={pos["maturity"]!r} → proposed (status … :maturity …) — one-field vocabulary extension, in the ruling request') + if "completion-percentage" in pc: + derivable.append(f'[project-context] completion-percentage={pc["completion-percentage"]} — derivable from milestone rows; dropped by v2 doctrine') + for sec in ("route-to-mvp", "blockers-and-issues", "critical-next-actions", "maintenance-status", "ecosystem"): + if data.get(sec): + n = len(data[sec].get("milestones", data[sec].get("actions", []))) if isinstance(data[sec], dict) else 0 + journal.append(f"[{sec}] — journal content ({n} rows); does NOT translate under option B; tombstoned to archive") + return f"""STATE v1 classification — {path} + STATAL (extract to CLADE status clause): {chr(10).join(' - ' + s for s in statal) or ' - none'} + COLLISIONS: {chr(10).join(' - ' + s for s in collision) or ' - none'} + DERIVABLE (dropped): {chr(10).join(' - ' + s for s in derivable) or ' - none'} + JOURNAL (archive, not translate): {chr(10).join(' - ' + s for s in journal) or ' - none'} + Ruling pending: standards#843 (family-3 decision spec).""" + + +def scorecard_scan(root): + rows = [] + for dirpath, _, files in os.walk(root): + for f in sorted(files): + if not f.endswith(".scorecard.a2ml"): + continue + p = os.path.join(dirpath, f) + data, _ = parse_a2ml(p) + sc = data.get("scorecard", {}) + counts = {} + leaks = 0 + for tier in ("must", "should", "could"): + entries = data.get(tier) or [] + pas = sum(1 for e in entries if e.get("status") == "pass") + fail = sum(1 for e in entries if e.get("status") == "fail") + counts[tier] = (pas, fail) + for e in entries: + for v in e.values(): + if isinstance(v, str) and re.search(r"/home/[A-Za-z0-9._-]+/", v): + leaks += 1 + rows.append((f, sc.get("spec_id", "?"), sc.get("assessed_date", "?"), sc.get("assessor", "?"), + counts["must"], counts["should"], counts["could"], leaks)) + out = ["file\tspec_id\tassessed_date\tassessor\tmust_pass\tmust_fail\tshould_pass\tshould_fail\tcould_pass\tcould_fail\tabsolute_path_leak_lines"] + out += ["\t".join(map(str, r[:4])) + "\t" + "\t".join(map(str, sum((list(c) for c in r[4:7]), []))) + f"\t{r[7]}" + for r in rows] + return "\n".join(out) + + +def main(argv): + ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("mode", choices=["clade", "meta-ecosystem", "agentic", "neurosym", "playbook", + "full", "state-scan", "scorecard-scan"]) + ap.add_argument("inputs", nargs="+") + ap.add_argument("--canonical-name") + ap.add_argument("--beholding-chora") + ap.add_argument("--out") + a = ap.parse_args(argv[1:]) + + try: + if a.mode == "scorecard-scan": + print(scorecard_scan(a.inputs[0])) + return 0 + if a.mode == "state-scan": + for p in a.inputs: + data, _ = parse_a2ml(p) + print(state_report(p, data)) + return 0 + for req in ("canonical_name", "beholding_chora"): + if not getattr(a, req): + print(f"error: --{req.replace('_', '-')} is required (refuse-to-emit)", file=sys.stderr) + return 2 + clauses = [] + headers = None + if a.mode == "full": + from pathlib import Path + d = Path(a.inputs[0]) + found = {} + for ext in (".machine_readable/descriptiles", ".machine_readable"): + for f in ("CLADE", "META", "ECOSYSTEM", "AGENTIC", "NEUROSYM", "PLAYBOOK"): + cand = d / ext / f"{f}.a2ml" + if cand.exists() and f not in found: + found[f] = cand + if "CLADE" not in found: + raise TranslateError(f"full: no CLADE.a2ml under {d}") + cpath = str(found["CLADE"]) + data, _ = parse_a2ml(cpath) + can, cl = emit_clade_clauses(cpath, data) + if can != a.canonical_name: + raise TranslateError(f"--canonical-name {a.canonical_name!r} != CLADE canonical-name {can!r}") + headers = spdx_of(cpath) + clauses.append(cl) + cname = can + if "META" in found: + clauses.append(emit_meta_clause(parse_a2ml(str(found["META"]))[0])) + if "ECOSYSTEM" in found: + clauses.append(emit_ecosystem_clause(parse_a2ml(str(found["ECOSYSTEM"]))[0], cname)) + for fam, key in (("AGENTIC", "agentic"), ("NEUROSYM", "neurosym"), ("PLAYBOOK", "playbook")): + if fam in found: + d2, prof = parse_a2ml(str(found[fam])) + clauses.append(emit_profiled_clause(key, d2, prof)) + # clade emit already includes identity/clade/forges/lineage/status; + # head needs :repo-uuid injected before identity clause: + cl0 = data["identity"] + clauses[0] = (' :repo-uuid #u5"github.com/%s/%s"\n\n' % (cl0["primary-owner"], cname)) + clauses[0] + else: + path = a.inputs[0] + data, prof = parse_a2ml(path) + headers = spdx_of(path) + if a.mode == "clade": + cname, cl0 = emit_clade_clauses(path, data) + if cname != a.canonical_name: + raise TranslateError(f"--canonical-name mismatch: {a.canonical_name} vs {cname}") + clauses.append(' :repo-uuid #u5"github.com/%s/%s"\n\n' % (data["identity"]["primary-owner"], cname) + cl0) + elif a.mode == "meta-ecosystem": + raise TranslateError("use 'full DESCRITILES_DIR' (meta and ecosystem join the repo deed, they are not standalone docs)") + else: + clauses.append(emit_profiled_clause(a.mode, data, prof)) + cname = a.canonical_name + text, fname = compose_deed(headers, a.canonical_name, a.beholding_chora, clauses, out_name=a.out) + if a.out: + with open(a.out, "w", encoding="utf-8") as fh: + fh.write(text) + print(f"EMITTED {a.out} (validated by deed_lint, dispatch {fname})") + else: + sys.stdout.write(text) + return 0 + except (TranslateError, deed_lint.LintError) as e: + print(f"FAIL-CLOSED: {e}", file=sys.stderr) + return 1 + + +if __name__ == "__main__": + sys.exit(main(sys.argv)) diff --git a/1-formats/deed/tools/deed_lint.py b/1-formats/deed/tools/deed_lint.py new file mode 100644 index 000000000..aea4ccdc0 --- /dev/null +++ b/1-formats/deed/tools/deed_lint.py @@ -0,0 +1,437 @@ +#!/usr/bin/env python3 +"""deed-lint — conformance validator for the DEED grammar (deed.anbf v1.0.0). + +Stdlib-only. Implements the normative grammar faithfully: + * header = 1* spdx-line (";;" SP "SPDX-" …) + * form = "(" doc-head 1*(sep (field/clause)) [sep] ")" + * doc-head ∈ {estate-deed, repo-deed, estate-atlas-deed, praxis-deed} + * field = keyword sep value clause = "(" symbol *(sep (field/clause)) [sep] ")" + * value = string / symbol / integer / boolean / uuid5 / quoted / list + * boolean = #t | #f (lowercase ONLY; true/false/1/0 are parse errors) + * uuid5 = %s"#u5" string (body is the RFC 4122 §4.3 NAME input) + * escapes exactly {" \\\\ \\n \\t} (\\r, \\uXXXX and all others INVALID) + * sep = 1*(SP / line-end / comment) (HTAB is INVALID — K9-consistent) + * symbol = ALPHA *(ALPHA/DIGIT/./"*"//"<"/">"/"="/"!"/"?"/"+"/"-"/"_") + * quoted = "'" (symbol / list) (only symbols/lists may be quoted) +Semantic checks (validator-enforced, per grammar): + * :schema-version STRING exactly once at form top level + * entire input consumed + * filename↔doc-head dispatch (estate-first side condition, stem≠"estate") + +Usage: deed_lint.py FILE... exit 0 iff all files conform + deed_lint.py --self-test +""" +import os +import re +import sys + +__all__ = ["validate", "LintError", "check_filename_dispatch"] + +SYMBOL_START = re.compile(r"[A-Za-z]") +SYMBOL_CONT = re.compile(r"[A-Za-z0-9.*/<>=!?+_-]") + + +class LintError(ValueError): + """A single conformance failure with a 1-based line number.""" + + def __init__(self, msg, line): + super().__init__(msg) + self.msg, self.line = msg, line + + def __str__(self): + return f"line {self.line}: {self.msg}" + + +class _Lexer: + def __init__(self, text): + self.t = text + self.i = 0 + self.n = len(text) + + def line(self, at=None): + return self.t.count("\n", 0, self.i if at is None else at) + 1 + + def peek(self, k=0): + j = self.i + k + return self.t[j] if j < self.n else "" + + def skip_sep(self): + """token-sep = 1*(SP / line-end / comment). Returns #separators seen.""" + seen = 0 + while self.i < self.n: + c = self.t[self.i] + if c == " ": + self.i += 1 + seen = 1 + elif c == "\n": + self.i += 1 + seen = 1 + elif c == "\r": + if self.t[self.i : self.i + 2] == "\r\n": + self.i += 2 + seen = 1 + else: + raise LintError("bare CR is not a line-end (CRLF or LF only)", self.line()) + elif c == ";": + while self.i < self.n and self.t[self.i] not in "\r\n": + self.i += 1 + seen = 1 # comment; the terminating line-end is consumed next loop + else: + break + return seen + + +def _lex_string(lx): + """string = DQUOTE *( str-char / escape ) DQUOTE ; exactly 4 escapes.""" + start = lx.i + out = [] + lx.i += 1 # opening quote + while True: + if lx.i >= lx.n: + raise LintError("unterminated string", lx.line(start)) + c = lx.t[lx.i] + if c == '"': + lx.i += 1 + return ("string", "".join(out)) + if c == "\\": + if lx.i + 1 >= lx.n: + raise LintError("dangling backslash", lx.line(start)) + e = lx.t[lx.i + 1] + if e not in ('"', "\\", "n", "t"): + raise LintError( + f"illegal escape \\{e!r} — only \\\" \\\\ \\n \\t exist in DEED " + "(\\r and \\uXXXX are parse errors; embed non-ASCII as raw UTF-8)", + lx.line(start), + ) + out.append({"n": "\n", "t": "\t"}.get(e, e)) + lx.i += 2 + continue + o = ord(c) + if o < 0x20 or c == '"' or c == "\\": + raise LintError(f"raw control character U+{o:04X} inside string (use legal escapes)", lx.line()) + out.append(c) + lx.i += 1 + + +def _lex_number(lx): + m = re.match(r"-?[0-9]+", lx.t[lx.i :]) + if m: + lx.i += m.end() + return ("integer", None) + return None + + +def _lex_symbol(lx): + if not SYMBOL_START.match(lx.peek()): + return None + j = lx.i + 1 + while j < lx.n and SYMBOL_CONT.match(lx.t[j]): + j += 1 + s = lx.t[lx.i : j] + lx.i = j + return ("symbol", s) + + +def _lex_value(lx): + """value = string / symbol / integer / boolean / uuid5 / quoted / list""" + c = lx.peek() + if c == '"': + return _lex_string(lx) + if c == "#": + two = lx.t[lx.i : lx.i + 3] + if two.startswith("#u5"): + lx.i += 3 + if lx.peek() != '"': + raise LintError('uuid5 must be followed immediately by a string: #u5"name"', lx.line()) + _lex_string(lx) + return ("uuid5", None) + if two[:2] in ("#t", "#f"): + nxt = lx.peek(2) + if nxt and nxt not in " \r\n()": + raise LintError( + f"booleans are exactly #t/#f (got {two + lx.t[lx.i+2:lx.i+24].split()[0][:20]!r}); " + "true/false/1/0 are parse errors", + lx.line(), + ) + lx.i += 2 + return ("boolean", two) + raise LintError(f"unrecognised #-form: only #t, #f, #u5\"…\" are legal", lx.line()) + if c == "(": + lx.i += 1 + items = [] + while True: + have = lx.skip_sep() + if lx.peek() == ")": + lx.i += 1 + return ("list", items) + items.append(_lex_value(lx)) + if c == "'" : + # quoted = ' (symbol / list) + lx.i += 1 + lx.skip_sep() + v = _lex_value(lx) + if v[0] not in ("symbol", "list"): + raise LintError(f"only symbols and lists may be quoted, not {v[0]}", lx.line()) + return ("quoted", v) + if c == ":": + raise LintError("stray keyword — a keyword may only lead a field", lx.line()) + if c and c.isdigit() or (c == "-" and lx.peek(1).isdigit()): + v = _lex_number(lx) + nxt = lx.peek() + if nxt and (SYMBOL_CONT.match(nxt) or nxt.isalpha()): + raise LintError("malformed token: number followed by identifier characters", lx.line()) + return v + v = _lex_symbol(lx) + if v: + return v + raise LintError(f"cannot lex value starting at {c!r} ('=' as a field separator is not a deed)", lx.line()) + + +def _lex_field_or_clause(lx): + c = lx.peek() + if c == ":": + lx.i += 1 + kw = _lex_symbol(lx) + if not kw: + raise LintError("malformed keyword: ':' must be followed by a symbol", lx.line()) + if not lx.skip_sep(): + raise LintError(f"keyword :{kw[1]} must be followed by a separator before its value", lx.line()) + val = _lex_value(lx) + if val[0] == "symbol" and val[1] in ("true", "false", "yes", "no"): + # Grammar note on the boolean production: "Never true, false, + # yes, no." These lex as symbols, so the ban is enforced here + # as a value-level semantic rule. + raise LintError( + f"boolean meaning must use #t/#f — bare symbol {val[1]!r} is forbidden as a value", lx.line() + ) + return ("field", kw[1], val) + if c == "(": + lx.i += 1 + head = _lex_symbol(lx) + if not head: + raise LintError("clause '(' must be followed immediately by a clause symbol (no separator)", lx.line()) + items = [] + while True: + lx.skip_sep() + if lx.peek() == ")": + lx.i += 1 + return ("clause", head[1], items) + if lx.peek() == "" and lx.i >= lx.n: + raise LintError(f"unbalanced parens: clause ({head[1]}) never closes", lx.line()) + items.append(_lex_field_or_clause(lx)) + if c == "" : + raise LintError("unexpected end of input (unbalanced parens)", lx.line()) + raise LintError(f"expected field (':keyword …') or clause ('(symbol …)'), got {c!r}", lx.line()) + + +def _parse_form(lx): + """"(" doc-head 1*(token-sep (field/clause)) [token-sep] ")" — no sep after '('.""" + if lx.peek() != "(": + raise LintError("a deed form must start with '('", lx.line()) + lx.i += 1 + head = _lex_symbol(lx) + heads = ("estate-deed", "repo-deed", "estate-atlas-deed", "praxis-deed") + if not head or head[1] not in heads: + got = head[1] if head else lx.peek() + raise LintError(f"invalid doc-head {got!r}; valid heads: {', '.join(heads)}", lx.line()) + if not lx.skip_sep(): + raise LintError(f"doc-head {head[1]} must be followed by a separator before the first field", lx.line()) + items = [] + while True: + have_sep = lx.skip_sep() + if lx.peek() == ")": + lx.i += 1 + break + if lx.peek() == "": + raise LintError("unbalanced parens: form never closes", lx.line()) + items.append(_lex_field_or_clause(lx)) + schema = [it for it in items if it[0] == "field" and it[1] == "schema-version"] + if len(schema) != 1: + raise LintError( + f"form must carry exactly one :schema-version STRING field (found {len(schema)})", lx.line() + ) + if schema[0][2][0] != "string": + raise LintError(":schema-version must be a STRING value", lx.line()) + if lx.skip_sep() != 0 and lx.i >= lx.n: + return head[1], items + if lx.i < lx.n: + raise LintError( + "trailing content after the form's closing ')' — a deed is exactly one form", lx.line() + ) + return head[1], items + + +def _parse_header(lx): + """header = 1* spdx-line ; spdx-line = ";;" SP %s"SPDX-" 1*text-char line-end""" + count = 0 + while True: + if lx.t[lx.i :].startswith(";; SPDX-"): + eol = lx.t.find("\n", lx.i) + if eol == -1: + raise LintError("SPDX header line has no line-end", lx.line()) + payload = lx.t[lx.i + 8 : eol] + if not payload.strip(): + raise LintError("SPDX header line is empty after ';; SPDX-'", lx.line()) + lx.i = eol + 1 + count += 1 + continue + break + if count == 0: + raise LintError( + "deed must begin with at least one SPDX header line (';; SPDX-License-Identifier: …')", lx.line() + ) + return count + + +def validate(text, filename=None): + """Validate DEED source text. Returns (head, items) on success; raises LintError.""" + if "\t" in text: + raise LintError("HTAB (tab) is an invalid separator anywhere in a deed (K9-consistent)", text.count("\n", 0, text.find("\t")) + 1) + try: + text.encode("utf-8") + except UnicodeEncodeError as e: # pragma: no cover + raise LintError(f"invalid UTF-8: {e}", 1) + lx = _Lexer(text) + _parse_header(lx) + lx.skip_sep() + head, items = _parse_form(lx) + if filename: + check_filename_dispatch(filename, head) + return head, items + + +def check_filename_dispatch(filename, head): + """estate-file exact-first; stems may contain dots (split on the final suffix).""" + import os + + base = os.path.basename(filename) + if base == "estate_chora.deed": + want = "estate-deed" + elif base == "ATLAS.deed": + want = "estate-atlas-deed" + elif base.endswith("_praxis.deed"): + want = "praxis-deed" + elif base.endswith("_chora.deed"): + stem = base[: -len("_chora.deed")] + if stem == "estate": + return # handled by estate-file branch (exact-first); unreachable + if not re.fullmatch(r"[A-Za-z0-9-._]+", stem) or not stem: + raise LintError(f"illegal deed filename stem {stem!r}", 1) + want = "repo-deed" + else: + raise LintError( + f"filename {base!r} matches no deed dispatch pattern " + "(estate_chora.deed | ATLAS.deed | _chora.deed | _praxis.deed)", + 1, + ) + if head != want: + raise LintError( + f"doc-head/filename mismatch: {base} dispatches to {want} but parses as {head}", 1 + ) + + +_SELF_TEST = r""" +# (valid head fields) +1. valid minimal deed : OK +""" + +_FIXTURES = [ + ("valid-minimal", True, + ';; SPDX-License-Identifier: CC-BY-SA-4.0\n(repo-deed :schema-version "1.0.0" :canonical-name "x" :repo-uuid #u5"github.com/o/x" :beholding-chora #u5"estate/chora")\n'), + ("valid-nested", True, + ';; SPDX-License-Identifier: CC-BY-SA-4.0\n(repo-deed\n :schema-version "1.0.0"\n:canonical-name "x" ; comment between\n (lineage :type hub :parent "" :previous-names ()) )\n'), + ("valid-booleans-uuid", True, + ';; SPDX-License-Identifier: CC-BY-SA-4.0\n(repo-deed :schema-version "1.0.0" (status :present #t :ended #f :note "legal escapes: \\n and \\t and \\\\ and \\"q\\""))\n'), + ("invalid-equals", False, + ';; SPDX-License-Identifier: CC-BY-SA-4.0\n(repo-deed :schema-version "1.0.0" :canonical-name = "x")\n'), + ("invalid-section", False, + ';; SPDX-License-Identifier: CC-BY-SA-4.0\n(repo-deed :schema-version "1.0.0"\n[status]\nphase = "active")\n'), + ("invalid-missing-schema", False, + ';; SPDX-License-Identifier: CC-BY-SA-4.0\n(repo-deed :canonical-name "x")\n'), + ("invalid-head", False, + ';; SPDX-License-Identifier: CC-BY-SA-4.0\n(chora-deed :schema-version "1.0.0")\n'), + ("invalid-true-literal", False, + ';; SPDX-License-Identifier: CC-BY-SA-4.0\n(repo-deed :schema-version "1.0.0" (status :present true))\n'), + ("invalid-tab", False, + ';; SPDX-License-Identifier: CC-BY-SA-4.0\n(repo-deed\t:schema-version "1.0.0")\n'), + ("invalid-escape-u", False, + ';; SPDX-License-Identifier: CC-BY-SA-4.0\n(repo-deed :schema-version "1.0.0" (m :s "bad \\u0041"))\n'), + ("invalid-trailing", False, + ';; SPDX-License-Identifier: CC-BY-SA-4.0\n(repo-deed :schema-version "1.0.0") trailing\n'), + ("invalid-no-header", False, + '(repo-deed :schema-version "1.0.0")\n'), + ("invalid-string-after-head", False, + ';; SPDX-License-Identifier: CC-BY-SA-4.0\n(repo-deed :schema-version "not-string-issue" :other 007 :sym github-actions :q \'(a b))\n'), # actually VALID — keep to prove quoted lists + 007 parse + ("invalid-unbalanced", False, + ';; SPDX-License-Identifier: CC-BY-SA-4.0\n(repo-deed :schema-version "1.0.0" (status :present #t)\n'), +] +# corrected expectation: the fixture labelled invalid-string-after-head above is actually valid +_FIXTURES[12] = ("valid-quoted-list-symbols-007", True, _FIXTURES[12][2]) + + +def _self_test(): + ok = True + for name, expect_ok, src in _FIXTURES: + try: + validate(src) + got = True + err = "" + except LintError as e: + got = False + err = str(e) + passed = got == expect_ok + ok &= passed + status = "PASS " if passed else "FAIL " + detail = "" if passed else f" (expected {'valid' if expect_ok else 'error'}; got {'valid' if got else err})" + print(f"{status}{name}{detail}") + print("SELF-TEST " + ("OK" if ok else "FAILED")) + return 0 if ok else 1 + + +def _fixtures(d): + """valid/ must parse; invalid/ must fail. Returns exit code.""" + import glob + bad = 0 + for sub, expect in (("valid", True), ("invalid", False)): + for f in sorted(glob.glob(os.path.join(d, sub, "*.deed"))): + try: + validate(open(f, encoding="utf-8").read(), filename=f) + got, err = True, "" + except LintError as e: + got, err = False, str(e) + mark = "PASS " if got == expect else "FAIL " + if got != expect: + bad += 1 + print(f"{mark}{f}" + ("" if got == expect else f" (unexpected: {err or 'valid'})")) + return 1 if bad else 0 + + +def main(argv): + if "--self-test" in argv: + return _self_test() + if "--fixtures" in argv: + d = argv[argv.index("--fixtures") + 1] + rc = _fixtures(d) + print("FIXTURES " + ("OK" if rc == 0 else "FAILED")) + return rc + files = [a for a in argv[1:] if not a.startswith("-")] + if not files: + print(__doc__) + return 2 + bad = 0 + for f in files: + try: + with open(f, encoding="utf-8") as fh: + validate(fh.read(), filename=f) + print(f"OK {f}") + except LintError as e: + bad += 1 + print(f"FAIL {f}: {e}") + except OSError as e: + bad += 1 + print(f"FAIL {f}: {e}") + return 1 if bad else 0 + + +if __name__ == "__main__": + sys.exit(main(sys.argv)) diff --git a/1-formats/deed/tools/fixtures/invalid/inequals_chora.deed b/1-formats/deed/tools/fixtures/invalid/inequals_chora.deed new file mode 100644 index 000000000..3183ae9e6 --- /dev/null +++ b/1-formats/deed/tools/fixtures/invalid/inequals_chora.deed @@ -0,0 +1,2 @@ +;; SPDX-License-Identifier: CC-BY-SA-4.0 +(repo-deed :schema-version "1.0.0" :canonical-name = "x") diff --git a/1-formats/deed/tools/fixtures/invalid/inescape-u_chora.deed b/1-formats/deed/tools/fixtures/invalid/inescape-u_chora.deed new file mode 100644 index 000000000..88d0bcfa9 --- /dev/null +++ b/1-formats/deed/tools/fixtures/invalid/inescape-u_chora.deed @@ -0,0 +1,2 @@ +;; SPDX-License-Identifier: CC-BY-SA-4.0 +(repo-deed :schema-version "1.0.0" (m :s "bad \u0041")) diff --git a/1-formats/deed/tools/fixtures/invalid/inhead_chora.deed b/1-formats/deed/tools/fixtures/invalid/inhead_chora.deed new file mode 100644 index 000000000..24885fe1a --- /dev/null +++ b/1-formats/deed/tools/fixtures/invalid/inhead_chora.deed @@ -0,0 +1,2 @@ +;; SPDX-License-Identifier: CC-BY-SA-4.0 +(chora-deed :schema-version "1.0.0") diff --git a/1-formats/deed/tools/fixtures/invalid/inmissing-schema_chora.deed b/1-formats/deed/tools/fixtures/invalid/inmissing-schema_chora.deed new file mode 100644 index 000000000..2bcc3549e --- /dev/null +++ b/1-formats/deed/tools/fixtures/invalid/inmissing-schema_chora.deed @@ -0,0 +1,2 @@ +;; SPDX-License-Identifier: CC-BY-SA-4.0 +(repo-deed :canonical-name "x") diff --git a/1-formats/deed/tools/fixtures/invalid/inno-header_chora.deed b/1-formats/deed/tools/fixtures/invalid/inno-header_chora.deed new file mode 100644 index 000000000..42a8b8a84 --- /dev/null +++ b/1-formats/deed/tools/fixtures/invalid/inno-header_chora.deed @@ -0,0 +1 @@ +(repo-deed :schema-version "1.0.0") diff --git a/1-formats/deed/tools/fixtures/invalid/insection_chora.deed b/1-formats/deed/tools/fixtures/invalid/insection_chora.deed new file mode 100644 index 000000000..e63678cff --- /dev/null +++ b/1-formats/deed/tools/fixtures/invalid/insection_chora.deed @@ -0,0 +1,4 @@ +;; SPDX-License-Identifier: CC-BY-SA-4.0 +(repo-deed :schema-version "1.0.0" +[status] +phase = "active") diff --git a/1-formats/deed/tools/fixtures/invalid/intab_chora.deed b/1-formats/deed/tools/fixtures/invalid/intab_chora.deed new file mode 100644 index 000000000..96d7a7027 --- /dev/null +++ b/1-formats/deed/tools/fixtures/invalid/intab_chora.deed @@ -0,0 +1,2 @@ +;; SPDX-License-Identifier: CC-BY-SA-4.0 +(repo-deed :schema-version "1.0.0") diff --git a/1-formats/deed/tools/fixtures/invalid/intrailing_chora.deed b/1-formats/deed/tools/fixtures/invalid/intrailing_chora.deed new file mode 100644 index 000000000..424cc18a3 --- /dev/null +++ b/1-formats/deed/tools/fixtures/invalid/intrailing_chora.deed @@ -0,0 +1,2 @@ +;; SPDX-License-Identifier: CC-BY-SA-4.0 +(repo-deed :schema-version "1.0.0") trailing diff --git a/1-formats/deed/tools/fixtures/invalid/intrue-literal_chora.deed b/1-formats/deed/tools/fixtures/invalid/intrue-literal_chora.deed new file mode 100644 index 000000000..6587f4f4a --- /dev/null +++ b/1-formats/deed/tools/fixtures/invalid/intrue-literal_chora.deed @@ -0,0 +1,2 @@ +;; SPDX-License-Identifier: CC-BY-SA-4.0 +(repo-deed :schema-version "1.0.0" (status :present true)) diff --git a/1-formats/deed/tools/fixtures/invalid/inunbalanced_chora.deed b/1-formats/deed/tools/fixtures/invalid/inunbalanced_chora.deed new file mode 100644 index 000000000..8828b7256 --- /dev/null +++ b/1-formats/deed/tools/fixtures/invalid/inunbalanced_chora.deed @@ -0,0 +1,2 @@ +;; SPDX-License-Identifier: CC-BY-SA-4.0 +(repo-deed :schema-version "1.0.0" (status :present #t) diff --git a/1-formats/deed/tools/fixtures/valid/booleans-uuid_chora.deed b/1-formats/deed/tools/fixtures/valid/booleans-uuid_chora.deed new file mode 100644 index 000000000..02879d127 --- /dev/null +++ b/1-formats/deed/tools/fixtures/valid/booleans-uuid_chora.deed @@ -0,0 +1,2 @@ +;; SPDX-License-Identifier: CC-BY-SA-4.0 +(repo-deed :schema-version "1.0.0" (status :present #t :ended #f :note "legal escapes: \n and \t and \\ and \"q\"")) diff --git a/1-formats/deed/tools/fixtures/valid/minimal_chora.deed b/1-formats/deed/tools/fixtures/valid/minimal_chora.deed new file mode 100644 index 000000000..653eb2409 --- /dev/null +++ b/1-formats/deed/tools/fixtures/valid/minimal_chora.deed @@ -0,0 +1,2 @@ +;; SPDX-License-Identifier: CC-BY-SA-4.0 +(repo-deed :schema-version "1.0.0" :canonical-name "x" :repo-uuid #u5"github.com/o/x" :beholding-chora #u5"estate/chora") diff --git a/1-formats/deed/tools/fixtures/valid/nested_chora.deed b/1-formats/deed/tools/fixtures/valid/nested_chora.deed new file mode 100644 index 000000000..36cb5901d --- /dev/null +++ b/1-formats/deed/tools/fixtures/valid/nested_chora.deed @@ -0,0 +1,5 @@ +;; SPDX-License-Identifier: CC-BY-SA-4.0 +(repo-deed + :schema-version "1.0.0" +:canonical-name "x" ; comment between + (lineage :type hub :parent "" :previous-names ()) ) diff --git a/1-formats/deed/tools/fixtures/valid/quoted-list-symbols-007_chora.deed b/1-formats/deed/tools/fixtures/valid/quoted-list-symbols-007_chora.deed new file mode 100644 index 000000000..5787e8052 --- /dev/null +++ b/1-formats/deed/tools/fixtures/valid/quoted-list-symbols-007_chora.deed @@ -0,0 +1,2 @@ +;; SPDX-License-Identifier: CC-BY-SA-4.0 +(repo-deed :schema-version "not-string-issue" :other 007 :sym github-actions :q '(a b)) diff --git a/1-formats/deed/tools/fixtures/valid/rsr-template-repo_chora.deed b/1-formats/deed/tools/fixtures/valid/rsr-template-repo_chora.deed new file mode 100644 index 000000000..92a54c4f1 --- /dev/null +++ b/1-formats/deed/tools/fixtures/valid/rsr-template-repo_chora.deed @@ -0,0 +1,119 @@ +;; SPDX-License-Identifier: MPL-2.0 +(repo-deed + :schema-version "1.0.0" + :canonical-name "rsr-template-repo" + :beholding-chora #u5"estate/chora" + + :repo-uuid #u5"github.com/hyperpolymath/rsr-template-repo" + + (identity :primary-forge github + :owner "hyperpolymath" + :prefixed-name "rm-rsr-template-repo") + + (clade :primary rm + :primary-name "Repo Management & Tooling" + :secondary () + :assigned "2026-03-16" + :rationale "Repo Management & Tooling (`rm`): the core value proposition is scaffolding repositories — CI/CD, AI manifests, ABI/FFI seams, governance — that other projects are instantiated from. `gv` was previously claimed as a secondary and is dropped: the template SHIPS governance files, but shipping them is not being a governance project. That authority lives in `standards` and `metadatastician-governance`. The inherited [\"gv\"] propagated into every repo created from this template, where it was even less meant.") + + (forges :github "hyperpolymath/rsr-template-repo" + :gitlab "hyperpolymath/rsr-template-repo" + :bitbucket "hyperpolymath/rsr-template-repo") + + (lineage :type standalone + :parent "" + :born "2026-03-16" + :previous-names () + :instantiated-from "") + + (status :phase active + :since "2026-03-16" + :present #t + :aliases () + :merged-into "" + :superseded-by "" + :successors () + :ended "" + (history + (entry :phase active + :since "2026-03-16" + :note "the estate's canonical repository template; in production use") + )) + + (meta + :version "0.1.0" + :last-updated "2026-04-11" + :type library + :languages () + :license MPL-2.0 + :author "Jonathan D.A. Jewell (hyperpolymath)" + :build-tool just + :container-runtime podman + :ci-platform github-actions + :package-manager guix + :scoping-first #t + :execution-order "axis-1 > axis-2 > axis-3" + :axis-1 "must > intend > like" + :axis-2 "corrective > adaptive > perfective" + :axis-3 "systems > compliance > effects" + (scoping + :sources "README, roadmap, status docs, maintenance checklist, CI/security docs" + :marker-scan "TODO/FIXME/XXX/HACK/STUB/PARTIAL" + :idris-unsound-scan "believe_me/assert_total") + :corrective-first #t + :adaptive-second #t + :adaptive-focus "scope-change reconciliation, stale-reference removal, obsolete-work culling" + :perfective-third #t + :perfective-source "axis-1 honest state after corrective/adaptive updates" + (axis-3 + :audit-focus "systems in place, documentation explains actual state, safety/security accounted for, observed effects reviewed" + :compliance-focus "seams/compromises/exception register, bounded exceptions, anti-drift checks" + :drift-risk-example "single exception broadening into policy violation (e.g. ->TypeScript spread)" + :effects-evidence "benchmark execution/results and maintainer status dialogue/review")) + + (ecosystem + :project "rsr-template-repo" + :ecosystem "hyperpolymath" + :position-type "repository-template" + :purpose "Canonical RSR-compliant repository template: scaffolding (CI/CD, AI manifests, ABI/FFI standards, container ecosystem, governance) that new hyperpolymath projects are instantiated from." + :not ("a project in its own right" "Scaffoldia (the full-featured repo designer)" "standards (the canon source this template operationalises)") + :pipeline-position "foundation" + :chain "standards → rsr-template-repo → (every estate repo)" + :pipeline-notes "rsr-template-repo turns the RSR standard into runnable scaffolding. New repos are created from it via `just repo-init`, which substitutes the {{PLACEHOLDER}} tokens." + :coordination standards + (related :name "standards" :relationship standard-source :notes "Defines the RSR standard, contractile canon, and policies that this template operationalises.") + (related :name "stapeln" :relationship build-tooling :notes "Layer-based container build system; the template ships stapeln.toml scaffolding.") + (related :name "selur-compose" :relationship build-tooling :notes "Service composition; the template ships selur-compose.toml scaffolding.") + (related :name "k9-svc" :relationship validation-tooling :notes "Runs the self-validating k9.ncl checks (.machine_readable/self-validating/).") + (related :name "cerro-torre" :relationship signing-tooling :notes "Container/image signing provider referenced by the container scaffolding.") + (related :name "svalinn" :relationship verification-tooling :notes "Supply-chain verification referenced by the container scaffolding.") + (related :name "vordr" :relationship verification-tooling :notes "Build/artifact verification referenced by the container scaffolding.")) + + (agentic + :version "0.1.0" :last-updated "2026-04-11" + (permissions :source #t + :tests #t + :docs #t + :config #t + :create-files #t) + (integrity :fail-closed #t + :require-evidence-per-step #t + :allow-silent-skip #f + :require-rerun-after-fix #t + :release-claim-requires-hard-pass #t) + (methodology :instructions-dir ".machine_readable/bot_directives/" :default-mode hybrid)) + + (neurosym + :version "0.1.0" :last-updated "2026-04-11" + (hypatia :scan-enabled #t + :scan-depth standard + :report-format "logtalk")) + + (playbook + :version "0.1.0" :last-updated "2026-04-11" + (skeleton :version "1.0" + :last-updated "2026-04-30" + :authority-allowlist ".machine_readable/root-allow.txt" + :enforcement-workflow ".github/workflows/estate-rules.yml") + (deployment) (incident-response) (release-process) (docs-format) (maintenance-operations)) +)