diff --git a/CHANGELOG.md b/CHANGELOG.md index f2de2c4..9737e88 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,12 @@ artifact. It never modifies repository code; its optional agent-interpretation step is disabled by default and defines no credentials. Minimal per-job permissions, `persist-credentials: false`, and full-SHA-pinned actions throughout. + - The shared validator is now consumed as the deterministic `engineering-loop-schemas v0.1.2` + bundle pinned to `0459d61b7b1d4e7b46709e6d3895770553e6fab0`. Its `manifest.json` records + source provenance, file sizes, SHA-256 hashes, and the declared package-import adaptation. + The mandatory `loop-schema-vendor` quality check validates the bundle offline, and regression + tests prove that manual tampering is rejected. The earlier `75a63eef...` integration described + below was an intermediate pre-release state and is superseded by this published bundle. ### Fixed diff --git a/VALIDATION.md b/VALIDATION.md index 368ae12..d11bc00 100644 --- a/VALIDATION.md +++ b/VALIDATION.md @@ -1,5 +1,71 @@ # Validation report +## Current validation — Phase 0-1 + +Validated on 2026-07-18 for the report-only Evidence-Gated Engineering Loop foundation. + +### Identity and provenance + +- Published repository commit: + `388cd8bd8e96c3ffe0b6e464a86ee6e0c6f54f2f`. +- Harness, plugin, and marketplace version: `0.6.0`. +- Shared schemas: `v0.1.2` at + `0459d61b7b1d4e7b46709e6d3895770553e6fab0`. +- Final integration: pull request + [#4](https://github.com/brunovicco/claude-python-engineering-harness/pull/4). +- The pull-request quality run validated source head + `b5c79dd0ab4d12a9072c0c0019ff3750b3d74777` before GitHub recreated the commit through squash + merge: + . + +### Results + +- Regression tests: 56 passed. +- Python compilation: passed. +- Ruff lint: passed. +- JavaScript workflow syntax: passed. +- Whitespace validation with `git diff --check`: passed. +- `loop-schema-vendor`: passed for `v0.1.2`. +- Positive bundle-integrity test: passed. +- Manual-tampering detection test: passed. +- Harness self-evaluation: `Overall: PASS`. +- Complete quality gates passed for all six profile/governance combinations: + - `service-none`; + - `service-agentic`; + - `library-none`; + - `library-agentic`; + - `workspace-none`; + - `workspace-agentic`. + +### Scope and limitations + +This validation covers Phase 0-1 only. The repository can validate contracts, execute existing +quality gates, verify vendored-schema integrity, render temporary profiles, and produce reports. +It does not provide a loop runner, state machine, evaluator runtime, autonomous candidate creation, +candidate promotion, merge, or deployment. + +The builder's report remains non-authoritative. A quality gate is the technical authority, and +human review remains required before any promotion decision. + +The self-evaluation workflow still uses the previously pinned `setup-uv` action while the main +quality workflow uses the newer approved pin. Aligning those pins is CI maintenance and should be +performed in a separate change from this documentation record. + +### Reproduce the Phase 0-1 validation + +```bash +python -m unittest discover -s tests -v +python -m compileall -q bootstrap.py template/.claude/hooks template/scripts \ + plugin/python-engineering-harness/scripts scripts +uvx --from ruff==0.15.20 ruff check --isolated bootstrap.py tests \ + template/.claude/hooks template/scripts plugin/python-engineering-harness/scripts scripts +node --check template/.claude/workflows/review-branch.js +python scripts/loop_self_evaluation.py --output-dir build/loop-self-evaluation +git diff --check +``` + +## Previous validation — 2026-07-16 + Validated on 2026-07-16 with Python 3.13 at source commit `97cbabcf4d0cf86ad0a7dcf8c8d1fd28dc5a58fe`. diff --git a/docs/LOOPS.md b/docs/LOOPS.md index 0a8e0b4..18b0249 100644 --- a/docs/LOOPS.md +++ b/docs/LOOPS.md @@ -51,10 +51,13 @@ Every loop run belongs to one of three levels of scrutiny, mirrored from - A builder never certifies its own result. Only a mechanically-derived `verdict` can. - A hard gate is default-FAIL and must reduce to a command with an exit - code. The set of hard gates a contract may reference - (`acceptance.hard_gates`) is exactly the named checks this harness's own + code. The contract-addressable hard gates + (`acceptance.hard_gates`) are exactly the named checks this harness's own `quality_gate.py` implements: `lock`, `lint`, `format`, `typing`, `tests`, `security`, `dependencies`, `architecture`, `mcp`, `governance`. + Separate mandatory infrastructure checks, including `loop-schema-vendor` and + `loop-contracts`, always protect provenance, integrity, and contract validation; they are not + arbitrary commands selectable by a builder. - Evidence is bound to exact commits (`baseline_sha`, `candidate_sha`) and a hashed environment (`uv_lock_sha256`), so a verdict can always be traced back to exactly what ran against exactly what code. @@ -81,13 +84,18 @@ Every completed run resolves to exactly one final state ## Vendoring -`template/scripts/_vendor_loop_schemas/` is a verbatim vendored copy of -`engineering-loop-schemas`' `src/loop_schemas/` at a pinned commit, recorded -in a header comment in each vendored file. Re-vendor from the source -repository rather than hand-editing; the one intentional deviation (the -package directory is named `_vendor_loop_schemas`, not `loop_schemas`, so -it does not collide with the `scripts/loop_*` denylist above) is documented -in that same header. +`template/scripts/_vendor_loop_schemas/` is a deterministic bundle rendered from +`engineering-loop-schemas v0.1.2` at the full commit +`0459d61b7b1d4e7b46709e6d3895770553e6fab0`. Its `manifest.json` records the source repository, +version, commit, file sizes, SHA-256 hashes, and the declared import adaptation. + +The bundle is not a byte-for-byte copy. During rendering, the package import in +`validate_contract.py` changes from `loop_schemas` to `_vendor_loop_schemas`; this keeps the +vendored package isolated and avoids collision with the protected `scripts/loop_*` namespace. The +adaptation is explicit in the manifest and covered by deterministic-rendering, integrity, and +tampering tests. `loop-schema-vendor` verifies the bundle offline on every generated-project +quality run. Fix the canonical schemas repository and render a new version instead of editing the +bundle manually. `validate_contract.py` is stdlib-only. Reading a YAML contract requires PyYAML to be importable in the environment `scripts/quality_gate.py` runs diff --git a/docs/LOOPS.pt-BR.md b/docs/LOOPS.pt-BR.md index 9d21ee0..c010acc 100644 --- a/docs/LOOPS.pt-BR.md +++ b/docs/LOOPS.pt-BR.md @@ -58,11 +58,14 @@ espelhando o README do `engineering-loop-schemas`: - Um builder nunca certifica o próprio resultado. Apenas um `verdict` derivado mecanicamente pode fazê-lo. - Um hard gate é default-FAIL e precisa se reduzir a um comando com código - de saída. O conjunto de hard gates que um contrato pode referenciar - (`acceptance.hard_gates`) é exatamente o conjunto de verificações + de saída. Os hard gates referenciáveis pelo contrato + (`acceptance.hard_gates`) são exatamente as verificações nomeadas que o `quality_gate.py` deste harness já implementa: `lock`, `lint`, `format`, `typing`, `tests`, `security`, `dependencies`, `architecture`, `mcp`, `governance`. + Verificações obrigatórias de infraestrutura, incluindo `loop-schema-vendor` e + `loop-contracts`, protegem proveniência, integridade e validação contratual em todas as + execuções; elas não são comandos arbitrários selecionáveis pelo builder. - A evidência é vinculada a commits exatos (`baseline_sha`, `candidate_sha`) e a um ambiente com hash (`uv_lock_sha256`), de modo que um veredito sempre possa ser rastreado até exatamente o que rodou contra @@ -90,13 +93,18 @@ Toda execução concluída se resolve em exatamente um estado final ## Vendoring -`template/scripts/_vendor_loop_schemas/` é uma cópia vendorizada literal -de `src/loop_schemas/` do `engineering-loop-schemas`, em um commit fixado, -registrado em um comentário de cabeçalho em cada arquivo vendorizado. -Revendorize a partir do repositório de origem em vez de editar manualmente; -o único desvio intencional (o diretório do pacote se chama -`_vendor_loop_schemas`, não `loop_schemas`, para não colidir com a -denylist `scripts/loop_*` acima) está documentado nesse mesmo cabeçalho. +`template/scripts/_vendor_loop_schemas/` é um bundle determinístico renderizado a partir do +`engineering-loop-schemas v0.1.2`, fixado no commit completo +`0459d61b7b1d4e7b46709e6d3895770553e6fab0`. Seu `manifest.json` registra repositório de origem, +versão, commit, tamanhos, hashes SHA-256 e a adaptação de import declarada. + +O bundle não é uma cópia byte a byte. Durante a renderização, o import de pacote em +`validate_contract.py` muda de `loop_schemas` para `_vendor_loop_schemas`; isso isola o pacote +vendorizado e evita colisão com o namespace protegido `scripts/loop_*`. A adaptação está explícita +no manifesto e coberta por testes de renderização determinística, integridade e adulteração. O +gate `loop-schema-vendor` verifica o bundle offline em cada quality gate de projeto gerado. +Corrija o repositório canônico de schemas e renderize uma nova versão, em vez de editar o bundle +manualmente. `validate_contract.py` é somente-stdlib. Ler um contrato YAML requer que o PyYAML seja importável no ambiente em que `scripts/quality_gate.py`