diff --git a/openspec/changes/migrate-catalog-skills-phase1/apply-report.md b/openspec/changes/migrate-catalog-skills-phase1/apply-report.md index 902ba560..aee0f620 100644 --- a/openspec/changes/migrate-catalog-skills-phase1/apply-report.md +++ b/openspec/changes/migrate-catalog-skills-phase1/apply-report.md @@ -5,7 +5,7 @@ - Strategy: `single-pr` with explicit user-approved `size-exception`. - Branch: `feat/migrate-catalog-skills-phase1`; base: `main`. - Scope: only the three MIT-backed Bobmatnyc entries `drizzle-orm`, `pydantic`, and `sqlalchemy`. -- Sibling source: `agents-skills` commit `c2e79fbb72d146305f82a8e979270795557d24fd` (PR #18) +- Sibling source: `agents-skills` merge commit `be4570aa6f23931f51692661d163ddd663b57488` (PR #18) contains the three migrated skill directories and `PROVENANCE.md`; CI checks out this committed revision into `agents-skills` for deterministic resolution. @@ -13,8 +13,8 @@ - Added the three canonical sibling skill directories under `agents-skills/skills/`, including all source-declared reference files and explicit full-source companions for the long Pydantic and - SQLAlchemy documents. These directories are committed in `agents-skills` commit `70298da` (PR - #18); the provenance record pins the upstream immutable source and target file hashes. + SQLAlchemy documents. These directories are committed in the `agents-skills` PR #18 merge; the + provenance record pins the upstream immutable source and target file hashes. - Added `agents-skills/PROVENANCE.md` with immutable repository commit, source paths, Git blob/tree identities, attribution, MIT license evidence, materialized file inventory, and companion status. - Remapped only the three catalog definitions and their Drizzle/Pydantic/SQLAlchemy technology @@ -28,6 +28,27 @@ - Preserved the ignored full-catalog E2E early return because unrelated external entries remain broken. +## QA Remediation Handoff (2026-08-14) + +- Added `tests/acceptance/phase1_catalog.sh`, a narrow black-box harness that launches an external + `agentsync` executable, copies only `drizzle-orm`, `pydantic`, and `sqlalchemy` into temporary + source fixtures, and asserts local installation, companions, registry keys, override precedence, + fail-closed missing-source behavior, and direct/suggestion project-root propagation. +- Documented the explicit target as + `make acceptance-phase1 AGENTSYNC_BIN=target/release/agentsync AGENTSYNC_SOURCE_REPO=../agents-skills`. + The harness fails clearly when the release-like binary is absent and never calls Rust modules + directly. The full-catalog early return remains untouched. +- Refreshed the eight stale materialized hashes in the merged `agents-skills` Phase 1 provenance + record and added `scripts/validate_provenance.py`, wired into `scripts/validate-skills.sh`, so + future materialized-byte drift is caught by a focused check. Run + `AGENTSYNC_SOURCE_REPO=../agents-skills tests/acceptance/test_phase1_provenance.sh` to invoke it. +- TDD evidence: the missing-binary harness contract failed before the harness existed and passed + after implementation; provenance validation first failed on the eight stale hashes and passed + after the refresh. +- No production source, catalog, registry manifest, full-catalog test, or unrelated untracked + sibling candidates were changed. QA remains `NOT TESTED`; `sdd-qa` must rerun against the + documented external target before any acceptance or archive decision. + ## Verification - Target sibling validator: pinned `skills-ref validate` passed for all three migrated directories diff --git a/openspec/changes/migrate-catalog-skills-phase1/state.yaml b/openspec/changes/migrate-catalog-skills-phase1/state.yaml index 0c50517e..07d94349 100644 --- a/openspec/changes/migrate-catalog-skills-phase1/state.yaml +++ b/openspec/changes/migrate-catalog-skills-phase1/state.yaml @@ -10,4 +10,4 @@ completed: - verify - qa next: archive -updated: 2026-08-13 +updated: 2026-08-14 diff --git a/openspec/changes/migrate-catalog-skills-phase1/tasks.md b/openspec/changes/migrate-catalog-skills-phase1/tasks.md index d7ca1631..8b3de56f 100644 --- a/openspec/changes/migrate-catalog-skills-phase1/tasks.md +++ b/openspec/changes/migrate-catalog-skills-phase1/tasks.md @@ -47,3 +47,10 @@ Size exception: Approved by user for the coherent three-skill Phase 1 unit; bran - [x] 3.2 Preserve the known full-catalog early return, `#[ignore]`, and `RUN_E2E` gate in `tests/test_catalog_integration.rs`; distinguish subset results from full-catalog status. - [x] 3.3 Run both-repo validation against the committed sibling revision: target validator/companion audit plus AgentSync focused/unit tests and formatting checks. - [x] 3.4 Verify `src/skills/registry.v1.toml`, `src/skills/registry.lock.toml`, unrelated sibling skills, and installed-state semantics remain unchanged. + +## QA Remediation + +- [x] 4.1 Add a reproducible external-CLI acceptance harness with isolated sibling, override, and missing-source fixtures for the three migrated skills. +- [x] 4.2 Exercise direct and suggestion installs through an explicit project root, assert companions and local registry keys, and preserve the full-catalog early return by keeping the harness Phase 1-only. +- [x] 4.3 Refresh the eight stale materialized SHA-256 entries in `../agents-skills/PROVENANCE.md` and add focused provenance validation to the sibling validation wrapper. +- [x] 4.4 Document the acceptance command and hand off results without changing production source or promoting QA to PASS. diff --git a/openspec/changes/migrate-catalog-skills-phase1/verify-report.md b/openspec/changes/migrate-catalog-skills-phase1/verify-report.md index 00b71482..a502f509 100644 --- a/openspec/changes/migrate-catalog-skills-phase1/verify-report.md +++ b/openspec/changes/migrate-catalog-skills-phase1/verify-report.md @@ -4,126 +4,168 @@ **PASS WITH WARNINGS** -The implemented scope is technically compliant for the three Bobmatnyc database/validation entries that were actually migrated. The 2026-08-13 refresh confirmed that the three generated entrypoint hashes in `agents-skills/PROVENANCE.md` now match the current files. Focused installation, resolver, catalog-boundary, companion, registry-key, formatting, and prior full Rust/clippy/coverage evidence remain passing. Approval is not a full 11/12-skill migration: sibling content is still untracked, pre-existing Clerk validator failures remain, the full-catalog E2E remains intentionally ignored/early-returned, and QA remains limited by the absence of a launchable acceptance target. +The QA remediation is technically compliant and executable. The new acceptance target launches an +external release-like `agentsync` binary, observes JSON/exit-code/filesystem/registry behavior, and +passes the Phase 1 sibling, override, missing-source, companion, and suggestion-root scenarios. +The eight stale provenance hashes are corrected in the current sibling worktree and all ten recorded +hashes validate. This report does not claim user/operator acceptance; `sdd-qa` may proceed. -## Change and scope +## Change, mode, and completeness | Item | Result | |---|---| | Change | `migrate-catalog-skills-phase1` | -| Repositories inspected | `agentsync` branch `feat/migrate-catalog-skills-phase1`; `agents-skills` branch `feat/phase1-auth-db-skills` | -| Declared applied scope | `drizzle-orm`, `pydantic`, `sqlalchemy` only | -| Applied agentsync diff | 321 changed lines across 6 tracked files; no commit/push performed | -| Sibling worktree | 3 migrated directories + `PROVENANCE.md` are untracked; 9 Clerk, `angular-architecture`, and `typescript-strict-patterns` are also untracked and excluded | -| Verification refresh | 2026-08-13; provenance hashes rechecked and focused checks rerun after the correction | -| Tasks | 1.1–3.4 marked complete in `tasks.md`; implementation evidence supports the scoped Bobmatnyc unit, with warnings below | -| Persistence mode | OpenSpec | - -## Build, tests, and static checks - -| Check | Result | Evidence | +| Persistence mode | `openspec` | +| Scope verified | QA-F-001 provenance remediation and QA-F-002 external acceptance target | +| Original tasks | 15/15 complete (`1.1`–`3.4`) | +| QA remediation tasks | 4/4 complete (`4.1`–`4.4`) | +| Incomplete tasks | 0 | +| Application/source behavior changes | None in this remediation; no `src/` or catalog diff | +| New verification surface | `Makefile` target and three `tests/acceptance/` scripts | +| Sibling skill-content changes | None; only provenance/wrapper/validator remediation files changed | +| Next phase | `qa` | + +The unrelated untracked Clerk, Angular, and TypeScript candidate directories in the local +`agents-skills` checkout were inspected only for exclusion and were not modified or counted as +Phase 1 changes. The three migrated skill directories remain tracked and their committed bytes are +unchanged relative to the supplied `be4570aa...` source revision. + +## Exact diff inspected + +### `agentsync` + +- Tracked diff: `Makefile`, `apply-report.md`, `qa-report.md`, `state.yaml`, and `tasks.md`. +- Untracked additions: `tests/acceptance/phase1_catalog.sh`, + `tests/acceptance/test_phase1_catalog_harness.sh`, and + `tests/acceptance/test_phase1_provenance.sh`. +- `git diff --name-only -- src tests/test_catalog_integration.rs tests/unit/provider.rs + tests/unit/suggest_catalog.rs tests/unit/suggest_install.rs` produced no existing application or + Phase 1 test-source delta; the only new test files are the acceptance scripts above. + +### `agents-skills` + +- Tracked diff: `PROVENANCE.md` (the eight refreshed hashes plus validator documentation) and + `scripts/validate-skills.sh` (the validator invocation). +- Untracked addition: `scripts/validate_provenance.py`. +- `git diff --name-status be4570aa... -- skills/drizzle-orm skills/pydantic skills/sqlalchemy` was + empty; the unrelated candidate paths remain untracked and untouched. +- `git diff --check` and `git -C ../agents-skills diff --check` passed. + +## Build, test, coverage, and validation evidence + +All commands below were run during this verification on 2026-08-14. + +| Command | Result | +|---|---| +| `cargo check --all-targets --all-features` | PASS | +| `cargo build --release` | PASS; release-like target rebuilt | +| `make acceptance-phase1 AGENTSYNC_BIN=target/release/agentsync AGENTSYNC_SOURCE_REPO=../agents-skills` | PASS; external CLI harness completed all four scenarios | +| `tests/acceptance/test_phase1_catalog_harness.sh` | PASS; missing-binary negative contract rejected the absent target | +| `AGENTSYNC_SOURCE_REPO=../agents-skills tests/acceptance/test_phase1_provenance.sh` | PASS; `[OK] validated materialized hashes` | +| `python3 scripts/validate_provenance.py --root .` in `agents-skills` | PASS; all 10 provenance hash entries validated | +| Pinned `skills-ref validate` for `skills/drizzle-orm`, `skills/pydantic`, and `skills/sqlalchemy` | PASS; each reported `Valid skill` | +| `cargo test --all-features --test test_catalog_integration phase1_bobmatnyc_catalog_entries_install_offline_and_register_local_ids -- --nocapture` | PASS; 1 passed | +| `cargo test --all-features --test all_tests 'unit::provider::' -- --nocapture` | PASS; 16 passed | +| `cargo test --all-features --test all_tests 'unit::suggest_catalog::' -- --nocapture` | PASS; 23 passed | +| `cargo test --all-features --test all_tests 'unit::suggest_install::' -- --nocapture` | PASS; 10 passed | +| `cargo test --all-features commands::skill::tests::direct_catalog_install_resolution_uses_the_sibling_agents_skills_checkout` | PASS; 1 passed | +| `cargo test --all-features commands::skill::tests::suggestion_catalog_install_resolution_uses_the_project_root` | PASS; 1 passed | +| `RUN_E2E=1 cargo test --all-features --test test_catalog_integration every_catalog_skill_installs_successfully -- --ignored --nocapture` | PASS only via the intentional early-return message; no full-catalog health claim | +| `cargo llvm-cov --all-features --test all_tests --test test_catalog_integration --summary-only` | PASS; 123 passed, 2 ignored in `all_tests`; 2 passed, 1 ignored in catalog integration; 34.17% line coverage, no threshold asserted | +| `cargo fmt --all -- --check`, `bash -n tests/acceptance/*.sh`, and `git diff --check` | PASS | + +The repository-wide `agents-skills/scripts/validate-skills.sh` was not used as the focused result +because it enumerates the unrelated untracked candidates and its later repository-wide checker has +known Clerk activation-cue failures. Its exact diff was inspected and confirms that it invokes +`validate_provenance.py`; the focused validator was run directly and passed. + +## External harness assessment + +| Property | Evidence | Judgment | |---|---|---| -| Focused Phase 1 integration | PASS | `cargo test --all-features --test test_catalog_integration phase1_bobmatnyc_catalog_entries_install_offline_and_register_local_ids` — 1 passed | -| Provider/catalog focused tests | PASS | `cargo test --all-features --test all_tests 'unit::provider::'` — 16 passed; Phase 1 provider tests include local override, sibling lookup, fail-closed behavior, and unrelated fallback | -| Catalog boundary test | PASS | `cargo test --all-features --test all_tests phase1_bobmatnyc_entries_use_curated_sources_and_preserve_boundaries` — 1 passed | -| Suggest install regression suite | PASS | `cargo test --all-features --test all_tests unit::suggest_install::` — 10 passed | -| Full Rust suite | PASS (prior evidence retained) | `cargo test --all-features` — 578 lib + 188 main + 123 integration + all standalone integration suites passed; not rerun for the provenance-only refresh | -| Formatting | PASS | `cargo fmt --all -- --check` and `git diff --check` rerun during refresh | -| Clippy | PASS (prior evidence retained) | `cargo clippy --all-targets --all-features -- -D warnings` passed previously; not rerun for the provenance-only refresh | -| Coverage command | PASS (evidence only, prior run retained) | `cargo llvm-cov --all-features --test all_tests --test test_catalog_integration --summary-only` previously reported 33.75% aggregate line coverage; not rerun and no threshold is asserted | -| Target `skills-ref` validation | PASS | Pinned validator passed separately for `skills/drizzle-orm`, `skills/pydantic`, and `skills/sqlalchemy` | -| Target repository validator | WARNING | `python3 scripts/validate_skills.py` fails on the nine pre-existing Clerk directories because each lacks the exact `Use when` activation cue; no migrated Bobmatnyc directory is among the failures | -| Provenance entrypoint hashes | PASS | SHA-256 recomputation matches the three `PROVENANCE.md` values: `drizzle-orm` `31aab8f3fff9dc3b4dd0ac593f33d6b5e6583885db5a373dbcfd805b3732714f`, `pydantic` `6769a7817671c8673e94221357d2e2058867594a006fdc05b3d551850cf4ff99`, `sqlalchemy` `35e1956c80ec9b8644d6b67de13909696ee83191534a51bf0d7dbfdfca65df4b` | -| Registry manifest/lock diff | PASS | `src/skills/registry.v1.toml` and `src/skills/registry.lock.toml` are unchanged, as required by design | +| External | Executes `target/release/agentsync` as a separate process; no Rust module, test helper, or in-process API is called | PASS | +| Observable | Asserts JSON `status`, process failure, stderr text, installed `SKILL.md`/companions, registry keys, source markers, and absence of unintended paths | PASS | +| Isolated | Uses `mktemp`, separate project roots, temporary `HOME`, temporary command CWD, and copies only the three Phase 1 directories; markers are added only to copies | PASS | +| Reproducible | Requires explicit `AGENTSYNC_BIN`/`AGENTSYNC_SOURCE_REPO`, preflights all three source directories, disables update checks, and cleans its temporary root with a trap | PASS | +| Phase 1 scope | Covers direct sibling installs for all three entries, override precedence, missing-source fail-closed behavior, companions, local registry keys, and suggestion project-root/provider/local-ID propagation | PASS | +| Full catalog boundary | Does not enumerate or install the full catalog; the separate ignored test still emits the intentional early-return message | PASS WITH WARNING; not a full-catalog result | + +The harness does not attempt interruption/retry behavior or network packet interception, and it does +not pin a source commit itself. Those are limitations for later acceptance/release hardening, not a +failure of the intended Phase 1 remediation target. The supplied `AGENTSYNC_SOURCE_REPO` is checked +by the separate provenance validator before QA handoff. ## Spec compliance matrix -| Requirement / scenario | Implementation evidence | Runtime evidence | Status | -|---|---|---|---| -| Provider ID reaches resolution | `src/skills/suggest.rs:503-515` resolves with `recommendation.provider_skill_id` and installs with `recommendation.skill_id`; existing suggest-install suite passed | `tests/unit/suggest_install.rs` existing provider-ID keyed fixtures; full suite passed | PASS | -| Local ID controls install state | `src/skills/suggest.rs:511-524` passes local ID to installer and inserts local ID into installed state | Focused offline test checks each local key in `registry.json`; passed | PASS | -| Approved local source resolves offline | `src/skills/provider.rs:186-237` checks test override, `AGENTSYNC_LOCAL_SKILLS_REPO`, then sibling checkout before catalog/provider fallback | Phase 1 integration installed all three entries with directory sources; passed | PASS | -| Missing curated source fails closed | `src/skills/provider.rs:244-252` blocks the three migrated IDs before provider fallback | `unit::provider::phase1_catalog_source_fails_closed_when_curated_content_is_missing`; passed | PASS | -| Local override precedence | `src/skills/provider.rs:191-220` checks local candidates before external catalog/provider paths | `phase1_catalog_source_uses_agentsync_local_skills_repo_before_provider` and sibling test; passed | PASS | -| Catalog remap preserves metadata | `src/skills/catalog.v1.toml` remaps only the three Bobmatnyc definitions and affected technology mappings, removes stale install sources, and retains local IDs/titles/summaries | `phase1_bobmatnyc_entries_use_curated_sources_and_preserve_boundaries`; passed | PASS | -| Base Clerk router and Wispbit SQLAlchemy remain external | Catalog diff retains base Clerk and Wispbit SQLAlchemy references; boundary test asserts Wispbit remains in SQLAlchemy mapping | Same catalog boundary test; passed | PASS | -| Companion files survive install | Focused integration expects four Drizzle references and SQLAlchemy quality reference; Pydantic source companion is present in sibling tree and entrypoint links to it | Focused integration passed; source link audit found all declared links present | PASS WITH WARNING | -| Provenance and license gate | `agents-skills/PROVENANCE.md` records Bobmatnyc repo, immutable commit, license evidence, attribution, source identities, companion status, and hashes matching all current materialized files; Clerk is explicitly blocked | Hash recomputation and both-repository inspection passed; sibling content remains uncommitted | PASS WITH WARNING | -| Focused subset cannot pass by omission | Test has explicit three-entry expected array and fails if a definition is absent; every entry installs and checks `SKILL.md`, companions, and registry key | Focused integration passed | PASS | -| Full-catalog E2E preservation | `tests/test_catalog_integration.rs:136-146` retains `#[ignore]`, `RUN_E2E` guard code, and an explicit early return explaining unrelated external failures | Full suite reports `every_catalog_skill_installs_successfully ... ignored`; no full-catalog green claim made | PASS WITH WARNING | -| Unrelated/Clerk exclusion | Catalog diff contains no Clerk remap; sibling provenance explicitly excludes Clerk, Angular, and TypeScript candidates; worktree inspection shows them still untracked | Target validator failures are limited to pre-existing Clerk files; no unrelated files in agentsync diff | PASS | - -## Correctness table - -| Area | Finding | Status | +| Requirement / scenario | Implementation and runtime evidence | Status | +|---|---|---| +| Qualified ID reaches resolution | `install_selected_with_reporter()` calls `provider.resolve(&recommendation.provider_skill_id)` and passes `recommendation.skill_id` only to installation; `unit::suggest_install::` passed 10 tests | PASS | +| Provider routing remains qualified | `SkillsShProvider` deterministic test for `dallay/agents-skills/docker-expert` passed within the 16 provider tests | PASS | +| Local ID controls installation state | Focused integration and external harness assert local install folders and `registry.json` keys for all three migrated IDs | PASS | +| Install-all preserves both IDs | Existing `install_all_skips_already_installed_recommendations` runtime coverage uses qualified provider fixtures and local IDs; external suggestion `--install --all` also asserts qualified `pydantic` identity and local result | PASS WITH WARNING; the new external flow intentionally narrows pending installs to the Phase 1 suggestion target rather than fabricating three detections | +| Approved source resolves offline | External CLI installs all three from a temporary sibling fixture; focused Rust integration passes and asserts directory resolution rather than online resolution | PASS | +| Missing source blocks | External CLI with an empty override exits non-zero, reports `refusing external fallback`, and creates no `pydantic` directory; provider negative test also passed | PASS | +| Remap preserves metadata | Catalog boundary test passed 23 tests, including local IDs, titles, summaries, removed old definitions, technology mappings, and retained Wispbit boundary | PASS | +| Complete attributable source is eligible | Current `PROVENANCE.md` records immutable source/license/attribution/companion metadata; all 10 materialized hashes validate and all three pinned `skills-ref` checks pass | PASS | +| Incomplete or unsupported source blocks | Only the approved three entries are locally mapped; Clerk and other unrelated candidates remain outside the catalog remap, as confirmed by exact diff and catalog boundary tests | PASS | +| Approved subset installs and registers | External CLI and focused integration both install every approved entry, verify `SKILL.md`/companions, and verify canonical local registry keys | PASS | +| Incomplete subset fails | Harness preflight rejects a missing source and the empty-source install path fails with an identified local skill and no output directory | PASS | +| Focused success remains scoped | The ignored full-catalog test retains `#[ignore]`, `RUN_E2E`, and the explicit early return; the ignored test passed only after printing the skip message | PASS WITH WARNING; full catalog intentionally untested | + +All applicable scenarios have runtime coverage. The install-all row is marked with a warning only to +make its deliberate Phase 1 suggestion-fixture narrowing explicit; it does not indicate a failing +implementation or a missing required remediation behavior. + +## Correctness and design coherence + +### Correctness + +| Finding / area | Evidence | Status | |---|---|---| -| Catalog mapping | Exactly three Bobmatnyc provider IDs changed to `dallay/agents-skills/{local_skill_id}`; install sources removed; Drizzle/Pydantic/SQLAlchemy technology mappings updated; Wispbit retained | PASS | -| Resolver | Local sources precede external fallback; Phase 1 IDs fail closed when absent; unrelated curated ID keeps provider fallback | PASS | -| Caller propagation | Direct install, update, and suggestion provider now receive project root; regression tests cover direct and suggestion paths | PASS | -| Install semantics | Provider ID is used only for resolution; local ID remains installer argument and registry key | PASS | -| TDD evidence | `tasks.md` records RED tasks as complete; added tests are present and passed at runtime. The report does not independently prove historical RED-before-production ordering, but runtime coverage is real | PASS WITH WARNING | -| Provenance | Bobmatnyc immutable source commit and MIT root-license evidence are recorded; all three entrypoint hashes now match current bytes; copied content remains uncommitted in sibling worktree | PASS WITH WARNING | -| Exclusions | No Clerk, Angular, TypeScript catalog or production changes; base Clerk and Wispbit boundaries retained | PASS | -| Full catalog | Not green and not claimed; intentionally ignored/early-returned | PASS WITH WARNING | - -## Design coherence - -| Design decision | Code/evidence | Status | +| QA-F-001 stale provenance hashes | Eight changed materialized hashes were refreshed; the new validator parsed and recomputed all 10 entries successfully | Resolved for current remediation worktree | +| QA-F-002 missing acceptance target | External binary harness, negative contract test, and reproducible `make acceptance-phase1` target now exist and pass | Resolved for technical handoff | +| Source isolation | Only `drizzle-orm`, `pydantic`, and `sqlalchemy` are copied by the harness; unrelated untracked candidates were not changed or counted | PASS | +| Production behavior | No `src/`, catalog, registry manifest, full-catalog test, or sibling skill-content change was introduced by remediation | PASS | +| Task completion | Tasks `4.1`–`4.4` are checked and evidence above covers each one | PASS | + +### Design coherence + +| Design decision | Evidence | Status | |---|---|---| -| Local sibling is source of truth | Resolver uses sibling `../agents-skills/skills/` and environment override | PASS | -| Qualified local IDs and no mutable Phase 1 HEAD fallback | Catalog uses `dallay/agents-skills/...`; migrated entries have no `install_source`; missing entries error before provider fallback | PASS | -| Unrelated external behavior preserved | Only narrow local-ID guard applies; unrelated `docker-expert` fallback test passed | PASS | -| No registry redesign | Curated registry manifests unchanged; focused test validates runtime `registry.json` only | PASS | -| Verbatim/companion policy | DB content carries source metadata; companion files are present and recursively installed. Entrypoints are normalized/generated rather than byte-identical upstream files, as documented | PASS WITH WARNING | -| Full-catalog skip preserved | Existing `#[ignore]` plus early return retained | PASS | +| Local sibling/override is the Phase 1 source of truth | Harness proves sibling resolution and `AGENTSYNC_LOCAL_SKILLS_REPO` precedence through the CLI | PASS | +| Qualified local IDs and fail-closed behavior | Catalog remap removes mutable Phase 1 install sources; provider tests and missing-source harness prevent fallback | PASS | +| Preserve install/registry semantics | All three external installs and the suggestion install use local IDs for folders and registry keys while exposing qualified provider IDs | PASS | +| Companions are recursive and visible | Harness and focused integration assert the Drizzle, Pydantic, and SQLAlchemy companion files | PASS | +| No registry redesign/full-catalog claim | Registry manifests are unchanged; full-catalog early return remains and is explicitly reported as not a health result | PASS WITH WARNING | +| Remediation stays outside production behavior | Acceptance and provenance checks are shell/Python tooling only; release build and focused tests pass | PASS | ## Issues ### CRITICAL -None for the applied three-entry technical scope. +None. No test command failed, no task is incomplete, and no applicable Phase 1 behavior lacks runtime +evidence after this remediation. ### WARNING | Finding | Judge A | Judge B | Severity | Status | |---|---|---|---|---| -| `agents-skills` migrated content and `PROVENANCE.md` are untracked on branch `feat/phase1-auth-db-skills`; no committed sibling revision exists to satisfy the strict wording of the spec/design gate | ✅ | ✅ | WARNING | Confirmed — technically tested from the current checkout, not immutable/committed | -| `agents-skills/scripts/validate_skills.py:114-136` validates the whole repository and fails on pre-existing Clerk files for missing `Use when`; the three migrated directories pass `skills-ref` and are not reported by the repository validator | ✅ | ✅ | WARNING | Confirmed — unrelated/pre-existing blocker, not a Bobmatnyc entry failure | -| Full-catalog E2E remains ignored and early-returned, so this change does not prove full catalog health | ✅ | ✅ | WARNING | Confirmed and explicitly out of scope | -| `agents-skills` has 9 Clerk plus 2 unrelated candidate directories still untracked; this worktree is not an isolated clean Phase 1 source snapshot | ✅ | ✅ | WARNING | Confirmed — excluded by provenance and agentsync diff | -| QA remains `NOT TESTED` for product acceptance because no launchable target or black-box harness is available | ✅ | ✅ | WARNING | Confirmed in `qa-report.md`; technical checks do not establish user/operator acceptance | -| No dedicated new test was added for `install-all` with three distinct provider/local IDs in this change; existing `install_all` coverage passed and `install_selected_with` implementation preserves the two-ID contract | ✅ | ❌ | SUGGESTION | Suspect/coverage gap, not a failing requirement for this scoped migration | +| Refreshed `PROVENANCE.md`, wrapper invocation, and validator are uncommitted in the local sibling worktree; merged commit `be4570aa...` still represents the pre-refresh provenance bytes until this remediation is persisted | ✅ | ✅ | WARNING | Confirmed; current worktree validator passes, archive requires a final committed-byte recheck | +| Full-catalog E2E remains ignored and intentionally early-returns while unrelated external entries remain unresolved | ✅ | ✅ | WARNING | Confirmed and explicitly out of scope; no full-catalog health claim | +| Harness has no interruption/retry or network-interception assertion and accepts an explicitly supplied source path rather than enforcing a Git commit itself | ✅ | ✅ | WARNING | Confirmed limitation; not required for the Phase 1 remediation target | +| `scripts/validate-skills.sh` remains repository-wide and will encounter unrelated candidate/Clerk validation issues in the dirty local sibling checkout | ✅ | ✅ | WARNING | Confirmed environment limitation; migrated directories pass individually and unrelated candidates remain untouched | ### SUGGESTION -- Commit only `PROVENANCE.md` and the three Bobmatnyc directories (plus their required companions) in the sibling repository before treating these sources as approved immutable inputs. -- If the target repository uses `scripts/validate_skills.py` as its required gate, run it against an isolated checkout or resolve the pre-existing Clerk activation-cue failures in a separate change; do not broaden this migration to Clerk without authoritative license evidence and companion decisions. -- Keep the full-catalog E2E skip until remaining external entries are remapped or migrated; do not change the verdict to full-catalog green based on the focused test. -- Re-run `sdd-qa` after the sibling content is committed/isolated and an acceptance target is available; the current QA report remains `NOT TESTED`. - -## Exact checks run in this refresh - -- SHA-256 recomputation script in `/Users/acosta/Dev/dallay/agents-skills` for `skills/drizzle-orm/SKILL.md`, `skills/pydantic/SKILL.md`, and `skills/sqlalchemy/SKILL.md` against `PROVENANCE.md` — all three matched. -- `PYTHONPATH=.tools/skills-ref/lib/python3.14/site-packages python3.14 .tools/skills-ref/bin/skills-ref validate skills/drizzle-orm` — `Valid skill`. -- Same pinned `skills-ref validate` command for `skills/pydantic` — `Valid skill`. -- Same pinned `skills-ref validate` command for `skills/sqlalchemy` — `Valid skill`. -- `python3 scripts/validate_skills.py` in `agents-skills` — exit 1 as expected; exactly nine pre-existing Clerk directories failed the `Use when` cue, and no migrated Bobmatnyc directory failed. -- `CARGO_NET_OFFLINE=true cargo test --all-features --test test_catalog_integration phase1_bobmatnyc_catalog_entries_install_offline_and_register_local_ids` — 1 passed. -- `CARGO_NET_OFFLINE=true cargo test --all-features --test all_tests 'unit::provider::'` — 16 passed. -- `CARGO_NET_OFFLINE=true cargo test --all-features --test all_tests phase1_bobmatnyc_entries_use_curated_sources_and_preserve_boundaries` — 1 passed. -- `CARGO_NET_OFFLINE=true cargo test --all-features --test all_tests unit::suggest_install::` — 10 passed. -- `cargo test --all-features commands::skill::tests::direct_catalog_install_resolution_uses_the_sibling_agents_skills_checkout` — 1 passed. -- `cargo test --all-features commands::skill::tests::suggestion_catalog_install_resolution_uses_the_project_root` — 1 passed. -- `CARGO_NET_OFFLINE=true RUN_E2E=1 cargo test --all-features --test test_catalog_integration every_catalog_skill_installs_successfully -- --ignored --nocapture` — intentional early-return message emitted; harness test returned `ok`; no full-catalog pass claimed. -- `cargo fmt --all -- --check` and `git diff --check` — no reported errors. -- `git status --short --branch` in both repositories — no commit or push performed; sibling migrated content/provenance and unrelated candidate content remain untracked. - -## Prior verification evidence retained - -- `cargo test --all-features` previously passed: 578 library tests, 188 binary tests, 123 `all_tests` integration tests, and all standalone integration suites shown by the command; ignored tests remained ignored. -- `cargo clippy --all-targets --all-features -- -D warnings` previously passed. -- `cargo llvm-cov --all-features --test all_tests --test test_catalog_integration --summary-only` previously completed with 33.75% aggregate line coverage and no asserted threshold. - -## Handoff - -This is technical verification only. It does not claim user/operator acceptance. Hand off to `sdd-qa` for capability-driven acceptance scenarios and `qa-report.md`; the existing QA report remains `NOT TESTED` because no launchable acceptance target or black-box harness is available. +- Commit/persist only the intended sibling provenance tooling and refreshed record before archive, then + rerun the provenance validator against the final merged bytes. +- Keep the Phase 1 harness explicitly scoped; do not convert its passing subset into a full-catalog + health claim. +- Have `sdd-qa` rerun the capability-driven acceptance report using the documented external target. + +## Final handoff + +This is technical verification only. The remediation passes with the warnings above, and `sdd-qa` +**may proceed** to perform acceptance QA against the external harness. QA must own the acceptance +verdict and update `qa-report.md`; this phase does not promote the previous `NOT TESTED` QA report to +user/operator acceptance or authorize archive.