test: add deterministic smoke coverage

.gitignore

@@ -4,6 +4,7 @@ __pycache__/
 *.pyo
 .DS_Store
 .firecrawl/
+.smoke/
 dist/
 *.tgz
 skills/bmad-story-automator/build/

docs/changelog/260602.md

@@ -0,0 +1,152 @@
+# Changelog - 260602
+
+## 260602-20:36:00 - Deterministic gate integration
+
+### Summary
+Promoted fast deterministic smoke gates into `verify` and added an explicit full deterministic release smoke wrapper.
+
+### Added
+- Added `smoke:deterministic-full` for prepared-repo reset, create, dev-loop, and finish-loop smoke coverage.
+
+### Changed
+- Updated `verify` to run version, package, CLI, helper contract, mode, Python, and shell smoke gates.
+- Updated Phase 06 gate map, implementation notes, TODO status, and handoff evidence.
+
+### Files
+- `docs/plans/automator-deterministic-smoke-coverage/TODO/phase-06.md`
+- `docs/plans/automator-deterministic-smoke-coverage/gate-map.md`
+- `docs/plans/automator-deterministic-smoke-coverage/handoff-log.md`
+- `docs/plans/automator-deterministic-smoke-coverage/implementation-notes.md`
+- `package.json`
+
+### QA Notes
+N/A
+
+## 260602-20:25:00 - Finish loop smoke coverage
+
+### Summary
+Added Phase 05 deterministic finish-loop coverage for automate, review, finalize, retrospective, wrapup, and host commit isolation.
+
+### Added
+- Added `smoke:finish-loop` and a temp git-backed finish-loop smoke runner.
+- Added assertions for automate done/skip, incomplete review diagnostics, review completion, smoke-repo commits, source-of-truth finalization, multi-epic retrospective skip semantics, wrapup marker removal, and host repo isolation.
+
+### Changed
+- Updated Phase 05 coverage baseline, gate map, implementation notes, TODO status, and handoff evidence.
+
+### Files
+- `docs/plans/automator-deterministic-smoke-coverage/TODO/phase-05.md`
+- `docs/plans/automator-deterministic-smoke-coverage/coverage-baseline.md`
+- `docs/plans/automator-deterministic-smoke-coverage/gate-map.md`
+- `docs/plans/automator-deterministic-smoke-coverage/handoff-log.md`
+- `docs/plans/automator-deterministic-smoke-coverage/implementation-notes.md`
+- `package.json`
+- `scripts/run-smoke-finish-loop.py`
+
+### QA Notes
+N/A
+
+## 260602-20:08:14 - Mode smoke coverage
+
+### Summary
+Added Phase 04 deterministic mode coverage for create startup, resume, validate, edit, and marker lifecycle helpers.
+
+### Added
+- Added `smoke:modes` and a temp-fixture mode smoke runner.
+- Added mode assertions for stop-hook states, existing-state discovery, workflow-derived route menus/hints, marker JSON/heartbeat lifecycle, validation/source-of-truth checks, and edit helper routes.
+
+### Changed
+- Updated Phase 04 coverage baseline, gate map, implementation notes, TODO status, and handoff evidence.
+
+### Files
+- `docs/plans/automator-deterministic-smoke-coverage/04-create-dev-resume-validate-edit-coverage.md`
+- `docs/plans/automator-deterministic-smoke-coverage/TODO/phase-04.md`
+- `docs/plans/automator-deterministic-smoke-coverage/coverage-baseline.md`
+- `docs/plans/automator-deterministic-smoke-coverage/gate-map.md`
+- `docs/plans/automator-deterministic-smoke-coverage/handoff-log.md`
+- `docs/plans/automator-deterministic-smoke-coverage/implementation-notes.md`
+- `package.json`
+- `scripts/run-smoke-modes.py`
+
+### QA Notes
+N/A
+
+## 260602-19:50:10 - Runtime helper contract smokes
+
+### Summary
+Added Phase 03 runtime helper contract smoke coverage and a named `smoke:contracts` gate.
+
+### Added
+- Added a skip-fail `smoke:contracts` runner for parser, monitor, runner, build-cmd, state-update, runtime-policy, marker/root, and success-verifier contract suites.
+- Added focused runtime helper contract tests for parser subprocess failure payloads, monitor terminal JSON, runner edge states, and `tmux-wrapper spawn` runner mode.
+
+### Changed
+- Updated Phase 03 coverage baseline, gate map, implementation notes, TODO status, and handoff evidence.
+
+### Files
+- `docs/plans/automator-deterministic-smoke-coverage/TODO/phase-03.md`
+- `docs/plans/automator-deterministic-smoke-coverage/coverage-baseline.md`
+- `docs/plans/automator-deterministic-smoke-coverage/gate-map.md`
+- `docs/plans/automator-deterministic-smoke-coverage/handoff-log.md`
+- `docs/plans/automator-deterministic-smoke-coverage/implementation-notes.md`
+- `package.json`
+- `scripts/run-smoke-contracts.py`
+- `tests/test_runtime_helper_contracts.py`
+
+### QA Notes
+N/A
+
+## 260602-19:41:15 - Package identity smoke contracts
+
+### Summary
+Added Phase 02 package tarball and prepared-repo install identity contracts.
+
+### Added
+- Added package contract assertions for npm pack contents, executable modes, forbidden generated files, tarball identity, and selected installed-file checksums.
+- Added smoke prep manifests for current package identity and installed automator manifests.
+
+### Changed
+- Updated smoke prepare to install the just-packed current tarball and verify prepared `.claude/skills` files against that tarball.
+- Updated Phase 02 plan evidence and package/install gate mapping.
+
+### Files
+- `docs/plans/automator-deterministic-smoke-coverage/TODO/phase-02.md`
+- `docs/plans/automator-deterministic-smoke-coverage/coverage-baseline.md`
+- `docs/plans/automator-deterministic-smoke-coverage/gate-map.md`
+- `docs/plans/automator-deterministic-smoke-coverage/handoff-log.md`
+- `docs/plans/automator-deterministic-smoke-coverage/implementation-notes.md`
+- `package.json`
+- `scripts/assert-package-contracts.py`
+- `scripts/smoke_prep/automator.py`
+- `scripts/smoke_prep/cli.py`
+- `scripts/smoke_prep/inputs.py`
+- `scripts/smoke_prep/package_contracts.py`
+- `scripts/smoke_prep/report.py`
+
+### QA Notes
+N/A
+
+## 260602-19:24:42 - Baseline deterministic smoke inputs
+
+### Summary
+Added Phase 01 deterministic smoke baseline, version alignment checks, and resolved smoke input recording.
+
+### Changed
+- Added version alignment and smoke input determinism scripts.
+- Aligned story automator workflow metadata to the current release version.
+- Updated smoke prep to record BMAD Method npm identity and install the resolved version.
+- Added Phase 01 coverage baseline, gate-map updates, implementation notes, and handoff evidence.
+
+### Files
+- `docs/plans/automator-deterministic-smoke-coverage/coverage-baseline.md`
+- `docs/plans/automator-deterministic-smoke-coverage/gate-map.md`
+- `docs/plans/automator-deterministic-smoke-coverage/handoff-log.md`
+- `docs/versioning.md`
+- `package.json`
+- `scripts/check-smoke-inputs.py`
+- `scripts/check-version-alignment.py`
+- `scripts/smoke_prep/inputs.py`
+- `skills/bmad-story-automator/workflow.md`
+
+### QA Notes
+N/A

docs/changelog/260618.md

@@ -0,0 +1,46 @@
+# Changelog - 260618
+
+## 260618-05:42:21 - Review remediation for deterministic smoke PR
+
+### Summary
+Resolved review findings around story parsing, deterministic smoke subprocess safety, package/prep error handling, marker cleanup, release docs, and changelog traceability.
+
+### Added
+- Added regression coverage for hyphenated explicit story IDs and bare numeric subheading parsing.
+- Added smoke-script contract tests for malformed npm/package JSON, `--help` behavior, subprocess timeouts, skip-install reports, and host env isolation.
+
+### Changed
+- Hardened smoke prep JSON/package assertions to fail with `SmokeError` instead of raw exceptions.
+- Split mode-smoke reporting out of the main runner and kept the runner under the repo LOC guard.
+- Updated release docs and plan evidence to distinguish fast `verify` gates from `smoke:deterministic-full`.
+
+### Files
+- `docs/changelog/260602.md`
+- `docs/changelog/260618.md`
+- `docs/development.md`
+- `docs/plans/automator-deterministic-smoke-coverage/README.md`
+- `docs/plans/automator-deterministic-smoke-coverage/gate-map.md`
+- `docs/plans/automator-deterministic-smoke-coverage/handoff-log.md`
+- `docs/versioning.md`
+- `scripts/run-smoke-automator.py`
+- `scripts/run-smoke-dev-loop.py`
+- `scripts/run-smoke-finish-loop.py`
+- `scripts/run-smoke-modes.py`
+- `scripts/smoke_prep/cli.py`
+- `scripts/smoke_prep/inputs.py`
+- `scripts/smoke_prep/mode_report.py`
+- `scripts/smoke_prep/package_contracts.py`
+- `scripts/smoke_prep/process.py`
+- `scripts/smoke_prep/report.py`
+- `skills/bmad-story-automator/src/story_automator/commands/orchestrator_epic_agents.py`
+- `skills/bmad-story-automator/src/story_automator/core/epic_parser.py`
+- `tests/test_cli_contracts.py`
+- `tests/test_smoke_script_contracts.py`
+
+### QA Notes
+- Focused regression suite: `PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=skills/bmad-story-automator/src python3 -m unittest tests.test_smoke_script_contracts tests.test_orchestrator_epic_agents tests.test_cli_contracts tests.test_epic_parser` pass.
+- Changelog timestamp assertion: pass for `docs/changelog/260602.md`.
+- `npm run pack:assert`: pass after package-surface correction.
+- `npm run verify`: pass after review-loop fix batch.
+- `git diff --check`: pass after review-loop fix batch.
+- Review-loop: parser reviewer had no actionable findings; runtime, docs, and contract findings fixed and reverified.

docs/development.md

@@ -14,9 +14,17 @@ PYTHONPATH=skills/bmad-story-automator/src python3 -m story_automator --help
 `npm run verify` expands to:
 
 - `npm run test:python`
-- `npm run pack:dry-run`
+- `npm run version:check`
+- `npm run pack:assert`
+- `npm run test:cli`
+- `npm run smoke:contracts`
+- `npm run smoke:modes`
 - `npm run test:smoke`
 
+Run `npm run smoke:deterministic-full` separately before release when prepared
+repo reset, install identity, create, dev-loop, and finish-loop coverage is
+needed.
+
 ## Smoke Test Coverage
 
 The smoke suite validates:
@@ -29,14 +37,30 @@ The smoke suite validates:
 - installed runtime policy, prompt templates, and parse contracts
 - prompt-building behavior for Claude and Codex child sessions
 
+## External Automator Smoke Prep
+
+Prepare the pinned `bma-d/gunz` smoke project in the ignored `.smoke/`
+workspace:
+
+```bash
+npm run smoke:prepare -- --reset
+```
+
+This clones only the `bmad-smoke-test` branch, checks out
+`fca6470d329668019dace305b5f0f3c9b62cb113`, installs BMAD core and BMM with
+`bmad-method@next`, then installs the project-local automator into that target
+project. The script leaves `.smoke/SMOKE_NEXT_STEPS.md` with the exact manual
+Claude Code entrypoint for the semi-automated smoke run.
+
 ## Repo Verification Flow
 
 ```mermaid
 flowchart TD
     A["Edit installer, skills, or runtime"] --> B["Run npm run test:python"]
-    B --> C["Run npm run pack:dry-run"]
-    C --> D["Run npm run test:smoke"]
+    B --> C["Run npm run pack:assert"]
+    C --> D["Run npm run smoke:contracts + smoke:modes"]
     D --> E["Run npm run verify"]
+    E --> F["Run npm run smoke:deterministic-full before release"]
 ```
 
 ## Packaging Surface
@@ -96,10 +120,11 @@ Publish steps:
 Recommended release checklist:
 
 1. `npm run verify`
-2. use `secrets` skill for npm auth material; search exact key names, then `secrets load <KEY>` into the publish shell; never print token values
-3. inspect the package dry-run output
-4. confirm README and docs match shipped behavior
-5. publish
+2. `npm run smoke:deterministic-full`
+3. use `secrets` skill for npm auth material; search exact key names, then `secrets load <KEY>` into the publish shell; never print token values
+4. inspect the package dry-run output
+5. confirm README and docs match shipped behavior
+6. publish
 
 For BMAD Method stable tags, preview tags, registry `next`, and npm dist-tags,
 use [Versioning And Release Channels](./versioning.md).

docs/plans/automator-deterministic-smoke-coverage/01-baseline-and-version-determinism.md

@@ -0,0 +1,53 @@
+# Phase 01 - Baseline And Version Determinism
+
+## Clean Context Start
+
+Before doing this phase, read [README.md](./README.md), this phase file, [TODO/phase-01.md](./TODO/phase-01.md), [gate-map.md](./gate-map.md), [implementation-notes.md](./implementation-notes.md), and relevant prior entries in [handoff-log.md](./handoff-log.md).
+
+Do not read later phase files or later TODO files as acceptance criteria for this phase.
+
+## Goal
+
+Create a source-of-truth coverage baseline for the automator workflow and add deterministic version/input checks so later smoke phases are not built on moving or stale package metadata.
+
+## Inputs
+
+- [skills/bmad-story-automator/workflow.md](../../../skills/bmad-story-automator/workflow.md)
+- [skills/bmad-story-automator/data/orchestration-policy.json](../../../skills/bmad-story-automator/data/orchestration-policy.json)
+- [package.json](../../../package.json)
+- [skills/module.yaml](../../../skills/module.yaml)
+- [skills/bmad-story-automator/pyproject.toml](../../../skills/bmad-story-automator/pyproject.toml)
+- [skills/bmad-story-automator/src/story_automator/__init__.py](../../../skills/bmad-story-automator/src/story_automator/__init__.py)
+- [.claude-plugin/plugin.json](../../../.claude-plugin/plugin.json)
+- [.claude-plugin/marketplace.json](../../../.claude-plugin/marketplace.json)
+
+## Implementation Steps
+
+1. Build a coverage inventory table that maps each automator mode and policy step to a deterministic gate status: `fact`, `gap`, `blocked`, `stale`, or `spec-only`.
+2. Add or update a repo-local deterministic metadata check that asserts version alignment across package, plugin, module, Python package, runtime `__init__`, and workflow frontmatter.
+3. Decide whether `bmad-method@next` should be pinned for deterministic prep or recorded/asserted as an explicit smoke input.
+4. Update [gate-map.md](./gate-map.md) with the metadata/version gate and any blocked input-pin gate.
+5. Record any stale metadata findings in [implementation-notes.md](./implementation-notes.md).
+
+## Verification
+
+- Run the new or updated metadata/version command.
+- Run the new or updated smoke input command.
+- Run `npm run test:cli`.
+- Run `git diff --check`.
+- Confirm [gate-map.md](./gate-map.md) has entries for version alignment and smoke input determinism.
+
+## Exit Criteria
+
+- Coverage baseline exists and classifies create, resume, validate, edit, create-story, dev-story, automate, review, commit/finalize, retrospective, wrapup, and package/install surfaces.
+- Stale version metadata either fixed or explicitly marked `stale` with a follow-up owner.
+- Moving external inputs are pinned or explicitly asserted.
+- Phase 01 handoff entry appended.
+
+## Implementation Notes Requirements
+
+Keep [implementation-notes.md](./implementation-notes.md) current while implementing. Record stale metadata, input pinning tradeoffs, and any reason a version surface cannot be aligned immediately.
+
+## Handoff Requirements
+
+Append a Phase 01 entry to [handoff-log.md](./handoff-log.md) with commands run, version surfaces checked, facts classified, blockers, and next recommended command.

docs/plans/automator-deterministic-smoke-coverage/02-package-and-prepared-repo-contracts.md

@@ -0,0 +1,59 @@
+# Phase 02 - Package And Prepared Repo Contracts
+
+## Clean Context Start
+
+Before doing this phase, read [README.md](./README.md), this phase file, [TODO/phase-02.md](./TODO/phase-02.md), [gate-map.md](./gate-map.md), [implementation-notes.md](./implementation-notes.md), and relevant earlier entries in [handoff-log.md](./handoff-log.md).
+
+Do not read later phase files or later TODO files as acceptance criteria for this phase.
+
+## Goal
+
+Harden the package and prepared-repo smoke contract so packed tarballs, installed files, dependency skill roots, and prepared `gunz` layout fail deterministically when installer drift occurs.
+
+This phase is release-blocking. A prepared repo smoke result is not trustworthy until it proves that `.smoke/gunz` installed the current packed tarball rather than a stale same-shape install.
+
+## Inputs
+
+- [scripts/prepare-smoke-test.py](../../../scripts/prepare-smoke-test.py)
+- [scripts/smoke_prep/](../../../scripts/smoke_prep)
+- [scripts/smoke-test.sh](../../../scripts/smoke-test.sh)
+- [scripts/run-smoke-automator.py](../../../scripts/run-smoke-automator.py)
+- [scripts/run-smoke-dev-loop.py](../../../scripts/run-smoke-dev-loop.py)
+- [package.json](../../../package.json)
+- [install.sh](../../../install.sh)
+
+## Implementation Steps
+
+1. Add a deterministic `npm run pack:assert` gate for required package files and forbidden generated/cache files using `npm pack --dry-run --json`.
+2. Capture and assert packed tarball identity in the same `npm run pack:assert` gate, or a separately named `npm run pack:identity` gate if implementation size warrants it, using `npm pack --json --pack-destination <tmp>`: package name, version, filename, integrity or shasum, generated tarball path, and selected checksums.
+3. Create or reuse a shared installed-file manifest for the real packed install into `.smoke/gunz`.
+4. Assert `.smoke/gunz` installed the current packed tarball, not a stale same-shape install: installed version surfaces match tarball metadata, selected installed files have checksums matching the tarball contents, and prep report records the tarball identity.
+5. Compare narrow installed-file checksums against the extracted tarball for `SKILL.md`, `scripts/story-automator`, policy JSON, parse contracts, prompt templates, `pyproject.toml`, review `contract.json`, and version surfaces.
+6. Extend `smoke:prepare` layout verification beyond helper `--help` to cover runtime source, policy JSON, parse/prompt files, templates, review skill contract, module metadata, and dependency skill entrypoints.
+7. Add a deterministic installed-root check for supported runtime roots when feasible: `.claude`, `.agents`, and `.codex`. If external BMAD install cannot prepare all roots, mark missing roots as `blocked` or `spec-only` in the coverage baseline.
+8. Use stable JSON and checksum assertions; do not accept terminal-output-only `npm pack` success as package proof.
+9. Update [gate-map.md](./gate-map.md) with package-content, installed-identity, installed-manifest, and prepared-repo gates.
+
+## Verification
+
+- Run the package content assertion.
+- Run `npm run smoke:prepare -- --reset`.
+- Run `npm run smoke:run`.
+- Run `npm run smoke:dev-loop`.
+- Run `git diff --check`.
+
+## Exit Criteria
+
+- The prepared external repo verifies the installed automator package identity and checksums, not just local source fixtures or same-shape installed files.
+- Package content assertions catch missing required files and unexpected generated files.
+- Unsupported or unavailable install roots are explicitly classified instead of silently ignored.
+- `npm run pack:assert` is fast enough for the future default `npm run verify`; prepared repo install checks remain explicit unless Phase 06 proves CI/runtime budget.
+- Phase 02 handoff entry appended.
+
+## Implementation Notes Requirements
+
+Record any installer-root limitations, BMAD Method pinning decisions, and package manifest tradeoffs in [implementation-notes.md](./implementation-notes.md).
+
+## Handoff Requirements
+
+Append a Phase 02 entry to [handoff-log.md](./handoff-log.md) with exact smoke prep command output summary, tarball/package details, installed manifest path, and next recommended command.