Add BoltSpec plan-testing suite (#76) - #77
Merged
Conversation
Unit coverage for the layer nothing tested before: plan logic, run with stubbed tasks/commands — no clones, no network, milliseconds per example. 13 examples across: - puppetsync::pipeline_stage: stage-list gating, skip_unchanged_targets (the simp#49 contract, via expect_command().with_targets), held-back- after-failure filtering, and the fatal non-Result error (simp#64) - The puppetsync plan's dynamic-inventory branch (simp#55): stubbed list_github_repos, snapshot written as a loadable repolist, static-over-generated merge precedence, list_pipeline_stages skips generation, and the no-repos_source passthrough - puppetsync::approve_github_prs / merge_github_prs (previously zero coverage of any kind): per-repo task params captured and asserted (unwrapping metadata-sensitive params), failure summarization fails the plan, stage-list opt-out - puppetsync::batch: one sync run per repolist, in order, via plan stubs Harness: spec/plans/plan_spec_helper.rb (shared context: modulepath incl. .modules, BoltSpec init, env-var defaults for token/Jira params, and the stub conventions learned during simp#76's probe — catch-all allow_out_message before specific expects, last-defined-wins). Fixture plans live in spec/fixtures/modules/puppetsync_test/, which is not on the real bolt-project modulepath. Wiring: plan specs require Bolt's Ruby (bolt_spec ships with openbolt), so .rspec excludes spec/plans from the plain-Ruby run and the CI bolt job runs them with -O /dev/null after installing rspec into .gems. Closes simp#76. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CI failed because `ruby -S rspec` only works when some rspec is already on PATH — locally it silently found the system-Ruby binstub from an earlier user install and then loaded libs from GEM_HOME, which is why the suite passed here. Invoke the .gems binstub explicitly instead (all docs/comments updated). Re-testing under a hostile environment (env -i, no PATH rspec, no PWD) also exposed that the approve/merge/batch specs relied on ambient PWD for the plans' project_dir default; they now pass project_dir explicitly. 13 examples green under `env -i PATH=/usr/bin:/bin`. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bundler isolation hides the packaged openbolt gem, and declaring openbolt in a Gemfile makes bundler vendor a second bolt from rubygems and test that copy instead of the OS package that runs real syncs (verified empirically). Record the rationale in the helper header so the question doesn't need re-deriving. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
nick-markowski
approved these changes
Jul 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #76
Adds the BoltSpec plan-testing suite planned in the issue: unit coverage for the layer nothing tested before — plan logic, run with stubbed tasks/commands. No clones, no network; the whole suite runs in ~1 second.
What's covered (13 examples)
puppetsync::pipeline_stage(spec/plans/pipeline_stage_spec.rb+ three fixture plans): stage-list gating, the Full idempotency: repos that need no changes must pass through cleanly #49skip_unchanged_targetscontract —expect_command('true').with_targets(['changed-repo'])makes BoltSpec require that the stubbed command ran only against the changed target — held-back-after-failure filtering (via a return-block building a mixed-exit-codeResultSet), and the Fail fast on unrecordable stage results instead of binding.pry #64 fatal error on non-Bolt-result stage returns.puppetsyncplan's dynamic-inventory branch (Build inventory dynamically from the GitHub API instead of hand-maintained repolists #55): withlist_github_reposstubbed, asserts the snapshot lands as a loadable repolist in a tmpdir project, staticrepos_configwins over generated entries,list_pipeline_stagesmode never calls the listing task (no stub declared — a call would raise), and the plain static path writes no snapshot.puppetsync::approve_github_prs/merge_github_prs— previously zero coverage of any kind: per-repo task parameters captured and asserted (target repo/branch from the repolist, fork user/branch, approval message), failure summarization fails the plan with the errors, and stage-list opt-out runs nothing.puppetsync::batch: onepuppetsyncrun per repolist, in order, viaexpect_planstubs.Harness & wiring
spec/plans/plan_spec_helper.rb— shared context: modulepath (fixtures +dist+modules+.modules),BoltSpec::Plans.init, env defaults for the token/Jira params, and the conventions from the probe (catch-allallow_out_messagebefore specific expects; stub matching is last-defined-wins).spec/fixtures/modules/puppetsync_test/— not on the real bolt-project modulepath, sobolt plan showis unaffected (verified).bolt_specships with openbolt), so.rspecexcludesspec/plans/from the plain-Ruby run, and the CI bolt job gets a step that installs rspec into.gemsand runs the suite with-O /dev/null.BoltSpec quirks encountered (documented in the helper / worth knowing for future specs)
"sensitive": truein task metadata arrive wrapped inSensitiveat the stub, and metaparams arrive as symbols — exactwith_paramsmatching fights this, so the suite uses return-blocks that capture and unwrap params, then asserts on the captures.return_for_targetsonly supports success shapes for commands; per-target failures need a return-block constructing aResultSetwith nonzeroexit_coderesults.Verification
--dry-runconfirming zero plan specs leak into the default runbolt plan showunaffected by the fixture module; workflow YAML validThe existing layers stay in place per the issue: subprocess specs own task internals, the
managed_filecontract script owns profile behavior (BoltSpec'sapply()support is stub-only), and the e2e scripts own the real-pipeline paths.🤖 Generated with Claude Code