Skip to content

Move to Cinc Workstation and add CI with automated releases - #27

Merged
jrwesolo merged 1 commit into
mainfrom
move-to-cinc-workstation
Sep 16, 2026
Merged

jrwesolo merged 1 commit into
mainfrom
move-to-cinc-workstation

Conversation

@jrwesolo

@jrwesolo jrwesolo commented Sep 16, 2026

Copy link
Copy Markdown
Owner

Moves the cookbook off Chef Workstation and onto Cinc Workstation, and adds CI that tests every pull request, then tags, releases, and publishes on merge.

Cinc migration

Test Kitchen selects Cinc through a single provisioner key, product_name: cinc. kitchen-dokken derives both the image (cincproject/cinc) and the client binary (/opt/cinc/bin/cinc-client) from it, so neither has to be set explicitly.

This deletes the Chef 19 Habitat workaround. That suite previously needed a separate chef/chef-hab image plus a hardcoded path into the Habitat package directory:

/hab/pkgs/chef/chef-infra-client/19.2.77/20260414044356/bin/chef-client

That path embedded a build timestamp and had to be updated by hand on every release. Cinc 19 ships as an omnibus package, so a plain version selector works.

Breaking change

The minimum supported Chef Infra Client is now 18, hence the major version bump to 3.0.0. Nodes on an older client will fail the chef_version metadata constraint. Users needing Chef Infra Client 12 through 17 should pin to ~> 2.4.

The driver for this is that cincproject/cinc:17 is published for amd64 only, with no manifest list at all, so the 17 suite cannot run on arm64 hosts. Rather than keep an untestable claim of support, the floor moves up to match what is actually exercised.

Test matrix

Suites now track major versions (18, 19) instead of exact pins, so Cinc patch releases no longer require a commit.

Platform coverage shifts toward what people actually deploy:

Before After Why
centos-stream-10 rockylinux-9 Stream 10 is upstream of RHEL 10; Rocky 9 is what is in production
ubuntu-25.10 ubuntu-24.04 25.10 is an interim release that goes EOL in July 2026
debian-13 unchanged current stable
fedora-43 unchanged leading indicator for future RHEL

Both new images were confirmed to publish amd64 and arm64 manifests. rocky is added to the supported platform list to match.

CI

New GitHub Actions workflow with separate lint, unit, and integration stages.

The integration matrix is generated at runtime by a discover job that runs kitchen list --json and emits the instance list. This keeps kitchen.yml as the single source of truth. Adding a platform or a Cinc major picks it up with no workflow edit. Each of the eight suite/platform combinations runs on its own runner, with fail-fast: false so one broken platform does not mask the others.

The Cinc Workstation version is pinned to the one documented in TESTING.md, so CI and the docs cannot drift and a bad upstream release cannot turn every pull request red on its own. Every job carries a timeout-minutes so a wedged container fails fast rather than holding a runner for the six hour default.

A version job runs on pull requests only. When a pull request bumps the version in metadata.rb, it asserts that the new version is not already tagged and that it is the newest entry in CHANGELOG.md with a link reference. When the version is unchanged from the base branch both assertions are skipped, so documentation-only pull requests are unaffected.

It does not run on main because the premise inverts there: once a release is tagged, metadata.rb keeps that version until the next bump, so every merge following a release would fail the check. It is also kept out of integration's needs, since a job skipped by a conditional causes its dependents to skip as well.

The check lives in .github/scripts/check-version rather than inline in the workflow, so it can be run and linted directly.

The Cinc Workstation install is factored into a composite action at .github/actions/setup-cinc, since GitHub Actions does not support YAML anchors and a composite action is the only way to share a step across jobs. The pinned version is that action's default input, so it lives in one place rather than in a workflow variable plus four references, and a single job can override it with with: version: without touching the shared default.

Releases

Merging to main now releases. Once lint, unit, and the whole integration matrix are green, a release job runs .github/scripts/release, which tags the merged commit and publishes a GitHub release whose notes are that version's section of CHANGELOG.md.

It is the mirror image of the pull request check. Both ask the same question, whether the tag already exists, and react in opposite directions:

tag exists tag does not exist
pull request fail, pick an unreleased version pass
merge to main exit 0, publish nothing tag and publish

That asymmetry is what makes every merge between releases a no-op instead of an error, and it means the release needs no state of its own to know whether it has already run.

The two scripts share .github/scripts/lib.bash, so they cannot drift on how the version is parsed. Drift there would let a release tag a version that the pull request check never validated.

Two supporting changes. cancel-in-progress is now scoped to pull requests, because a second merge landing mid-release should not cancel the first run. And contents: write is granted to the release job alone; the workflow default stays contents: read.

Publishing to Supermarket

A publish job follows release and shares the cookbook to Supermarket with knife supermarket share.

It is gated on a GitHub environment named supermarket, configured with a required reviewer, self-review allowed, and a deployment branch policy limited to main. The job therefore does not start, and the Supermarket key does not become readable, until the deployment is approved by hand. Supermarket cannot issue a scoped or expiring credential, so an approval gate is the mitigation available for keeping a long-lived key in Actions. Both SUPERMARKET_USER and SUPERMARKET_KEY live on that environment.

release now reports whether it actually published, and publish is skipped when it did not:

    outputs:
      released: ${{ steps.release.outputs.released }}
    if: needs.release.outputs.released == 'true'

Without that, every documentation merge would queue an approval request for a release that never happened, and an approval you learn to click without reading is worse than no gate at all.

Why publishing follows tagging

Supermarket evaluates a version_tag quality metric at share time. From its source, VersionTagWorker#evaluate:

if tags.include?(cookbook_version) || tags.include?("v#{cookbook_version}")

It asks GitHub for the repository's tags and accepts either spelling, so the v prefix used here is fine. What it does not tolerate is the tag not existing yet, and nothing re-evaluates the metric afterwards.

v2.4.1 currently fails this metric, and the reason is visible in the timestamps:

Event Time (UTC)
Shared to Supermarket 22:53:09
GitHub release v2.4.1 created 22:57:11
GitHub release v2.4.1 published 22:59:09

It was shared four minutes before its tag existed. That version cannot be repaired, but ordering publish after release makes the correct sequence a property of the pipeline rather than something to remember. v3.0.0 will pass.

Idempotency

publish asks Supermarket whether the version is already there and exits without uploading if so, so a re-run of a partially failed job is safe. The check distinguishes 404 from an unanswered request, and treats anything other than 200 or 404 as an error rather than as permission to publish.

The tag is resolved and verified before anything else happens, so a publish without a release fails on the fundamental problem rather than on a symptom of it. Everything after that point reads its identity from the tag rather than the working tree, and the version, which has to be read first since it is what names the tag, is checked against the tag rather than trusted.

The cookbook is staged by archiving the tag, not by archiving whatever is checked out. In the workflow those are the same commit, but only by implication: actions/checkout puts both jobs on the run's github.sha and the release job tags that same sha. Naming the tag makes the guarantee explicit, makes the script correct when re-run or used by hand, and turns a publish without a release into a loud failure rather than a quietly wrong upload.

The key is written under a subshell umask 077 into a staging directory that mktemp -d already creates at 0700, and the whole directory is removed on exit. The umask is deliberately not global: tar -x would then extract the cookbook at 0600, and knife stages files with FileUtils.cp, which preserves the source mode into the published tarball.

Worth flagging before merge: this pull request is what creates the release and publish jobs, so merging it will tag v3.0.0, then wait for your approval before sharing it to Supermarket.

Scheduled runs

The test jobs also run weekly, Mondays at 12:00 UTC. The suites track Cinc major versions and the dokken base images move underneath those tags, so a green main can go red with no change in this repository.

A scheduled run cannot tag or ship anything: version is pull-request-only, release is push-only, and publish needs release. Worth knowing that GitHub disables scheduled workflows after 60 days of repository inactivity, and notifies the owner when it does.

Concurrency

Pushes to main no longer share a concurrency group with each other:

  group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event_name == 'pull_request' && 'pr' || github.run_id }}

Pull request runs still share a group per ref, so a new push cancels the run it supersedes. Pushes to main get a group to themselves because the publish job holds its run open while it waits for a deployment approval, and pending approvals live for thirty days. A shared group would park every later merge behind an unapproved release.

Also

  • Drop the unused default_source :supermarket from the Policyfile. Every cookbook in it resolves from a local path, so nothing was ever fetched from Supermarket.
  • Apply Cookstyle fixes, including making unified_mode true unconditional now that the client floor is 18.
  • Ignore the undotted kitchen.local.yml, matching the undotted kitchen.yml already in use.
  • Add a CI status badge to the README, pinned to main so it reports the branch rather than whatever ran most recently. The existing reference links were renumbered to keep numeric order matching document order, which is the convention the file already used.
  • Drop release dates from CHANGELOG.md. The tag and the GitHub release already record when a version shipped, and a date written by hand while a pull request is open is wrong whenever that pull request does not merge the same day. The headings also lose their redundant collapsed-link brackets, [v3.0.0][] becoming [v3.0.0], which resolves against the same link definition.
  • Correct chefignore. Chef matches its globs with File.fnmatch and no flags, so a bare directory name never matches the files beneath it and .github had to become .github/*. For the same reason kitchen.yml* did not cover an undotted kitchen.local.yml, which knife would otherwise pack out of a developer's working directory, so it becomes kitchen*.yml*. chefignore now excludes itself as well. Every tracked path was enumerated against the final glob list: the published artifact is metadata.rb, LICENSE, README.md, libraries/ and resources/, and nothing else.
  • Correct two resource comments that still cited a Chef 16 compatibility concern, and replace the commented-out ChefSpec requires in spec/spec_helper.rb with a note on why ChefSpec is not used here (the resources read and write the live filesystem at converge time).

Testing

Cookstyle and RSpec are clean locally: 17 files no offenses, 66 examples 0 failures.

The published artifact was verified with a real knife supermarket share --dry-run, which builds the actual tarball and lists it without uploading. Of 31 tracked files, 10 are published: metadata.rb, the generated metadata.json, LICENSE, README.md, libraries/, and resources/. No spec/, no test/, no kitchen.yml, no .github/, no chefignore, no Policyfile. That is the first direct confirmation that the chefignore corrections above do what they claim.

The scripts were exercised directly against a scratch clone rather than only in CI. check-version was run through seven cases: a bumped and documented version, a version unchanged from the base branch, an already-tagged version, a stale changelog heading, a missing link reference, that same case with the reference added, and an unparseable metadata.rb. release was run through four against a stubbed gh: untagged with notes present, already tagged, untagged with no changelog section, and a missing GITHUB_SHA, confirming in the first two cases that it writes released=true and released=false respectively. publish was run through three against a stubbed cinc: a version absent from Supermarket, one already present, and an unreachable Supermarket, checking in the first that the staged key file lands at mode 600 and that the staging directory is cleaned up. Changelog note extraction was checked for the newest entry, a middle entry, and the oldest, which is bounded by the link reference block rather than a following heading.

CI ran green on the full eight instance integration matrix, covering Cinc 18 and 19 across Debian 13, Rocky Linux 9, Fedora 43, and Ubuntu 24.04. That was this platform set's first run, and its first run on amd64.

@jrwesolo
jrwesolo force-pushed the move-to-cinc-workstation branch 10 times, most recently from 245d3fa to 9dec8d7 Compare September 16, 2026 04:04
Replace Chef Workstation with Cinc Workstation throughout development and
testing. Test Kitchen now selects Cinc through `product_name: cinc`, which
kitchen-dokken uses to derive both the image (`cincproject/cinc`) and the
client binary (`/opt/cinc/bin/cinc-client`), so neither needs to be set
explicitly.

This removes the Chef 19 Habitat workaround entirely. That suite needed a
separate `chef/chef-hab` image plus a hardcoded path to the client inside
the Habitat package directory, which had to be updated by hand on every
release. Cinc 19 ships as an omnibus package, so a plain version selector
works.

The Chef Infra Client 17 suite is dropped because `cincproject/cinc:17` is
published for amd64 only and cannot run on arm64 hosts. The minimum
supported client is raised to 18 to match, which is a breaking change and
the reason for the major version bump.

Suites now track major versions rather than exact pins, so patch releases
no longer require a commit. Platform coverage moves toward what is actually
deployed: CentOS Stream 10 is replaced by Rocky Linux 9, and the interim
Ubuntu 25.10 release by Ubuntu 24.04 LTS. Rocky Linux is added to the
supported platform list to match.

Add GitHub Actions CI with separate lint, unit, and integration stages. The
integration matrix is derived from `kitchen list --json` so that kitchen.yml
remains the single source of truth, and each suite and platform combination
runs on its own runner in parallel. The Cinc Workstation version is pinned
to the one documented in TESTING.md so that an upstream release cannot break
every pull request on its own, and every job carries a timeout so a wedged
container fails fast instead of holding a runner for hours.

A pull request that bumps the version is also checked for two things: that
the new version is not already tagged, and that it is the newest entry in
the changelog with a link reference. Both are skipped when the version is
unchanged, so documentation-only pull requests are unaffected. The check
runs on pull requests only, because on main the premise inverts: once a
release is tagged, metadata.rb keeps that version until the next bump, so
every merge that follows a release would fail it. It is deliberately left
out of the integration gate, since a job skipped by a conditional also
skips the jobs that need it. The check lives in
.github/scripts/check-version rather than inline in the workflow so that it
can be run and linted directly.

Merges to main release automatically. Once lint, unit and the full
integration matrix are green, .github/scripts/release tags the merged commit
and publishes a GitHub release whose notes are the matching section of the
changelog. It is the mirror image of the pull request check: both ask
whether the tag already exists, and where the check fails a pull request
that reuses a released version, the release simply exits without publishing.
That is what makes every merge between releases a no-op rather than an
error. The two scripts share .github/scripts/lib.bash so they cannot drift
on how the version is parsed, which would otherwise allow a release to tag a
version that was never validated. Cancelling in-progress runs is now limited
to pull requests, so a second merge cannot interrupt a release, and write
permission is granted to the release job alone rather than to the workflow.

A successful release then shares the cookbook to Chef Supermarket. That job
sits behind a GitHub environment, so the deployment waits for a human to
approve it before the job starts and before the Supermarket key becomes
readable. Supermarket has no way to issue a scoped or expiring credential,
so an approval gate is the available mitigation for keeping a long-lived key
in Actions. The release job now reports whether it actually published, and
the share is skipped when it did not, so a merge that ships no new version
never asks for an approval it does not need.

The share resolves the tag by name and archives that, rather than archiving
whatever is checked out. In the workflow the two are the same commit, but
only by implication, and the implication breaks the moment the script is
re-run or used by hand. Resolving the tag also makes a publish without a
release fail loudly instead of quietly shipping the wrong tree.

Sharing deliberately follows tagging rather than running beside it.
Supermarket evaluates its version tag quality metric at share time by asking
GitHub for the repository's tags, accepting either the bare version or a
v-prefixed one, and nothing re-evaluates that metric afterwards. Version
2.4.1 was shared four minutes before its tag existed and fails the metric
permanently as a result. Tagging first makes that ordering a property of the
pipeline instead of something to remember.

The test jobs also run weekly. The suites track Cinc major versions and the
dokken base images move underneath those tags, so a green main can go red
with no change here. A scheduled run cannot tag or ship anything, since the
version job is pull-request-only and release and publish are push-only.

Pushes to main no longer share a concurrency group. The publish job holds
its run open while it waits for a deployment approval and pending approvals
live for thirty days, so a shared group would park every later merge behind
an unapproved release.

Release dates are dropped from the changelog, and the README gains a CI
status badge. The tag and the GitHub release
already record when a version shipped, and a date written by hand while a
pull request is open is wrong whenever that pull request does not merge the
same day.

The Cinc Workstation install is factored into a composite action at
.github/actions/setup-cinc. GitHub Actions does not support YAML anchors, so
a composite action is the only way to share a step across jobs. The pinned
version becomes that action's default input, which puts it in one place
rather than in a workflow-level variable plus four references, and lets an
individual job override it without touching the shared default.

Also drop the unused `default_source :supermarket` from the Policyfile, as
every cookbook in it is resolved from a local path, and apply the Cookstyle
fixes that the raised client floor makes available. Correct two resource
comments that still referred to a Chef 16 compatibility concern the raised
floor has made moot, and ignore the undotted `kitchen.local.yml`.

Correct chefignore as well. Chef matches its globs with File.fnmatch and no
flags, so a bare directory name never matches the files beneath it: `.github`
had to become `.github/*`. For the same reason `kitchen.yml*` did not cover
an undotted `kitchen.local.yml`, which knife would otherwise pack out of a
developer's working directory, so it becomes `kitchen*.yml*`. chefignore now
also excludes itself. The published artifact is left as metadata.rb, LICENSE,
README.md, libraries and resources.
@jrwesolo
jrwesolo force-pushed the move-to-cinc-workstation branch from 9dec8d7 to fda7181 Compare September 16, 2026 04:17
@jrwesolo jrwesolo changed the title Move development and testing to Cinc Workstation Move to Cinc Workstation and add CI with automated releases Sep 16, 2026
@jrwesolo
jrwesolo marked this pull request as ready for review September 16, 2026 04:21
@jrwesolo
jrwesolo merged commit 9859cf8 into main Sep 16, 2026
14 checks passed
@jrwesolo
jrwesolo deleted the move-to-cinc-workstation branch September 16, 2026 04:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant