Skip to content

Add a host image for POSIX builds and the QA gates - #23

Merged
hacker-cb merged 1 commit into
masterfrom
add-host-image
Aug 23, 2026
Merged

Add a host image for POSIX builds and the QA gates#23
hacker-cb merged 1 commit into
masterfrom
add-host-image

Conversation

@hacker-cb

Copy link
Copy Markdown
Contributor

A fourth image, for the target the other three do not cover: the machine the build
runs on. It compiles and tests a C++ project against the host's own toolchain and
carries the QA gates that run beside such a build — ruff, mypy, pytest,
clang-format, clang-tidy and lychee.

Why an image rather than a list of apt-get install lines in a CI job

A static analyzer reads the host's standard library, not the project's. libc++
(macOS) inlines the throw inside std::string and std::function; libstdc++
(Linux, and every CI runner) hides them behind external __throw_*, where the
analyzer cannot follow. Same pinned clang-tidy, different findings — so "tidy is
green" means one thing on a laptop and another in CI, and the disagreement is
invisible until a colleague reports it. This image is the Linux answer, and it is
the same answer on either architecture: both linux/amd64 and linux/arm64 are
built natively, so a developer on Apple Silicon is not waiting on QEMU for the
analysis they asked for.

What is in it

Layer Contents
apt build-essential (the sanitizer runtimes come with it), cmake, ninja-build, ccache, pkg-config, libgtest-dev, libgmock-dev, git, curl, jq, python3, python3-venv
pip, in a venv first on PATH ruff, mypy, pytest, jsonschema, clang-format, clang-tidy
from source paho.mqtt.c, fetched at a pinned commit and built with SSL off and HIGH_PERFORMANCE on
release binary lychee, verified against a per-architecture SHA-256

Ubuntu 24.04 ships PEP 668, so a plain pip install into /usr refuses; the venv
is what makes ruff and clang-tidy callable by bare name anyway.

Pins

Everything the image installs by name is pinned, in the regime
images/versions.json's _readme already describes and scripts/check-pins.sh
enforces: pip with ==, paho to a commit rather than a mutable tag, lychee to a
literal checksum per architecture — the amd64 tarball is not the arm64 one, so a
single literal would leave one leg unbuildable.

Only UBUNTU_BASE_TAG reaches CI as a build arg, because a variant's tag has to
name every value it is built with and a tool version can never appear there. The
rest are ARG defaults, and _readme, CLAUDE.md and .github/dependabot.yml
now say so.

Two tools are wired up, not merely installed

Installing ccache does not put it in the path of a build: /usr/lib/ccache
ships gcc and g++ wrappers but no cc or c++, which are the names CMake
looks for first — so a PATH entry leaves a CMake project compiling straight
through /usr/bin/c++ with the cache untouched. CMAKE_{C,CXX}_COMPILER_LAUNCHER
is read by CMake itself and holds whatever compiler the project picks.

CMAKE_EXPORT_COMPILE_COMMANDS=ON is what makes clang-tidy -p build resolve
after an ordinary cmake -S . -B build, rather than after a re-configure with one
more flag nobody remembers.

The verification layer asserts rather than lists

Beyond printing versions, the last layer requires that:

  • clang-format and clang-tidy report the numbers pip was told to install —
    the wheel and the wrapper it installs are two different things;
  • jsonschema appears at its pinned version in the freeze it writes to
    /opt/qa-packages.txt — it is the one pin no tool here prints a version for;
  • CMake actually took the ccache launcher;
  • images/host/smoke/, a two-file CMake project, configures, builds and passes
    under ctest — proving find_package(GTest) resolves, that GMock links, and
    that the paho which loads reports the version pinned beside its commit;
  • clang-tidy then runs, over that project's own source and the compile
    database the build just wrote. A wheel whose binary cannot find its resource
    directory answers --version perfectly and fails on the first real file.

That build runs with CCACHE_DISABLE=1: warming the cache as root would leave
/opt/ccache/tmp root-owned and every later -u $(id -u) run failing on it. The
smoke sources stay at /opt/smoke-src, so the same check can be re-run against a
published image — the README says how.

Nothing in the Dockerfile pipes a tool's output anywhere. The image sets no
SHELL, dash has no pipefail, and gcc --version | head -1 would take head's
exit status — leaving a missing compiler green in the layer written to catch
exactly that.

Running as the invoking user

The documented invocation mounts a checkout and passes -u $(id -u):$(id -g), so
the repository is owned by a uid the image has no passwd entry for:

  • git config --system --add safe.directory '*' — without it git refuses the
    checkout as dubious ownership and takes git ls-files and git describe with
    it, inside the container only. It stays '*' rather than naming /workspace,
    because a consumer running this image as a job container: gets its checkout
    under /__w/<repo>/<repo>, which a narrowed entry would refuse.
  • HOME=/home/build rather than /tmp, which a caller is entitled to replace
    (--tmpfs /tmp, a volume over it, a cleanup step) and would take HOME with it
    mid-run.
  • 1777 on both that and the ccache directory: an arbitrary uid has to write
    them, and the sticky bit keeps one uid from replacing another's files.

CI

host.yml is its own workflow — nothing in this repo is built FROM this image,
so like platformio.yml it builds on every branch and pushes on master alone.
Same three-job shape as the other families: prepare resolves the matrices from
images/versions.json, host-build builds each platform on a runner of its own
architecture and pushes by digest, host-manifest assembles the multi-arch tags —
ubuntu-<version>, ubuntu-<version>-sha-<short-commit>, plus latest and
sha-<short-commit> for the primary variant.

Two documentation sentences in CLAUDE.md said "both workflows" where there are
now three, and are corrected with them.

Verified locally

Built on linux/arm64 and exercised against the built image: every example in
images/host/README.md (build, ctest, git ls-files/git describe under a
foreign uid, clang-format, clang-tidy -p build, lychee, sanitizer build,
ccache hit, ruff/mypy/pytest), plus ./scripts/lint.sh and
./scripts/check-versions.sh. The verification layer was checked negatively too:
building with a substituted PAHO_VERSION fails the image, as it should.

A static analyzer reads the host's standard library, not the project's:
libc++ inlines the throw inside std::string and std::function, libstdc++
hides them behind external __throw_*, and the same pinned clang-tidy
therefore reports findings on macOS that CI never sees - and misses code
that is only red on a laptop. This image is the Linux answer, buildable
and runnable on either architecture.

It also replaces the apt-plus-build-paho preamble a consuming CI job runs
before every POSIX leg: build-essential, CMake, Ninja, ccache and
GTest/GMock, paho.mqtt.c built from a pinned commit with the same flags
that job used, and the QA toolchain (ruff, mypy, pytest, jsonschema,
clang-format, clang-tidy, lychee) in a venv first on PATH - Ubuntu 24.04
ships PEP 668, so there is no system interpreter to install into.

Everything it installs by name is pinned: pip with ==, paho to a commit
rather than a mutable tag, lychee to a per-architecture SHA-256 (the
amd64 tarball is not the arm64 one, so a single literal would leave one
leg unbuildable). Only UBUNTU_BASE_TAG reaches CI as a build arg, because
it is the one value the variant tag can name; the rest are ARG defaults,
the regime images/versions.json's _readme describes and check-pins.sh
enforces.

Two tools are wired up rather than merely installed, because installed is
not the same as in the path of a build. /usr/lib/ccache ships gcc and g++
wrappers but no cc or c++ - the names CMake looks for first - so a PATH
entry would leave a CMake project compiling through /usr/bin/c++ with the
cache untouched; CMAKE_{C,CXX}_COMPILER_LAUNCHER is read by CMake itself
and holds whatever compiler the project picks. CMAKE_EXPORT_COMPILE_COMMANDS
is what makes `clang-tidy -p build` resolve after an ordinary
`cmake -S . -B build`, rather than after a re-configure with one more flag.

The verification layer asserts rather than lists. The two clang tools have
to report the numbers pip was told to install - the wheel and the wrapper
are different things - jsonschema has to appear at its pinned version in
the freeze, CMake has to have taken the ccache launcher, and smoke/ is
configured, built and ctest-ed inside the build: it proves
find_package(GTest) resolves, that GMock links, and that the paho which
loads reports the version pinned beside its commit. clang-tidy is then run
over that same source, because a wheel whose binary cannot find its
resource directory answers --version perfectly and fails on the first real
file. That build runs with CCACHE_DISABLE=1: warming the cache as root
would leave /opt/ccache/tmp root-owned and every later `-u $(id -u)` run
failing on it. Nothing in the file pipes a tool's output anywhere, because
the image sets no SHELL and dash has no pipefail, so `gcc --version |
head -1` would take head's exit status and leave a missing compiler green.

The image runs as the invoking user: git safe.directory (a mounted
checkout is owned by a uid with no passwd entry, and without it both
`git ls-files` and `git describe` fail inside the container only), a HOME
of the image's own rather than /tmp, which a caller may replace mid-run,
and 1777 on both that and the ccache directory - an arbitrary uid has to
write them, and the sticky bit keeps one uid from replacing another's
files.

host.yml is its own workflow - nothing in this repo is built FROM it, so
it pushes on master alone, like platformio.yml.

The prose that counted the workflows is corrected with them: two sentences
in CLAUDE.md said "both workflows" where there are now three, and
dependabot.yml's list of what it deliberately does not track gains this
image's pins - ARG defaults, a commit ref and two literal checksums, none
of which any ecosystem resolves.
Copilot AI lite review requested due to automatic review settings August 23, 2026 07:56

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new host Docker image to this repository’s images/<name>/ roster, intended for POSIX (host) builds and QA gates (CMake/Ninja/GTest/GMock, plus ruff/mypy/pytest/clang-format/clang-tidy/lychee), along with the corresponding CI workflow and documentation updates.

Changes:

  • Introduces images/host/ (Dockerfile + smoke-test CMake project + image README) to provide a reproducible Linux environment for host builds and static analysis on both amd64 and arm64.
  • Extends images/versions.json with a host image definition (platform runners, timeout, and the Ubuntu base tag arg) so matrices are generated consistently via existing scripts.
  • Adds .github/workflows/host.yml to build/push the new image family (build validation on PRs, tag publication on master) and updates root/guide docs to reference the new image.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.

Show a summary per file
File Description
README.md Adds the Host workflow badge, adds host to the images table, and updates repo layout/docs references accordingly.
images/versions.json Registers the new host image, including its platforms and build args, enabling matrix generation and version consistency checks.
images/host/smoke/smoke.cpp Adds a small runtime/link smoke test covering GMock linkage and Paho version reporting.
images/host/smoke/CMakeLists.txt Adds the CMake build definition for the smoke test used in the Dockerfile verification layer.
images/host/README.md Documents tags, usage patterns, pin verification, and CI/container usage for the host image.
images/host/Dockerfile Implements the host image build (apt + pinned venv tools + pinned Paho build + pinned lychee) and includes a verification layer that asserts installs/linking/tool behavior.
CLAUDE.md Updates repository conventions to include the new host image and workflow in the documented roster/patterns.
.github/workflows/host.yml Adds a new CI workflow to build multi-arch host images using matrices derived from images/versions.json.
.github/dependabot.yml Documents that host image pins are intentionally manual (ARG defaults / commit pin / checksums), outside Dependabot’s scope.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@hacker-cb
hacker-cb merged commit fb3ecbb into master Aug 23, 2026
23 checks passed
@hacker-cb
hacker-cb deleted the add-host-image branch August 23, 2026 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants