From a0759b4f3dbdce68c903feec681eeeb6d4fee06d Mon Sep 17 00:00:00 2001 From: Pavel Sokolov Date: Mon, 24 Aug 2026 05:50:21 +0300 Subject: [PATCH 1/2] Label the ESP-IDF version on both ESP images MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes #26. A consumer that pins these images by digest - the only pin nothing can move underneath it - deliberately does not keep the tag, so the tag is the one place it cannot read the version from. Until now that left no cheap way to check which ESP-IDF an image carries: the answer existed only inside it, behind a 3.6-7.5 GB pull. The Matter half already had the registry-readable half of this (org.opencontainers.image.revision carries ESP_MATTER_REF); the IDF half had nothing, since org.opencontainers.image.version there is 24.04, inherited from the ubuntu base. Both images now carry `dev.jethome.idf.version`, in the same family the host image uses. esp-idf has the value already - IDF_BASE_TAG, which names its own base - and only needs it re-declared after FROM, or the LABEL would expand to the empty string without Docker objecting. esp-matter takes a new IDF_VERSION arg. Not because the label needs it - Docker inherits a base image's LABELs, so building on this repository's esp-idf would carry the label down by itself - but because inheritance cannot *check* the value, and carries nothing at all when BASE_IMAGE points elsewhere. CI passes that base as a digest, and a digest names no version, so the value comes from images/versions.json, taken from the same `base_tag` the digest was resolved from. That is the same value written twice, which is safe because the checker refuses a variant built with a version its own tag does not name: setting IDF_VERSION to v5.5.5 on the variant tagged idf-v5.4.1-matter-v1.4.2 fails ./scripts/check-versions.sh, verified. **The label is asserted, not declared.** Both verification layers already ran `idf.py --version` and printed it; both now compare it for equality against the labelled version, so an image whose label contradicts its contents fails to build rather than shipping. Equality rather than a substring, because `*v5.4.1*` also matches v5.4.11 - and with the expected value in the failure message, since a build arg is not otherwise visible in the log. esp-idf asserts only when IDF_BASE_TAG *is* a version. Its README offers `latest` and minor-line tags like `v5.3` as valid bases, and an alias never equals the release it resolves to, so a strict comparison would have failed builds that are perfectly valid. The skip is announced rather than silent, and applies locally only: CI always passes an exact version, so every published image is asserted. The docs get the same treatment as the code. The registry-side recipe needs the platform key - `{{ index (index .Image "linux/amd64").Config.Labels … }}` - because these tags are multi-arch indexes and `.Image.Config` on an index is a template error, not an answer; both forms were run against the published images. The `docker image inspect` variant is marked as reading the local daemon rather than the registry. And every custom-build recipe that overrides BASE_IMAGE now passes IDF_VERSION beside it, in the image README and in CLAUDE.md, or the build asserts the default version and fails after the expensive SDK install. Note on what was NOT verified locally: `espressif/idf:v5.5.5` would not pull here (Surge intercepts traffic in TUN mode and drops the connection around 400 MB, while that image carries a single 3.4 GB layer), so neither ESP image was built on this machine. The exact output shape - one line, `ESP-IDF v` - was read out of the master run's logs for four combinations: both IDF versions, both images, both architectures. The assertion's own logic was exercised in a plain container against a match, a mismatch, an alias and an empty answer. The build itself is checked by the pull request, which builds both images on both architectures. --- CLAUDE.md | 12 ++++++++++-- images/esp-idf/Dockerfile | 30 ++++++++++++++++++++++++++++- images/esp-idf/README.md | 37 ++++++++++++++++++++++++++++++++++++ images/esp-matter/Dockerfile | 23 ++++++++++++++++++++-- images/esp-matter/README.md | 29 ++++++++++++++++++++++++++++ images/versions.json | 9 ++++++--- 6 files changed, 132 insertions(+), 8 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 3128492..c3e5f07 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -78,7 +78,12 @@ before changing anything here. its own tag, so base and advertised version cannot drift apart. It reaches the Dockerfile through a single `BASE_IMAGE` build-arg — one argument rather than repo + tag, because a digest needs `@` where a tag needs `:`, and because it makes - the base repository overridable for a fork or a local build. + the base repository overridable for a fork or a local build. A digest names no + version, so the ESP-IDF version travels separately as `IDF_VERSION` in `args`, + taken from the same `base_tag`; both ESP images label it as + `dev.jethome.idf.version` and both assert it against `idf.py --version`, which is + what keeps a label a consumer reads from the registry from outliving the base it + describes. - **`images/versions.json` is the single source of truth for what CI builds.** A `prepare` job runs `scripts/check-versions.sh`, then `scripts/versions-matrix.sh build|manifest`, and every other job takes its matrix from @@ -360,7 +365,10 @@ Four places, none of them checked automatically: the image afterwards and only applies to a single named image. Building `esp-matter` this way pulls its base from GHCR — to build it on an esp-idf you just built, pass the base explicitly: - `docker build --build-arg BASE_IMAGE=jethome-dev-esp-idf:local images/esp-matter`. + `docker build --build-arg BASE_IMAGE=jethome-dev-esp-idf:local --build-arg + IDF_VERSION= images/esp-matter` — the second arg because the version is + asserted against the base, and its default belongs to the *published* base + rather than to whatever you just built. Note that a version-bump branch cannot use the Dockerfile default at all: it names a tag that only exists once the branch lands. - There is no local workflow runner. Workflow changes are validated by pushing the diff --git a/images/esp-idf/Dockerfile b/images/esp-idf/Dockerfile index 879a01b..f659f89 100644 --- a/images/esp-idf/Dockerfile +++ b/images/esp-idf/Dockerfile @@ -6,6 +6,19 @@ ARG IDF_BASE_TAG=v5.5.5 FROM espressif/idf:${IDF_BASE_TAG} +# Re-declared after FROM: an ARG stated before it is out of scope below, so the +# LABEL would otherwise expand to the empty string - silently, since Docker does +# not object to an undefined variable. +ARG IDF_BASE_TAG + +# The ESP-IDF version this image carries, readable from the registry without +# pulling several gigabytes to find out. The tag says it too, but a consumer that +# pins by digest - the only pin that cannot be moved under it - deliberately does +# not keep the tag, so the tag is exactly what it cannot read. The verification +# layer at the bottom holds this to what the SDK itself reports, so it is a fact +# about the image rather than a claim it makes. +LABEL dev.jethome.idf.version="${IDF_BASE_TAG}" + # Use bash for all RUN commands: the layers below call the bash builtin `source`, # which /bin/sh does not have. (This once said "needed for QEMU emulation # compatibility" - CI no longer emulates anything, but the reason above always was @@ -68,6 +81,13 @@ WORKDIR /workspace # The freeze is written to a file rather than piped, so the image carries its own # version snapshot and no pipe can swallow a failure. # +# The idf.py line is an assertion too, and for a reason of its own: the +# dev.jethome.idf.version label near the top of this file is what a consumer reads +# instead of pulling several gigabytes, and a label is only worth reading if the +# image cannot publish one that contradicts itself. Compared for equality against +# what the SDK reports (`ESP-IDF v5.5.5`), so a tag naming one version while the +# base carries another fails here rather than downstream. +# # The esptool line is an assertion, not a listing: `grep -E '^esptool'` would pass # on any version, and "any version" is exactly the bug - ESP-IDF's own environment # ships 4.12.0 and anything reinstalling it would go unnoticed. It repeats the @@ -82,7 +102,15 @@ RUN export IDF_PATH_FORCE=1 && \ grep -qx 'esptool==5.3.1' /opt/esp/python-packages.txt && \ grep -E '^(pytest|esptool)' /opt/esp/python-packages.txt && \ jq --version && \ - idf.py --version && \ + idf_reported="$(idf.py --version)" && \ + echo "${idf_reported}" && \ + { case "${IDF_BASE_TAG}" in \ + v[0-9]*.[0-9]*.[0-9]*) \ + [ "${idf_reported}" = "ESP-IDF ${IDF_BASE_TAG}" ] \ + || { echo "expected 'ESP-IDF ${IDF_BASE_TAG}', got '${idf_reported}'" >&2; \ + exit 1; } ;; \ + *) echo "IDF_BASE_TAG '${IDF_BASE_TAG}' is not an exact version - label not asserted" ;; \ + esac; } && \ echo "ESP-IDF installation verified successfully" CMD ["/bin/bash"] diff --git a/images/esp-idf/README.md b/images/esp-idf/README.md index 8c901f6..20e813d 100644 --- a/images/esp-idf/README.md +++ b/images/esp-idf/README.md @@ -59,6 +59,43 @@ tag itself is rewritten on every rebuild. - **CI/CD**: Use version tags (`idf-v`) for reproducibility - **Debugging**: Use commit tags (`sha-`) to reproduce exact build +### Reading the ESP-IDF version without pulling the image + +A consumer that pins by digest — the only pin nothing can move underneath it — +does not keep the tag, so the tag is the one place it cannot read the version +from. The image carries it as a label instead, readable straight from the +registry: + +```bash +docker buildx imagetools inspect \ + ghcr.io/jethome-iot/jethome-dev-esp-idf@sha256: \ + --format '{{ index (index .Image "linux/amd64").Config.Labels "dev.jethome.idf.version" }}' +``` + +The platform key is not decoration: these tags are multi-arch indexes, so +`imagetools` hands the template a map of one image per platform rather than a +single image, and `.Image.Config` on an index is a template error rather than an +answer. Both platforms carry the same label; pick either. + +Locally, on an image **already pulled** (this one reads the local daemon, not the +registry): + +```bash +docker image inspect \ + --format '{{index .Config.Labels "dev.jethome.idf.version"}}' +``` + +The label is not a claim the image makes about itself: its build fails unless +`idf.py --version` inside the image reports exactly that version, so the cheap +answer above and the expensive one (`docker run … idf.py --version`) cannot +disagree. + +That holds for every published image, since CI builds each one from an exact +version. A local build given an alias — `IDF_BASE_TAG=latest`, or a minor-line +tag like `v5.3` — cannot be checked that way (an alias never equals the release +it resolves to), so the build says the assertion was skipped and the label +repeats the alias. Publish exact versions; aliases are a local convenience. + ### Pull Image ```bash diff --git a/images/esp-matter/Dockerfile b/images/esp-matter/Dockerfile index fd18b15..cde9bc4 100644 --- a/images/esp-matter/Dockerfile +++ b/images/esp-matter/Dockerfile @@ -17,12 +17,23 @@ ARG ESP_MATTER_VERSION=v1.6 # rebuild reproduce the same tree instead of picking up whatever landed upstream. # Update with ./scripts/update-matter-ref.sh. ARG ESP_MATTER_REF=2c125619610b9a16bb8e92f5ebbf55111638615e +# The ESP-IDF version underneath, for the label and - the actual reason it is an +# arg - for the assertion. The label alone would need nothing: Docker inherits a +# base image's LABELs, so building on this repository's esp-idf already carries +# `dev.jethome.idf.version` down. What inheritance cannot do is *check* it, and it +# carries nothing at all when BASE_IMAGE points at some other image. Since CI +# passes that base as a digest, and a digest names no version, the value comes +# from images/versions.json instead - taken there from the same `base_tag` the +# digest was resolved from, with the checker refusing a value the variant's own +# tag does not name. +ARG IDF_VERSION=v5.5.5 FROM ${BASE_IMAGE} # Re-declare build arguments after FROM ARG ESP_MATTER_VERSION ARG ESP_MATTER_REF +ARG IDF_VERSION # Use bash for all RUN commands: the layers below call the bash builtin `source`, # which /bin/sh does not have. (This once said "needed for QEMU emulation @@ -63,7 +74,8 @@ RUN git init esp-matter && \ # Answers "what is actually in here" without starting the container. LABEL org.opencontainers.image.revision="${ESP_MATTER_REF}" \ - org.opencontainers.image.version="${ESP_MATTER_VERSION}" + org.opencontainers.image.version="${ESP_MATTER_VERSION}" \ + dev.jethome.idf.version="${IDF_VERSION}" # Install ESP-Matter (without host tools to reduce image size) # Source ESP-IDF environment first to activate Python venv with pip @@ -104,7 +116,14 @@ RUN source ${IDF_PATH}/export.sh && \ { [ -z "${base_esptool}" ] && echo "no base snapshot - esptool not asserted" \ || grep -qx "${base_esptool}" /opt/esp/python-packages.txt; } && \ grep -E '^(pytest|esptool)' /opt/esp/python-packages.txt && \ - idf.py --version && \ + idf_reported="$(idf.py --version)" && \ + echo "${idf_reported}" && \ +# The inherited SDK, held to the version this image labels. IDF_VERSION arrives as +# a build arg because BASE_IMAGE is a digest and says nothing about the version - +# so without this the label would be the one thing here nobody checks. + { [ "${idf_reported}" = "ESP-IDF ${IDF_VERSION}" ] \ + || { echo "expected 'ESP-IDF ${IDF_VERSION}', got '${idf_reported}'" >&2; \ + exit 1; }; } && \ echo "ESP-Matter ${ESP_MATTER_VERSION} installation verified successfully" # Use custom entrypoint to activate environments (like entrypoint.sh in esp-idf image) diff --git a/images/esp-matter/README.md b/images/esp-matter/README.md index 97577ed..baad5d4 100644 --- a/images/esp-matter/README.md +++ b/images/esp-matter/README.md @@ -78,6 +78,25 @@ Two things worth knowing before pinning: docker image inspect --format '{{index .Config.Labels "org.opencontainers.image.revision"}}' ``` + The ESP-IDF underneath is readable the same way, on an image already pulled: + + ```bash + docker image inspect --format '{{index .Config.Labels "dev.jethome.idf.version"}}' + ``` + + And from the registry, with nothing pulled at all — the platform key is + required, because these tags are multi-arch indexes and `imagetools` then hands + the template one image per platform rather than a single one: + + ```bash + docker buildx imagetools inspect ghcr.io/jethome-iot/jethome-dev-esp-matter@sha256: \ + --format '{{ index (index .Image "linux/amd64").Config.Labels "dev.jethome.idf.version" }}' + ``` + + That one is checked rather than declared: the build fails unless `idf.py + --version` in the finished image reports exactly the labelled version, so it + cannot drift from the base the image was actually built on. + ### Pull Image ```bash @@ -277,6 +296,7 @@ docker build -t jethome-dev-esp-matter:local . ```bash docker build \ --build-arg BASE_IMAGE=ghcr.io/jethome-iot/jethome-dev-esp-idf:idf-v \ + --build-arg IDF_VERSION=v \ --build-arg ESP_MATTER_VERSION=v \ -t jethome-dev-esp-matter:local . ``` @@ -289,6 +309,14 @@ Available build arguments: digest fits: `…/jethome-dev-esp-idf@sha256:…`, which is what CI passes to pin the exact base its own run produced. It also lets a fork build against its own base instead of this repository's. +- `IDF_VERSION` - the ESP-IDF version that base carries, for the + `dev.jethome.idf.version` label. **Pass it whenever `BASE_IMAGE` is not the + default**: it does not follow the base, so a build on `idf-v` would + otherwise assert the default version and fail — after the SDK install, which is + the expensive half of the build. It is passed rather than derived because + `BASE_IMAGE` is a digest and a digest names no version; the build asserts the + value against what `idf.py --version` reports, so passing the wrong one fails + the build instead of publishing a label that lies. - `ESP_MATTER_VERSION` - the Matter specification this image carries, used for the image label and the verification line (default: see Dockerfile) @@ -298,6 +326,7 @@ To build on an ESP-IDF image you built yourself: ./scripts/build.sh esp-idf docker build \ --build-arg BASE_IMAGE=jethome-dev-esp-idf:local \ + --build-arg IDF_VERSION=v \ -t jethome-dev-esp-matter:local images/esp-matter ``` diff --git a/images/versions.json b/images/versions.json index d2d96e5..7291a68 100644 --- a/images/versions.json +++ b/images/versions.json @@ -81,7 +81,8 @@ "base_tag": "idf-v5.5.5", "upstream_branch": "release/v1.6", "args": { - "ESP_MATTER_VERSION": "v1.6" + "ESP_MATTER_VERSION": "v1.6", + "IDF_VERSION": "v5.5.5" }, "pin": { "ESP_MATTER_REF": "2c125619610b9a16bb8e92f5ebbf55111638615e" @@ -93,7 +94,8 @@ "base_tag": "idf-v5.5.5", "upstream_branch": "release/v1.5", "args": { - "ESP_MATTER_VERSION": "v1.5.1" + "ESP_MATTER_VERSION": "v1.5.1", + "IDF_VERSION": "v5.5.5" }, "pin": { "ESP_MATTER_REF": "44dfffd046f04552a87fa5ff7cf9695be486f177" @@ -105,7 +107,8 @@ "base_tag": "idf-v5.4.1", "upstream_branch": "release/v1.4.2", "args": { - "ESP_MATTER_VERSION": "v1.4.2" + "ESP_MATTER_VERSION": "v1.4.2", + "IDF_VERSION": "v5.4.1" }, "pin": { "ESP_MATTER_REF": "2e4e0050c0bf6167f6bfc3351086c4e1126a6893" From ac2a68448ad834209599f4975e265cae1b588df1 Mon Sep 17 00:00:00 2001 From: Pavel Sokolov Date: Mon, 24 Aug 2026 06:18:21 +0300 Subject: [PATCH 2/2] Anchor the exact-version test, and keep the two versions apart in the docs Copilot's three points on #28. The `case` glob `v[0-9]*.[0-9]*.[0-9]*` was not the test its own comment described: `*` swallows anything, so `v5.5.5-rc1` and `v5.5.5-dirty` were taken for exact versions and asserted against - the opposite of what that branch exists for. An anchored `[[ =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]` says it exactly; bash is already this image's SHELL. Verified against v5.5.5, v5.5.5-rc1, v5.5.5-dirty, v5.x.y, v5.3, latest and v5.10.12. The esp-matter custom-build recipe used the same `v` placeholder for IDF_VERSION and ESP_MATTER_VERSION, which invites copying one value into both; they are now `v` and `v`, and the base tag matches. CLAUDE.md's local recipe said `IDF_VERSION=` with no leading `v`, which would have produced a confusing assertion failure - it now shows the form the SDK reports. --- CLAUDE.md | 3 ++- images/esp-idf/Dockerfile | 14 +++++++------- images/esp-matter/README.md | 12 +++++++----- 3 files changed, 16 insertions(+), 13 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index c3e5f07..4bbe9e7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -366,7 +366,8 @@ Four places, none of them checked automatically: `esp-matter` this way pulls its base from GHCR — to build it on an esp-idf you just built, pass the base explicitly: `docker build --build-arg BASE_IMAGE=jethome-dev-esp-idf:local --build-arg - IDF_VERSION= images/esp-matter` — the second arg because the version is + IDF_VERSION=v images/esp-matter` (the leading `v`, as the SDK + reports it — the assertion compares the two literally) — the second arg because the version is asserted against the base, and its default belongs to the *published* base rather than to whatever you just built. Note that a version-bump branch cannot use the Dockerfile default at all: it diff --git a/images/esp-idf/Dockerfile b/images/esp-idf/Dockerfile index f659f89..d353b02 100644 --- a/images/esp-idf/Dockerfile +++ b/images/esp-idf/Dockerfile @@ -104,13 +104,13 @@ RUN export IDF_PATH_FORCE=1 && \ jq --version && \ idf_reported="$(idf.py --version)" && \ echo "${idf_reported}" && \ - { case "${IDF_BASE_TAG}" in \ - v[0-9]*.[0-9]*.[0-9]*) \ - [ "${idf_reported}" = "ESP-IDF ${IDF_BASE_TAG}" ] \ - || { echo "expected 'ESP-IDF ${IDF_BASE_TAG}', got '${idf_reported}'" >&2; \ - exit 1; } ;; \ - *) echo "IDF_BASE_TAG '${IDF_BASE_TAG}' is not an exact version - label not asserted" ;; \ - esac; } && \ + { if [[ "${IDF_BASE_TAG}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then \ + [ "${idf_reported}" = "ESP-IDF ${IDF_BASE_TAG}" ] \ + || { echo "expected 'ESP-IDF ${IDF_BASE_TAG}', got '${idf_reported}'" >&2; \ + exit 1; }; \ + else \ + echo "IDF_BASE_TAG '${IDF_BASE_TAG}' is not an exact version - label not asserted"; \ + fi; } && \ echo "ESP-IDF installation verified successfully" CMD ["/bin/bash"] diff --git a/images/esp-matter/README.md b/images/esp-matter/README.md index baad5d4..0854228 100644 --- a/images/esp-matter/README.md +++ b/images/esp-matter/README.md @@ -295,9 +295,9 @@ docker build -t jethome-dev-esp-matter:local . ```bash docker build \ - --build-arg BASE_IMAGE=ghcr.io/jethome-iot/jethome-dev-esp-idf:idf-v \ - --build-arg IDF_VERSION=v \ - --build-arg ESP_MATTER_VERSION=v \ + --build-arg BASE_IMAGE=ghcr.io/jethome-iot/jethome-dev-esp-idf:idf-v \ + --build-arg IDF_VERSION=v \ + --build-arg ESP_MATTER_VERSION=v \ -t jethome-dev-esp-matter:local . ``` @@ -310,7 +310,9 @@ Available build arguments: exact base its own run produced. It also lets a fork build against its own base instead of this repository's. - `IDF_VERSION` - the ESP-IDF version that base carries, for the - `dev.jethome.idf.version` label. **Pass it whenever `BASE_IMAGE` is not the + `dev.jethome.idf.version` label. Same spelling the SDK itself reports, leading + `v` included (`v5.5.5`), and distinct from `ESP_MATTER_VERSION` — the two are + different versions and the placeholders above keep them apart. **Pass it whenever `BASE_IMAGE` is not the default**: it does not follow the base, so a build on `idf-v` would otherwise assert the default version and fail — after the SDK install, which is the expensive half of the build. It is passed rather than derived because @@ -326,7 +328,7 @@ To build on an ESP-IDF image you built yourself: ./scripts/build.sh esp-idf docker build \ --build-arg BASE_IMAGE=jethome-dev-esp-idf:local \ - --build-arg IDF_VERSION=v \ + --build-arg IDF_VERSION=v \ -t jethome-dev-esp-matter:local images/esp-matter ```