From 731bbe7fa5f5f7f904529812e49448f6b26b25c0 Mon Sep 17 00:00:00 2001 From: Wu Sheng Date: Sun, 30 Aug 2026 22:31:58 +0800 Subject: [PATCH] Release from a fresh clone, ask for the versions, and rotate the changelog Three changes to the release tooling, plus the 5.0.0 changelog rotation they imply. release.sh now clones apache/skywalking-helm into tools/releasing/ and does everything there. `make release-src` archives the working TREE, not HEAD, so releasing from a working copy ships whatever untracked files are sitting in it. That was previously handled by refusing to run against a dirty tree, which is correct but puts the problem on the operator -- and the obvious workaround is a trap: `tar` does not read .gitignore, so ignoring a directory makes it invisible to the dirty-tree check while leaving it in the tarball. Verified by doing it. Hence three layers: the Makefile excludes the clone, .gitignore keeps it out of commits, and verify_artifacts now inspects the finished archive for the clone, a stray .tgz, charts/ or Chart.lock. Both scripts ask for the versions. release.sh defaults from the checkout's Chart.yaml, derives the next minor, and offers both for confirmation; release-passed.sh cannot default from Chart.yaml, because by the time a vote passes master has moved on, so it reads what is actually staged in svn instead. Its arguments are now order-independent and the version is optional. Both require plain MAJOR.MINOR.PATCH: the values reach a git tag, an svn path and a branch name. release.sh opens the next-version PR itself, as swck's does. It rotates docs/changes/changes.md to changes-$VERSION.md, renders a fresh one from the new docs/changes/changes.tpl, bumps Chart.yaml, and inserts the released version into the docs menu after "Current Version" so released versions stay newest-first. The chart bump uses sed rather than yq: yq rewrites the whole document, which on this file reindents every list and turns a one-line change into a 53-line diff. Measured, not assumed. The 5.0.0 rotation is applied here, since 5.0.0 is already tagged and staged: changes.md becomes changes-5.0.0.md, master moves to 5.1.0, and the Changelog menu entry becomes a catalog so released versions have somewhere to go. This is a deliberate change of practice -- master previously kept the released version until the next release prepared itself. release-passed.sh no longer tells you to bump Chart.yaml by hand; release.sh opens that PR now. Verified end to end against a local fixture -- a bare git origin plus an svnadmin repo standing in for dist/dev -- rather than by reading: exit 0 through all nine stages, six artifacts signed and verified, chart renders 33 resources, source tarball clean, svn staging correct, and the next-version commit carrying exactly a one-line Chart.yaml bump, the rotated changelog and two menu lines. --- Makefile | 1 + chart/skywalking/Chart.yaml | 2 +- docs/README.md | 2 +- docs/changes/changes-5.0.0.md | 70 +++++++ docs/changes/changes.md | 65 +----- docs/changes/changes.tpl | 11 ++ docs/contributing/release.md | 79 ++++++-- docs/menu.yml | 6 +- tools/releasing/.gitignore | 19 ++ tools/releasing/release-passed.sh | 63 +++++- tools/releasing/release.sh | 319 ++++++++++++++++++++---------- 11 files changed, 446 insertions(+), 191 deletions(-) create mode 100644 docs/changes/changes-5.0.0.md create mode 100644 docs/changes/changes.tpl create mode 100644 tools/releasing/.gitignore diff --git a/Makefile b/Makefile index f0f862b..699cd89 100644 --- a/Makefile +++ b/Makefile @@ -60,6 +60,7 @@ release-src: clean --exclude .gitignore \ --exclude .DS_Store \ --exclude .github \ + --exclude tools/releasing/skywalking-helm \ . && \ mv $(TMPDIR)/$(RELEASE_SRC).tgz . diff --git a/chart/skywalking/Chart.yaml b/chart/skywalking/Chart.yaml index e60513d..4f02da7 100644 --- a/chart/skywalking/Chart.yaml +++ b/chart/skywalking/Chart.yaml @@ -16,7 +16,7 @@ apiVersion: v2 name: skywalking-helm home: https://skywalking.apache.org -version: 5.0.0 +version: 5.1.0 description: Helm Chart for Apache SkyWalking icon: https://raw.githubusercontent.com/apache/skywalking-kubernetes/master/logo/sw-logo-for-chart.jpg sources: diff --git a/docs/README.md b/docs/README.md index b37e416..a8925ce 100644 --- a/docs/README.md +++ b/docs/README.md @@ -55,7 +55,7 @@ the pod reports Ready and nobody can sign in until you configure users. See - **Every value** — [Chart Values](reference/skywalking-chart-values.md) - **Contributing** — [Run the E2E Tests](contributing/e2e-tests.md), [Package and Publish a Release](contributing/release.md) -- **What changed in 5.0.0** — [Changelog](changes/changes.md) +- **What changed** — [Changelog](changes/changes.md), and the [5.0.0 release notes](changes/changes-5.0.0.md) ## Related documentation diff --git a/docs/changes/changes-5.0.0.md b/docs/changes/changes-5.0.0.md new file mode 100644 index 0000000..b4a5f82 --- /dev/null +++ b/docs/changes/changes-5.0.0.md @@ -0,0 +1,70 @@ +# Changelog + +## 5.0.0 + +Targets SkyWalking OAP 11.0.0, Horizon UI 1.0.0 and BanyanDB 0.11.0. See +[Upgrade](../upgrade/upgrading.md) for the migration steps. + +### Breaking changes + +- **OAP 11 requires BanyanDB 0.11.x.** OAP pins the BanyanDB server API versions it accepts + (`SW_STORAGE_BANYANDB_COMPATIBLE_SERVER_API_VERSIONS`, `0.11` in 11.0.0) and checks them with + string equality, so pairing OAP 11 with BanyanDB 0.10.x makes OAP refuse to start. The three + versions move together — see [Version Compatibility](../evaluate/version-compatibility.md). +- **`oap.ports.admin` is required.** OAP 11 enables every admin feature module by default and + serves `/status/*` and `/debugging/*` on the admin port only; they are no longer mirrored on + `oap.ports.rest`. Horizon UI reads status, inspect, DSL debugging and the dashboard template + store from it. +- **The legacy booster UI is no longer supported.** OAP 11 deleted `apm-webapp` and the + `skywalking-booster-ui` submodule along with the `docker.ui` build target, so + `apache/skywalking-ui` publishes no `11.x` tag — only `horizon-*` tags. Replace + `ui.image.tag=` with `ui.image.tag=horizon-1.0.0`. +- **`oap.config.ui-initialized-templates` does nothing.** OAP 11 removed the on-disk dashboard + seed files and `UITemplateInitializer`, along with the sidebar menu storage, the + `UIConfigurationManagement` GraphQL mutations and `SW_ENABLE_UPDATE_UI_TEMPLATE`. Horizon UI + ships its own dashboard library and manages templates over the admin REST port. +- **Horizon is configured by environment variable, and no ConfigMap is mounted by default.** The + image ships a complete env-tokenized `/app/horizon.yaml`; the chart sets only what it computes + (`HORIZON_SERVER_PORT`, `HORIZON_OAP_QUERY_URL`, and the admin, Zipkin and public URLs when + configured) and leaves the rest to `ui.extraEnv` / `ui.envFromSecret`. `ui.config` is now opt-in: + setting it renders a ConfigMap and mounts it *over* the image's file, so fields you do not write + fall back to Horizon's defaults. If you carried a `ui.config` block from the pre-release `main` + values, move it to environment variables — see [Configure Horizon](../ui/configure.md). +- **The SWCK charts are removed.** `chart/operator` and `chart/adapter` packaged + [apache/skywalking-swck](https://github.com/apache/skywalking-swck) — its image, its CRDs and its + version — and had no relationship to `chart/skywalking`. They were never released to Docker Hub, + so no released artifact disappears, but installs from source or from the `ghcr.io` snapshot + channel will break. They belong with the operator, where the CRDs are generated alongside the + code that consumes them. +- **The UI no longer proxies `/graphql`.** Callers that talked to the UI's GraphQL endpoint + (for example `swctl --base-url=http:///graphql`) must target the OAP service directly on + `oap.ports.rest`. + +### Features + +- `ui.extraVolumes` / `ui.extraVolumeMounts`, for the two Horizon settings that take a filesystem + path: `auth.tokensFile` and `sourceMaps.bootMountDir`. +- `server.publicUrl` is derived from the first `ui.ingress.hosts` entry when an ingress is enabled, + so single sign-on callbacks and the OAuth issuer are built from the address operators actually + reach — see [UI Service and Ingress](../expose/ui-service-and-ingress.md). +- `server.port` is derived from `ui.service.internalPort`, so the BFF binds the port the container + exposes. +- `oap.extraEnv` (a list, so entries can carry `valueFrom`) and `oap.envFromSecret`, applied to the + OAP Deployment and the init Job. Note Kubernetes gives an explicit `env` entry precedence over + `envFrom`, and the chart sets `SW_ES_PASSWORD` / `SW_DATA_SOURCE_PASSWORD` itself — so sourcing + those from a Secret needs `oap.extraEnv`. +- Horizon's config hot-reload works again. The chart previously mounted `horizon.yaml` with + `subPath`, which Kubernetes never updates in place, so the file watcher could not fire. +- `tools/releasing/release.sh` and `release-passed.sh`, plus + [the release guide](../contributing/release.md) — the Apache process was previously unwritten. +- The E2E suite is rebuilt around Horizon: every assertion runs through the UI's API rather than + OAP's GraphQL, so it exercises the path the chart is responsible for wiring. +- Documentation moved into `docs/` and is published at + [skywalking.apache.org/docs/skywalking-helm](https://skywalking.apache.org/docs/skywalking-helm/next/readme/). + +### Corrections + +- Horizon UI does **not** refuse to start without configured users. It boots, serves the login + page, and answers `/api/auth/health` with 200 — which is this chart's readiness probe — so the + pod reports Ready and nobody can sign in. Earlier documentation claimed a `CrashLoopBackOff`. + See [Set Up Logins](../ui/logins.md). diff --git a/docs/changes/changes.md b/docs/changes/changes.md index b4a5f82..c469c68 100644 --- a/docs/changes/changes.md +++ b/docs/changes/changes.md @@ -1,70 +1,11 @@ # Changelog -## 5.0.0 - -Targets SkyWalking OAP 11.0.0, Horizon UI 1.0.0 and BanyanDB 0.11.0. See -[Upgrade](../upgrade/upgrading.md) for the migration steps. +## 5.1.0 ### Breaking changes -- **OAP 11 requires BanyanDB 0.11.x.** OAP pins the BanyanDB server API versions it accepts - (`SW_STORAGE_BANYANDB_COMPATIBLE_SERVER_API_VERSIONS`, `0.11` in 11.0.0) and checks them with - string equality, so pairing OAP 11 with BanyanDB 0.10.x makes OAP refuse to start. The three - versions move together — see [Version Compatibility](../evaluate/version-compatibility.md). -- **`oap.ports.admin` is required.** OAP 11 enables every admin feature module by default and - serves `/status/*` and `/debugging/*` on the admin port only; they are no longer mirrored on - `oap.ports.rest`. Horizon UI reads status, inspect, DSL debugging and the dashboard template - store from it. -- **The legacy booster UI is no longer supported.** OAP 11 deleted `apm-webapp` and the - `skywalking-booster-ui` submodule along with the `docker.ui` build target, so - `apache/skywalking-ui` publishes no `11.x` tag — only `horizon-*` tags. Replace - `ui.image.tag=` with `ui.image.tag=horizon-1.0.0`. -- **`oap.config.ui-initialized-templates` does nothing.** OAP 11 removed the on-disk dashboard - seed files and `UITemplateInitializer`, along with the sidebar menu storage, the - `UIConfigurationManagement` GraphQL mutations and `SW_ENABLE_UPDATE_UI_TEMPLATE`. Horizon UI - ships its own dashboard library and manages templates over the admin REST port. -- **Horizon is configured by environment variable, and no ConfigMap is mounted by default.** The - image ships a complete env-tokenized `/app/horizon.yaml`; the chart sets only what it computes - (`HORIZON_SERVER_PORT`, `HORIZON_OAP_QUERY_URL`, and the admin, Zipkin and public URLs when - configured) and leaves the rest to `ui.extraEnv` / `ui.envFromSecret`. `ui.config` is now opt-in: - setting it renders a ConfigMap and mounts it *over* the image's file, so fields you do not write - fall back to Horizon's defaults. If you carried a `ui.config` block from the pre-release `main` - values, move it to environment variables — see [Configure Horizon](../ui/configure.md). -- **The SWCK charts are removed.** `chart/operator` and `chart/adapter` packaged - [apache/skywalking-swck](https://github.com/apache/skywalking-swck) — its image, its CRDs and its - version — and had no relationship to `chart/skywalking`. They were never released to Docker Hub, - so no released artifact disappears, but installs from source or from the `ghcr.io` snapshot - channel will break. They belong with the operator, where the CRDs are generated alongside the - code that consumes them. -- **The UI no longer proxies `/graphql`.** Callers that talked to the UI's GraphQL endpoint - (for example `swctl --base-url=http:///graphql`) must target the OAP service directly on - `oap.ports.rest`. - ### Features -- `ui.extraVolumes` / `ui.extraVolumeMounts`, for the two Horizon settings that take a filesystem - path: `auth.tokensFile` and `sourceMaps.bootMountDir`. -- `server.publicUrl` is derived from the first `ui.ingress.hosts` entry when an ingress is enabled, - so single sign-on callbacks and the OAuth issuer are built from the address operators actually - reach — see [UI Service and Ingress](../expose/ui-service-and-ingress.md). -- `server.port` is derived from `ui.service.internalPort`, so the BFF binds the port the container - exposes. -- `oap.extraEnv` (a list, so entries can carry `valueFrom`) and `oap.envFromSecret`, applied to the - OAP Deployment and the init Job. Note Kubernetes gives an explicit `env` entry precedence over - `envFrom`, and the chart sets `SW_ES_PASSWORD` / `SW_DATA_SOURCE_PASSWORD` itself — so sourcing - those from a Secret needs `oap.extraEnv`. -- Horizon's config hot-reload works again. The chart previously mounted `horizon.yaml` with - `subPath`, which Kubernetes never updates in place, so the file watcher could not fire. -- `tools/releasing/release.sh` and `release-passed.sh`, plus - [the release guide](../contributing/release.md) — the Apache process was previously unwritten. -- The E2E suite is rebuilt around Horizon: every assertion runs through the UI's API rather than - OAP's GraphQL, so it exercises the path the chart is responsible for wiring. -- Documentation moved into `docs/` and is published at - [skywalking.apache.org/docs/skywalking-helm](https://skywalking.apache.org/docs/skywalking-helm/next/readme/). - -### Corrections +### Bugs -- Horizon UI does **not** refuse to start without configured users. It boots, serves the login - page, and answers `/api/auth/health` with 200 — which is this chart's readiness probe — so the - pod reports Ready and nobody can sign in. Earlier documentation claimed a `CrashLoopBackOff`. - See [Set Up Logins](../ui/logins.md). +### Documentation diff --git a/docs/changes/changes.tpl b/docs/changes/changes.tpl new file mode 100644 index 0000000..8e937cd --- /dev/null +++ b/docs/changes/changes.tpl @@ -0,0 +1,11 @@ +# Changelog + +## NEXT_RELEASE_VERSION + +### Breaking changes + +### Features + +### Bugs + +### Documentation diff --git a/docs/contributing/release.md b/docs/contributing/release.md index acdc2a9..c14e79d 100644 --- a/docs/contributing/release.md +++ b/docs/contributing/release.md @@ -137,33 +137,61 @@ gpg --list-secret-keys --keyid-format=long ## 3. Build, verify, tag, upload and call the vote — `release.sh` -**From a pristine clone of the release commit.** Linux and macOS both work — the build was -verified end to end on each. A pristine clone matters for a different reason: preflight refuses to -run against a dirty tree, and `release-src` archives the working *tree*, so any untracked file -sitting in the checkout would otherwise be shipped inside the source release. +**Run it from your own checkout.** Linux and macOS both work — the build was verified end to end +on each. ```shell -git clone git@github.com:apache/skywalking-helm && cd skywalking-helm - -bash tools/releasing/release.sh --dry-run # everything except the tag push and the svn commit +bash tools/releasing/release.sh --dry-run # everything except the four writes bash tools/releasing/release.sh ``` -`--dry-run` still builds, signs and verifies, and still runs the svn checkout — it prints -`svn status` instead of committing, and creates no tag. Use it to find out whether the build is good -before anything leaves your machine. +You do not need a clean tree, because the script does not build from your tree. It clones +`apache/skywalking-helm` fresh into `tools/releasing/skywalking-helm/` and does everything there. +That is not tidiness: `make release-src` archives the working *tree*, not `HEAD`, so releasing from +a working copy ships whatever untracked files happen to be sitting in it — editor state, an agent +directory, a half-finished values file. Cloning removes the question. Your checkout is only read, +for the default version. + +Note that ignoring such a directory does **not** protect you — `tar` does not read `.gitignore`, so +a gitignored directory is silently archived. That is why the clone is excluded in the Makefile *and* +`verify_artifacts` inspects the finished tarball. + +It asks for both versions before doing anything, defaulting from your checkout's `Chart.yaml`: + +``` +=== Versions === + release version: 5.1.0 (from your checkout's chart/skywalking/Chart.yaml) + next dev version: 5.2.0 + + Are these correct? [y/N] +``` + +Answer anything but `y` to type them in. Both must be plain `MAJOR.MINOR.PATCH` — they end up in a +git tag, an svn path and a branch name, and refusing anything else keeps shell metacharacters out of +all three. + +`--dry-run` still clones, builds, signs and verifies, and still runs the svn checkout and the +next-version commit. It skips exactly four things: the tag push, the svn commit, the branch push and +the PR. ### The order matters | Stage | What it does | | --- | --- | +| `resolve_versions` | asks for the release and next-dev versions, defaulting from your `Chart.yaml` | | `preflight` | refuses to start (see below) | -| `build` | `make clean` then `make release` — six files in the repository root | -| `verify_artifacts` | signature, checksum and a real render of the packaged chart | +| `clone_repo` | fresh clone into `tools/releasing/skywalking-helm/`; everything below runs there | +| `build` | `make clean` then `make release` — six files in the clone | +| `verify_artifacts` | signature, checksum, a real render, and an inspection of the source tarball | | `tag` | `git tag -a v$VERSION` and `git push origin v$VERSION` | | `upload_to_svn` | sparse checkout of `dist/dev/skywalking`, then `svn add` + `svn commit` | +| `prepare_next_version` | rotates the changelog, bumps the chart, opens the next-version PR | | `vote_mail` | prints the mail, with the real checksums and commit hash filled in | +`clone_repo` also refuses to continue unless master's `Chart.yaml` already says the release version. +This project tags master as it stands, so if the version is not already there then master is not +ready — and setting it inside the clone would tag a commit that exists nowhere else. + The tag is pushed **after** the build and the artifact checks, deliberately. A tag pushed first survives a failed build, and preflight then refuses to re-run because `v$VERSION` exists — so the irreversible step comes last, and a failure leaves nothing on the remote to clean up. @@ -177,12 +205,10 @@ Preflight refuses to start when: read; - `gpg` holds no secret key. `make release` signs with `gpg --batch`, so without one the run would fail *after* building and packaging everything; +- `gh` is not authenticated. It opens the next-version PR at the very end, so an unauthenticated + `gh` would otherwise surface only after the vote candidate is already staged; - `dist/dev/skywalking` cannot be read — a network problem, or svn credentials that are not set up; -- the working tree is dirty — `release-src` archives the working *tree*, not `HEAD`; -- a `*.tgz`, `*.tgz.asc` or `*.tgz.sha512` is lying in the repository root. Those are gitignored, so - `git status` cannot see them, and a leftover from a previous release would be embedded in this - release's source tarball; -- `v$VERSION` already exists, or `dist/dev/skywalking/helm/$VERSION` already does. The second catches +- `v$VERSION` already exists on the remote, or `dist/dev/skywalking/helm/$VERSION` already does. The second catches a re-run after a partial upload, which would otherwise only surface at `svn commit` — after the build, the signing and the tag push. @@ -194,7 +220,23 @@ the remote; preflight is the only place where stopping is free. are present, that `gpg --batch --verify` passes and that `shasum -a 512 -c` passes. It then runs `helm template` over the packaged chart and requires at least one rendered `kind:` — a chart that lints but renders nothing is a valid chart. That render uses `oap.storageType=elasticsearch` with -`elasticsearch.enabled` left at its default `true`, so it exercises the ECK path. +`elasticsearch.enabled` left at its default `true`, so it exercises the ECK path. Finally it lists +the source tarball and fails if it contains the build clone, a `.tgz`, a `charts/` directory or a +`Chart.lock`. + +### The next-version PR + +`prepare_next_version` runs after the candidate is staged, on a `bump-to-$NEXT` branch of the clone: + +- `chart/skywalking/Chart.yaml` moves to the next dev version — with `sed`, not `yq`, because `yq` + rewrites the whole document and turns a one-line bump into a fifty-line reindent; +- `docs/changes/changes.md` becomes `docs/changes/changes-$VERSION.md`, and a fresh changelog is + rendered from `docs/changes/changes.tpl`; +- `docs/menu.yml` gains the released version, inserted directly after `Current Version` so the + in-progress changelog keeps the top of the menu and released versions stay newest-first. + +Merge it once the vote thread is open. It cannot affect the artifacts under vote — those were built +from the tag, before this branch existed. ### Send the vote mail @@ -358,6 +400,7 @@ as a no *without* aborting, so it walks the whole plan and does none of it. | Stage | Prompt | Declining | | --- | --- | --- | +| `resolve_version` | confirm the version to publish | — | | `preflight` | — | fails if `svn` / `gh` / `git` are missing, if `gh` is not authenticated, if `dist/release/skywalking` cannot be read, or if `dist/dev/skywalking/helm/$VERSION` does not exist | | `promote_artifacts` | `svn mv` from `dist/dev` to `dist/release` | aborts | | `remove_previous` | remove everything under `release/helm/` other than `$VERSION` | **skips and continues** — the one exception | diff --git a/docs/menu.yml b/docs/menu.yml index 365c278..448dbac 100644 --- a/docs/menu.yml +++ b/docs/menu.yml @@ -89,4 +89,8 @@ catalog: - name: "Package and Publish a Release" path: "/contributing/release" - name: "Changelog" - path: "/changes/changes" + catalog: + - name: "Current Version" + path: "/changes/changes" + - name: "5.0.0" + path: "/changes/changes-5.0.0" diff --git a/tools/releasing/.gitignore b/tools/releasing/.gitignore new file mode 100644 index 0000000..8567444 --- /dev/null +++ b/tools/releasing/.gitignore @@ -0,0 +1,19 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# release.sh clones the repository here and builds the release inside it. Throwaway, and it +# belongs in no commit. Note this does NOT keep it out of the source tarball -- tar does not read +# .gitignore -- which is why the Makefile excludes it and release.sh inspects the finished archive. +skywalking-helm/ diff --git a/tools/releasing/release-passed.sh b/tools/releasing/release-passed.sh index 0ec1c10..4c92c23 100755 --- a/tools/releasing/release-passed.sh +++ b/tools/releasing/release-passed.sh @@ -32,18 +32,29 @@ SVN_DEV_URL="https://dist.apache.org/repos/dist/dev/skywalking" SVN_RELEASE_URL="https://dist.apache.org/repos/dist/release/skywalking" PRODUCT_NAME="skywalking-helm" -VERSION="${1:-}" +# Order-independent, so `--dry-run 5.0.0` and `5.0.0 --dry-run` both work, and the version may be +# left out entirely -- it is then read from the staging area and confirmed. +VERSION="" DRY_RUN=false -case "${2:-}" in - "") ;; - --dry-run) DRY_RUN=true ;; - *) echo "ERROR: unknown argument '${2}' -- did you mean --dry-run?" >&2; exit 1 ;; -esac +for arg in "$@"; do + case "${arg}" in + --dry-run) DRY_RUN=true ;; + [0-9]*.[0-9]*.[0-9]*) VERSION="${arg}" ;; + *) echo "ERROR: unknown argument '${arg}' -- usage: $0 [] [--dry-run]" >&2; exit 1 ;; + esac +done log() { echo " $*"; } step() { echo; echo "=== $* ==="; } die() { echo "ERROR: $*" >&2; exit 1; } +ask() { + # `read` exits 1 at EOF, which under set -e would kill the run with no message. + local reply + read -r -p " $1" reply || die "nothing on stdin -- run this script from a terminal" + printf '%s' "${reply}" +} + # Declining ABORTS. These steps are ordered and dependent -- saying no to the # svn promotion and then continuing would create a GitHub release, and publish # the chart, for artifacts still sitting in the dev area. @@ -60,10 +71,43 @@ confirm() { # --------------------------------------------------------------------------- +resolve_version() { + step "Version" + + # Unlike release.sh this cannot default from Chart.yaml: by the time the vote passes, master + # has usually moved on to the next development version. The staging area is the authority on + # what is actually up for release. + if [[ -z "${VERSION}" ]]; then + local staged count + staged=$(svn ls "${SVN_DEV_URL}/helm" 2>/dev/null | sed 's#/$##' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' || true) + count=$(printf '%s' "${staged}" | grep -c . || true) + + if [[ "${count}" -eq 1 ]]; then + VERSION="${staged}" + log "staged for release: ${VERSION}" + elif [[ "${count}" -gt 1 ]]; then + log "more than one version is staged:" + printf '%s\n' "${staged}" | sed 's/^/ /' + else + log "nothing is staged under ${SVN_DEV_URL}/helm" + fi + else + log "version given on the command line: ${VERSION}" + fi + + local reply + if [[ -n "${VERSION}" ]]; then + reply=$(ask "Publish ${VERSION}? [y/N] ") + [[ "${reply}" == "y" || "${reply}" == "Y" ]] || VERSION="" + fi + [[ -n "${VERSION}" ]] || VERSION=$(ask "Enter the version to publish: ") + + [[ "${VERSION}" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]] || die "version '${VERSION}' is not MAJOR.MINOR.PATCH" + TAG="v${VERSION}" +} + preflight() { step "Preflight" - [[ -n "${VERSION}" ]] || die "usage: $0 [--dry-run] e.g. $0 5.0.0" - TAG="v${VERSION}" log "publishing ${VERSION}" local missing="" @@ -213,11 +257,12 @@ remaining() { log "2. send the ANNOUNCE mail above to dev@skywalking.apache.org and announce@apache.org" log " from your @apache.org address, with the vote permalink filled in" log "3. close the 5.0.0 milestone here and 'Helm - ${VERSION}' on apache/skywalking" - log "4. bump chart/skywalking/Chart.yaml to the next development version" + log "4. the next-version PR was opened by release.sh; merge it if you have not already" } # --------------------------------------------------------------------------- +resolve_version preflight promote_artifacts remove_previous diff --git a/tools/releasing/release.sh b/tools/releasing/release.sh index e79b07c..6ce8799 100755 --- a/tools/releasing/release.sh +++ b/tools/releasing/release.sh @@ -16,21 +16,30 @@ # Everything up to and including the call for vote. # -# bash tools/releasing/release.sh # build, verify, upload, print the vote mail -# bash tools/releasing/release.sh --dry-run # do everything except svn commit +# bash tools/releasing/release.sh # ask for the versions, then do it +# bash tools/releasing/release.sh --dry-run # everything except the four writes # -# The version comes from chart/skywalking/Chart.yaml -- the same place the Makefile reads it, so -# there is one source of truth and no way to sign 5.0.0 while uploading it as 5.0.1. +# The release is built from a FRESH CLONE this script makes for itself, under +# tools/releasing/, not from your checkout. `make release-src` archives the working tree rather +# than HEAD, so releasing from a working copy ships whatever untracked files happen to be sitting +# in it -- editor state, agent directories, a half-finished values file. Cloning removes the +# question entirely. Your own checkout is only read, for the default version. # -# After this: send the printed mail, wait 72h, then tools/releasing/release-passed.sh. +# The four irreversible writes, all skipped by --dry-run: +# the git tag, the svn commit, the next-version branch push, and the PR. +# +# After this: send the printed mail, merge the next-version PR, wait 72h, then +# tools/releasing/release-passed.sh. set -euo pipefail SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) PROJECT_DIR=$(cd "${SCRIPT_DIR}/../.." && pwd) -CHART_FILE="${PROJECT_DIR}/chart/skywalking/Chart.yaml" -SVN_DEV_URL="https://dist.apache.org/repos/dist/dev/skywalking" PRODUCT_NAME="skywalking-helm" +REPO_URL="https://github.com/apache/skywalking-helm.git" +CLONE_DIR="${SCRIPT_DIR}/${PRODUCT_NAME}" +SVN_DEV_URL="https://dist.apache.org/repos/dist/dev/skywalking" +CHART_FILE_REL="chart/skywalking/Chart.yaml" DRY_RUN=false case "${1:-}" in @@ -43,40 +52,81 @@ log() { echo " $*"; } step() { echo; echo "=== $* ==="; } die() { echo "ERROR: $*" >&2; exit 1; } -# upload_to_svn stages the signed artifacts in a temp directory. The cleanup is registered here, -# at script scope, over a script-scope variable: an EXIT trap runs after the function has already -# returned, so it cannot see a `local`, and under `set -u` the unbound name would make the trap -# fail and take the script's exit status with it. +ask() { + # `read` exits 1 at EOF. Without this the script dies with no message at all when it is piped + # or run from CI -- the exact silent failure this script is careful to avoid elsewhere. + local reply + read -r -p " $1" reply || die "nothing on stdin -- run this script from a terminal" + printf '%s' "${reply}" +} + +# The svn staging area lives in a temp directory, outside both your checkout and the clone, so it +# can never be swept up by `make release-src`. The cleanup is registered here, at script scope, +# over a script-scope variable: an EXIT trap runs after the function has already returned and so +# cannot see a `local`, and under `set -u` the unbound name would make the trap itself fail and +# take the script's exit status with it. WORKDIR="" cleanup() { [[ -n "${WORKDIR}" ]] && rm -rf "${WORKDIR}"; return 0; } trap cleanup EXIT # --------------------------------------------------------------------------- +resolve_versions() { + step "Versions" + + local current major minor + current=$(awk '/^version: /{print $2; exit}' "${PROJECT_DIR}/${CHART_FILE_REL}" 2>/dev/null || true) + + RELEASE_VERSION="${current}" + NEXT_RELEASE_VERSION="" + if [[ "${RELEASE_VERSION}" =~ ^([0-9]+)\.([0-9]+)\.[0-9]+$ ]]; then + major="${BASH_REMATCH[1]}" + minor="${BASH_REMATCH[2]}" + NEXT_RELEASE_VERSION="${major}.$((minor + 1)).0" + fi + + log "release version: ${RELEASE_VERSION:-} (from your checkout's ${CHART_FILE_REL})" + log "next dev version: ${NEXT_RELEASE_VERSION:-}" + echo + + local reply + reply=$(ask "Are these correct? [y/N] ") + if [[ "${reply}" != "y" && "${reply}" != "Y" ]]; then + RELEASE_VERSION=$(ask "Enter release version: ") + NEXT_RELEASE_VERSION=$(ask "Enter next dev version: ") + fi + + # Both get interpolated into a git tag, an svn path and a branch name. Refusing anything that + # is not plain semver keeps all three predictable, and keeps shell metacharacters out of them. + [[ "${RELEASE_VERSION}" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]] || die "release version '${RELEASE_VERSION}' is not MAJOR.MINOR.PATCH" + [[ "${NEXT_RELEASE_VERSION}" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]] || die "next version '${NEXT_RELEASE_VERSION}' is not MAJOR.MINOR.PATCH" + [[ "${RELEASE_VERSION}" != "${NEXT_RELEASE_VERSION}" ]] || die "the release and next versions are both ${RELEASE_VERSION}" + + TAG="v${RELEASE_VERSION}" + log "releasing ${RELEASE_VERSION}, then opening a PR to move master to ${NEXT_RELEASE_VERSION}" +} + preflight() { step "Preflight" - # No OS check. There used to be one: `make clean` was a backslash-continued rm - # whose later -rf tokens sat mid-argument-list, which GNU rm permutes and BSD rm - # does not, so on macOS it exited 2 and left chart/skywalking/charts/ behind. - # That recipe is now a single rm -rf over one operand list, which both accept, - # and the whole path -- clean, release-src, package, gpg sign, shasum, and the - # verify checks below -- has been run through to completion on macOS. + # Running this from inside the throwaway clone would nest another clone beneath it. + [[ "${PROJECT_DIR}" != "${CLONE_DIR}" ]] || die "you are inside the throwaway clone -- run this from your own checkout" - # Report every missing tool at once. Dying on the first means one failed run - # per package, and this check exists precisely to spend zero of them. + # Report every missing tool at once. Dying on the first means one failed run per package, and + # this check exists precisely to spend zero of them. local missing="" - for tool in helm gpg shasum svn git make tar awk; do + for tool in helm gpg shasum svn git make tar awk yq gh; do command -v "${tool}" >/dev/null || missing="${missing} ${tool}" done [[ -z "${missing}" ]] || die "not installed:${missing}" - # Present is not the same as usable, and each of these fails LATE otherwise: - # a missing signing key after the whole build, bad svn credentials after the - # tag is already pushed. - # 3.8, not 3: the chart is published only as an OCI artifact, and `helm push` to an - # oci:// registry landed in 3.8. Helm 4 is fine -- Chart.yaml is apiVersion v2, which - # both majors read -- so this bounds from below only, and does not cap the major. + # Present is not the same as usable, and each of these otherwise fails LATE: a missing signing + # key after the whole build, bad svn credentials after the tag is already pushed, an + # unauthenticated gh after the vote candidate is already staged. + # + # 3.8, not 3: the chart is published only as an OCI artifact, and `helm push` to an oci:// + # registry landed in 3.8. Helm 4 is fine -- Chart.yaml is apiVersion v2, which both majors + # read -- so this bounds from below only, and does not cap the major. local helm_ver helm_major helm_minor helm_ver=$(helm version --short 2>/dev/null | sed 's/^v//; s/[-+].*//') helm_major=${helm_ver%%.*} @@ -86,79 +136,67 @@ preflight() { die "helm 3.8 or newer is required, found ${helm_ver}" fi - gpg --list-secret-keys >/dev/null 2>&1 && [[ -n "$(gpg --list-secret-keys --with-colons 2>/dev/null | grep '^sec')" ]] \ - || die "gpg has no secret key -- 'make release' signs the artifacts and would fail after the build" + local seckeys + seckeys=$(gpg --list-secret-keys --with-colons 2>/dev/null | grep -c '^sec' || true) + [[ "${seckeys}" -gt 0 ]] || die "gpg has no secret key -- 'make release' signs the artifacts and would fail after the build" + + gh auth status >/dev/null 2>&1 || die "gh is not authenticated -- run 'gh auth login'. It opens the next-version PR at the end." svn ls "${SVN_DEV_URL}" >/dev/null 2>&1 \ || die "cannot read ${SVN_DEV_URL} -- check your network and your ASF svn credentials" - cd "${PROJECT_DIR}" - [[ -z "$(git status --porcelain)" ]] || die "working tree is dirty -- the source tarball archives the working tree, not HEAD" - - # release-src tars the working directory, and *.tgz is gitignored -- so a - # leftover chart package from a previous release is invisible to git status - # and would be embedded in this release's source archive. - local strays - strays=$(ls -1 ./*.tgz ./*.tgz.asc ./*.tgz.sha512 2>/dev/null || true) - [[ -z "${strays}" ]] || die "stray release artifacts in the working tree, run 'make clean' first: -${strays}" - - # awk, not `grep | awk`: grep exits 1 when it matches nothing, pipefail promotes that to the - # pipeline, and `set -e` then kills the script during the assignment -- silently, and before the - # check below can report anything. awk exits 0 either way, so the check is reachable. - VERSION=$(awk '/^version: /{print $2; exit}' "${CHART_FILE}") - [[ -n "${VERSION}" ]] || die "could not read a 'version:' line from ${CHART_FILE}" - TAG="v${VERSION}" - log "version ${VERSION} (from chart/skywalking/Chart.yaml)" - - # `if`, not `X && die`. An && list that fails on its left side returns non-zero, and when it is - # the last statement in a function that becomes the function's return value -- which `set -e` - # then treats as a failed call, killing the run with no message. `if` returns 0 when the - # condition is false, which is the normal path here. - if git rev-parse "${TAG}" >/dev/null 2>&1; then - die "tag ${TAG} already exists -- bump Chart.yaml or delete the tag" + # `if`, not `X && die`. An && list that fails on its left side returns non-zero, and as a + # function's last statement that becomes the function's return value, which `set -e` treats as + # a failed call and uses to kill the run -- with no message at all. + if git ls-remote --exit-code --tags "${REPO_URL}" "refs/tags/${TAG}" >/dev/null 2>&1; then + die "tag ${TAG} already exists on the remote -- bump the version, or delete the tag" fi - # A re-run after a partial upload would mkdir a local ${VERSION} over a path that already exists - # in svn, and only find out at commit time -- after the build, the signing and the tag push. # Safe to read a non-zero exit as "not there" only because the svn check above already # established that the repository is reachable and the credentials work. - if svn ls "${SVN_DEV_URL}/helm/${VERSION}" >/dev/null 2>&1; then - die "${SVN_DEV_URL}/helm/${VERSION} already exists -- delete it, or bump the version" + if svn ls "${SVN_DEV_URL}/helm/${RELEASE_VERSION}" >/dev/null 2>&1; then + die "${SVN_DEV_URL}/helm/${RELEASE_VERSION} already exists -- delete it, or bump the version" fi + log "all checks passed" +} + +clone_repo() { + step "Clone ${REPO_URL}" + + # Not --depth 1: the tag is created and pushed from here, and prepare_next_version branches + # from here. Both want real history. + rm -rf "${CLONE_DIR}" + git clone --quiet "${REPO_URL}" "${CLONE_DIR}" + cd "${CLONE_DIR}" + log "cloned $(git rev-parse --short HEAD) on $(git rev-parse --abbrev-ref HEAD)" + + # This project tags master as it stands, with the version bump landing in its own PR + # beforehand. If the clone does not already say RELEASE_VERSION then master is not ready, and + # setting it here would tag a commit that exists nowhere but this throwaway directory. + local cloned + cloned=$(awk '/^version: /{print $2; exit}' "${CHART_FILE_REL}") + [[ "${cloned}" == "${RELEASE_VERSION}" ]] \ + || die "master's ${CHART_FILE_REL} says ${cloned}, not ${RELEASE_VERSION} -- land a 'Ready to release ${RELEASE_VERSION}' PR first" + log "${CHART_FILE_REL} says ${cloned}, matching the release version" } build() { step "Build" - cd "${PROJECT_DIR}" + cd "${CLONE_DIR}" make clean make release } -tag() { - # Deliberately after the build and the artifact checks: a tag pushed before - # them survives a failure, and preflight then refuses to re-run because the - # tag exists. Fail before the irreversible step, not after it. - step "Tag ${TAG}" - cd "${PROJECT_DIR}" - if ${DRY_RUN}; then - log "dry run: not creating or pushing ${TAG}" - else - git tag -a "${TAG}" -m "Release Apache SkyWalking Helm ${VERSION}" - git push origin "${TAG}" - fi -} - verify_artifacts() { step "Verify the artifacts" - cd "${PROJECT_DIR}" + cd "${CLONE_DIR}" - # Six files: the source tarball and the packaged chart, each signed and - # checksummed. Both are voted artifacts for this project. + # Six files: the source tarball and the packaged chart, each signed and checksummed. Both are + # voted artifacts for this project. local expected=( - "${PRODUCT_NAME}-${VERSION}-src.tgz" - "${PRODUCT_NAME}-${VERSION}.tgz" + "${PRODUCT_NAME}-${RELEASE_VERSION}-src.tgz" + "${PRODUCT_NAME}-${RELEASE_VERSION}.tgz" ) for f in "${expected[@]}"; do @@ -173,55 +211,130 @@ verify_artifacts() { # A chart that lints but renders nothing is a valid chart. Render it. local rendered - rendered=$(helm template rel "${PRODUCT_NAME}-${VERSION}.tgz" \ + rendered=$(helm template rel "${PRODUCT_NAME}-${RELEASE_VERSION}.tgz" \ --set oap.image.tag=x --set ui.image.tag=x --set oap.storageType=elasticsearch 2>/dev/null | grep -c '^kind:' || true) [[ "${rendered}" -gt 0 ]] || die "the packaged chart renders no resources" log "packaged chart renders ${rendered} resources" + + # tar does not honour .gitignore, so nothing about ignoring the clone directory keeps it out of + # the archive. Check the archive itself. + local strays + strays=$(tar tzf "${PRODUCT_NAME}-${RELEASE_VERSION}-src.tgz" \ + | grep -E "tools/releasing/${PRODUCT_NAME}/|\.tgz\$|/charts/|Chart\.lock" || true) + [[ -z "${strays}" ]] || die "the source tarball contains things it should not: +${strays}" + log "source tarball carries no build output" +} + +tag() { + # Deliberately after the build and the artifact checks: a tag pushed before them survives a + # failure, and preflight then refuses to re-run because the tag exists. Fail before the + # irreversible step, not after it. + step "Tag ${TAG}" + cd "${CLONE_DIR}" + if ${DRY_RUN}; then + log "dry run: not creating or pushing ${TAG}" + else + git tag -a "${TAG}" -m "Release Apache SkyWalking Helm ${RELEASE_VERSION}" + git push origin "${TAG}" + log "pushed ${TAG} at $(git rev-parse --short "${TAG}")" + fi } upload_to_svn() { - step "Upload to ${SVN_DEV_URL}/helm/${VERSION}" - cd "${PROJECT_DIR}" + step "Upload to ${SVN_DEV_URL}/helm/${RELEASE_VERSION}" + cd "${CLONE_DIR}" WORKDIR=$(mktemp -d) - local workdir="${WORKDIR}" - # Sparse checkout: a full checkout of dist/dev/skywalking pulls every - # sub-project's staging area, which is gigabytes. - svn co --depth empty "${SVN_DEV_URL}" "${workdir}/skywalking" >/dev/null - svn up --depth empty "${workdir}/skywalking/helm" >/dev/null 2>&1 || true - mkdir -p "${workdir}/skywalking/helm/${VERSION}" - cp "${PRODUCT_NAME}-${VERSION}"*.tgz* "${workdir}/skywalking/helm/${VERSION}/" + # Sparse checkout: a full checkout of dist/dev/skywalking pulls every sub-project's staging + # area, which is gigabytes. + svn co --depth empty "${SVN_DEV_URL}" "${WORKDIR}/skywalking" >/dev/null + svn up --depth empty "${WORKDIR}/skywalking/helm" >/dev/null 2>&1 || true + mkdir -p "${WORKDIR}/skywalking/helm/${RELEASE_VERSION}" + cp "${PRODUCT_NAME}-${RELEASE_VERSION}"*.tgz* "${WORKDIR}/skywalking/helm/${RELEASE_VERSION}/" - cd "${workdir}/skywalking/helm" - svn add "${VERSION}" + cd "${WORKDIR}/skywalking/helm" + svn add "${RELEASE_VERSION}" >/dev/null if ${DRY_RUN}; then log "dry run: not committing. staged files:" svn status | sed 's/^/ /' else - svn commit -m "Draft Apache SkyWalking Helm release ${VERSION}" + svn commit -m "Draft Apache SkyWalking Helm release ${RELEASE_VERSION}" log "uploaded" fi } +prepare_next_version() { + step "Next iteration ${NEXT_RELEASE_VERSION}" + cd "${CLONE_DIR}" + + local branch="bump-to-${NEXT_RELEASE_VERSION}" + git checkout --quiet -b "${branch}" + + # sed, not `yq -i`: yq rewrites the whole document, and on this file that reindents every list + # and drops a blank line -- a 53-line diff for a one-line change. + sed -i.bak -E "s/^version: .*/version: ${NEXT_RELEASE_VERSION}/" "${CHART_FILE_REL}" + rm -f "${CHART_FILE_REL}.bak" + log "${CHART_FILE_REL} -> ${NEXT_RELEASE_VERSION}" + + # Rotate the changelog: what was being written becomes the released version's own page, and a + # fresh one starts from the template. + git mv docs/changes/changes.md "docs/changes/changes-${RELEASE_VERSION}.md" + sed "s/NEXT_RELEASE_VERSION/${NEXT_RELEASE_VERSION}/g" docs/changes/changes.tpl > docs/changes/changes.md + log "changes.md -> changes-${RELEASE_VERSION}.md, and a new changelog for ${NEXT_RELEASE_VERSION}" + + # Insert the released version directly after "Current Version", so the in-progress changelog + # keeps the top of the Changelog menu and the released ones stay in descending order. + yq -i "(.catalog[] | select(.name == \"Changelog\") | .catalog) |= [.[] | select(.name == \"Current Version\")] + [{\"name\": \"${RELEASE_VERSION}\", \"path\": \"/changes/changes-${RELEASE_VERSION}\"} | .name style=\"double\" | .path style=\"double\"] + [.[] | select(.name != \"Current Version\")]" docs/menu.yml + log "docs/menu.yml -> Changelog gains ${RELEASE_VERSION}" + + git add "${CHART_FILE_REL}" docs + git commit --quiet -m "Start the next iteration ${NEXT_RELEASE_VERSION}" + + if ${DRY_RUN}; then + log "dry run: not pushing ${branch}, not opening a PR. It would carry:" + git show --stat --oneline HEAD | sed 's/^/ /' + else + git push --quiet --set-upstream origin "${branch}" + gh pr create --repo apache/skywalking-helm --base master --head "${branch}" \ + --title "Start the next iteration ${NEXT_RELEASE_VERSION}" \ + --body "Opened by \`tools/releasing/release.sh\` while staging the ${RELEASE_VERSION} vote. + +- \`${CHART_FILE_REL}\` moves to ${NEXT_RELEASE_VERSION} +- \`docs/changes/changes.md\` becomes \`changes-${RELEASE_VERSION}.md\`, and a fresh changelog starts for ${NEXT_RELEASE_VERSION} +- \`docs/menu.yml\` gains the ${RELEASE_VERSION} changelog entry + +Merge once the ${RELEASE_VERSION} vote thread is open. It does not affect the artifacts under vote, which were built from \`${TAG}\`." + log "PR opened" + fi +} + vote_mail() { step "Vote mail -- copy from here" - cd "${PROJECT_DIR}" - - local checksums - checksums=$(for f in "${PRODUCT_NAME}-${VERSION}"*.tgz.sha512; do printf ' - '; cat "${f}"; done) + cd "${CLONE_DIR}" + + local checksums commit + checksums=$(for f in "${PRODUCT_NAME}-${RELEASE_VERSION}"*.tgz.sha512; do printf ' - '; cat "${f}"; done) + # HEAD is wrong here: prepare_next_version has already moved the clone onto the bump branch, so + # resolve the tag itself, falling back in a dry run to what would have been tagged. + # + # --verify --quiet, not `2>/dev/null`: a plain `git rev-parse` ECHOES an unresolvable ref back on + # STDOUT and exits 128, so `$(git rev-parse X 2>/dev/null || git rev-parse master)` captures the + # literal "v5.1.0^{commit}" AND the fallback hash, and the vote mail goes out with a broken link. + commit=$(git rev-parse --verify --quiet "${TAG}^{commit}" || git rev-parse --verify master) cat <