Harden the release preflight, and bring the docs back in line with the tests

docs/changes/changes.md

@@ -23,10 +23,19 @@ Targets SkyWalking OAP 11.0.0, Horizon UI 1.0.0 and BanyanDB 0.11.0. See
   seed files and `UITemplateInitializer`, along with the sidebar menu storage, the
   `UIConfigurationManagement` GraphQL mutations and `SW_ENABLE_UPDATE_UI_TEMPLATE`. Horizon UI
   ships its own dashboard library and manages templates over the admin REST port.
-- **`ui.config` is empty by default.** The chart now writes only the values it computes, as
-  `${HORIZON_*:default}` tokens, and Horizon is configured by environment variable. A literal
-  written into `ui.config` makes that field's `HORIZON_*` variable inert — see
-  [Configure Horizon](../ui/configure.md).
+- **Horizon is configured by environment variable, and no ConfigMap is mounted by default.** The
+  image ships a complete env-tokenized `/app/horizon.yaml`; the chart sets only what it computes
+  (`HORIZON_SERVER_PORT`, `HORIZON_OAP_QUERY_URL`, and the admin, Zipkin and public URLs when
+  configured) and leaves the rest to `ui.extraEnv` / `ui.envFromSecret`. `ui.config` is now opt-in:
+  setting it renders a ConfigMap and mounts it *over* the image's file, so fields you do not write
+  fall back to Horizon's defaults. If you carried a `ui.config` block from the pre-release `main`
+  values, move it to environment variables — see [Configure Horizon](../ui/configure.md).
+- **The SWCK charts are removed.** `chart/operator` and `chart/adapter` packaged
+  [apache/skywalking-swck](https://github.com/apache/skywalking-swck) — its image, its CRDs and its
+  version — and had no relationship to `chart/skywalking`. They were never released to Docker Hub,
+  so no released artifact disappears, but installs from source or from the `ghcr.io` snapshot
+  channel will break. They belong with the operator, where the CRDs are generated alongside the
+  code that consumes them.
 - **The UI no longer proxies `/graphql`.** Callers that talked to the UI's GraphQL endpoint
   (for example `swctl --base-url=http://<ui>/graphql`) must target the OAP service directly on
   `oap.ports.rest`.
@@ -40,6 +49,16 @@ Targets SkyWalking OAP 11.0.0, Horizon UI 1.0.0 and BanyanDB 0.11.0. See
   reach — see [UI Service and Ingress](../expose/ui-service-and-ingress.md).
 - `server.port` is derived from `ui.service.internalPort`, so the BFF binds the port the container
   exposes.
+- `oap.extraEnv` (a list, so entries can carry `valueFrom`) and `oap.envFromSecret`, applied to the
+  OAP Deployment and the init Job. Note Kubernetes gives an explicit `env` entry precedence over
+  `envFrom`, and the chart sets `SW_ES_PASSWORD` / `SW_DATA_SOURCE_PASSWORD` itself — so sourcing
+  those from a Secret needs `oap.extraEnv`.
+- Horizon's config hot-reload works again. The chart previously mounted `horizon.yaml` with
+  `subPath`, which Kubernetes never updates in place, so the file watcher could not fire.
+- `tools/releasing/release.sh` and `release-passed.sh`, plus
+  [the release guide](../contributing/release.md) — the Apache process was previously unwritten.
+- The E2E suite is rebuilt around Horizon: every assertion runs through the UI's API rather than
+  OAP's GraphQL, so it exercises the path the chart is responsible for wiring.
 - Documentation moved into `docs/` and is published at
   [skywalking.apache.org/docs/skywalking-helm](https://skywalking.apache.org/docs/skywalking-helm/next/readme/).
 

docs/contributing/e2e-tests.md

@@ -16,7 +16,7 @@ the traffic to generate, and the assertions to make. The phases are:
 | Phase | What it does here |
 | --- | --- |
 | `setup` | Creates the kind cluster from `file: kind28.yaml`, loads `init-system-environment: env` into the shell environment, then runs the `steps:` in order — install tooling, install Istio, `helm install` the chart, deploy bookinfo, start traffic. Each step may declare `wait:` conditions; the whole phase has `timeout: 25m`. |
-| `trigger` | Unused. All five cells generate load with a `wrk` Deployment (`test/e2e/traffic-gen.yaml`) applied as a setup step, so no cell has a `trigger:` block. |
+| `trigger` | Unused. The three traffic cells generate load with a `wrk` Deployment (`test/e2e/traffic-gen.yaml`) applied as a setup step, so no cell has a `trigger:` block. |
 | `verify` | Runs each `query:` and matches its output against a template in `test/e2e/expected/`, retrying on `retry: {count: 30, interval: 10s}`. |
 | `cleanup` | No cell declares `cleanup:`, so infra-e2e's own default applies — `always` when `CI=true` (GitHub Actions always tears down), `success` otherwise. A locally *failed* run therefore leaves the cluster up for debugging. |
 
@@ -38,9 +38,9 @@ interpolates into a base URL. The OAP's own ports are not exposed at all.
 and `{{ notEmpty .version }}`, so a case asserts "these services are present" or "this field has a
 value", not an exact payload.
 
-## The five cells
+## The four cells
 
-All five run the same fixture — kind `kindest/node:v1.28.15` (one control plane, three workers),
+Three of them run the same fixture — kind `kindest/node:v1.28.15` (one control plane, three workers),
 namespace `istio-system`, `fullnameOverride=skywalking`, `oap.replicas=1`, Satellite enabled, Horizon
 UI at `$UI_REPO:$UI_TAG` — and differ in which OAP and which storage they install, plus the flags
 each of those pairings needs.
@@ -51,29 +51,40 @@ each of those pairings needs.
 | `test/e2e/e2e-oap11-banyandb-standalone.yaml` | 11.0.0 | BanyanDB 0.11 (`$BANYANDB_REPO:$BANYANDB_TAG`) | `oap.storageType=banyandb`, `elasticsearch.enabled=false`, `banyandb.enabled=true`, `banyandb.standalone.enabled=true`, `banyandb.cluster.enabled=false`, `banyandb.auth.enabled=true`. |
 | `test/e2e/e2e-oap11-banyandb-cluster.yaml` | 11.0.0 | BanyanDB 0.11, cluster mode | As above but `banyandb.standalone.enabled=false`, `banyandb.cluster.enabled=true`. |
 
-The three OAP 11 cells also switch Zipkin on — `oap.ports.zipkin-query=9412` plus
-`SW_RECEIVER_ZIPKIN=default` and `SW_QUERY_ZIPKIN=default` — because the chart only emits
-`oap.zipkinUrl` into Horizon's config when that port is set, and one verify case asserts Horizon can
-reach it.
+All three also switch Zipkin on — `oap.ports.zipkin-query=9412` plus `SW_RECEIVER_ZIPKIN=default`
+and `SW_QUERY_ZIPKIN=default` — because the chart only emits `oap.zipkinUrl` into Horizon's config
+when that port is set, and one verify case asserts Horizon can reach it. Each runs seven cases.
 
-### How the OAP 10.4 cells differ
+### The fourth cell: `e2e-config-override.yaml`
 
-The admin host, and the `/ui-management` template store mounted on it, arrived in OAP 11. On 10.4
-the chart must not render an admin port (`oap.ports.admin=null`) and Horizon must run its template
-store from the bundle in its own image (`ui.config.templates.mode=readonly`).
+The other three take the default configuration path — Horizon set up purely by environment variable,
+with no ConfigMap created at all. This cell covers the opt-in third mechanism for **both**
+components: a file rendered into a ConfigMap and mounted over the image's own.
 
-The important part is what happens to the assertions: they are **inverted, not dropped**. Those
-Horizon endpoints answer HTTP 200 either way and report the failure in the body, so the 10.4 cells
-still call them and expect the negative answer:
+| | |
+| --- | --- |
+| Storage | BanyanDB 0.11 standalone — the cheapest backend that still exercises a real one |
+| UI override | `ui.config.session.ttlMinutes=17`, which makes the chart render and mount the UI ConfigMap |
+| OAP override | `oap.config.log4j2\.xml=<Configuration …>`, mounted at `/skywalking/config/log4j2.xml` |
+| Cases | Four |
 
-| Case | OAP 11 expects | OAP 10.4 expects |
-| --- | --- | --- |
-| `GET /api/preflight?refresh=1` | `expected/horizon-admin-live.yml` — `adminReachable: true`, `templatesMode: live`, `uiManagement: true` | `expected/horizon-admin-readonly.yml` — `adminReachable: false`, `templatesMode: readonly` |
-| `GET /api/admin/templates/sync-status?force=true` | `expected/horizon-templates-live.yml` — `mode: live`, `unreachable: false` | `expected/horizon-templates-readonly.yml` — `mode: readonly`, `unreachable: false` |
+It is deliberately cheap — no Istio, no bookinfo, no traffic generator. What is under test is
+whether the overrides are read, which needs none of that, so it skips the `MESH` fixture entirely
+and never queries a metric.
+
+The mount is the riskier path, and the reason this cell exists. Mounting over Horizon's
+`/app/horizon.yaml` *replaces* it, so the chart has to merge its computed values back in as
+`${HORIZON_*:default}` tokens. Two of the four cases exist to catch that regression rather than to
+check the ConfigMap: logging in proves the `HORIZON_AUTH_LOCAL_USERS` token survived the mount
+(without it the pod is Ready and nobody can sign in), and `/api/oap/info` proves `queryUrl` did not
+revert to `127.0.0.1`.
+
+### There are no OAP 10.4 cells
 
-`unreachable: false` in readonly mode is the point of that second row: readonly still serves
-templates, from the image, without ever contacting the OAP. The OAP 10.4 cells run six verify cases
-to the OAP 11 cells' seven — the missing one is the Zipkin check, since they do not enable Zipkin.
+CI covers the OAP 11 line only. [Version Compatibility](../evaluate/version-compatibility.md) still
+documents running Horizon 1.0.0 against OAP 10.4.0 — `oap.ports.admin=null` and
+`HORIZON_TEMPLATES_MODE=readonly` — but nothing here tests it, so treat that combination as
+documented rather than verified.
 
 ## The traffic fixture: everything lands in `MESH`
 
@@ -246,7 +257,7 @@ bash test/e2e/script/horizon.sh http://localhost:8080 get /api/layer/MESH/servic
 ## How CI runs them
 
 `.github/workflows/e2e.ci.yaml` runs the suite from one matrix job, `als`, with `fail-fast: false`
-and a matrix of five entries — one per cell — each with a 60-minute timeout, so one failing storage
+and a matrix of four entries — one per cell — each with a 60-minute timeout, so one failing storage
 backend does not cancel the other four:
 
 ```yaml
@@ -256,7 +267,7 @@ strategy:
     test:
       - name: Horizon + OAP 11 + Elasticsearch
         config: test/e2e/e2e-oap11-elasticsearch.yaml
-      # …and the two OAP 11 BanyanDB cells plus the two OAP 10.4 cells
+      # …the two OAP 11 BanyanDB cells, and e2e-config-override.yaml
 ```
 
 Each entry logs in to `ghcr.io` (the Satellite and BanyanDB 0.11 images live there), sets up Go 1.24,
@@ -278,9 +289,10 @@ filter applies only to the push trigger, so a docs-only pull request still runs
 
 ## Adding a case
 
-1. Add the `query:` to **all five** cells unless it is version-specific, and put the expectation in
-   `test/e2e/expected/`. If the answer differs between OAP 11 and 10.4, invert it into a second
-   expected file rather than skipping the case — see the table above.
+1. Add the `query:` to **all three traffic cells**, and put the expectation in
+   `test/e2e/expected/`. Leave `e2e-config-override.yaml` alone unless the case is about
+   configuration: it installs no Istio and generates no traffic, so anything asking for a metric
+   will never pass there.
 2. Go through `test/e2e/script/horizon.sh`. Anything that calls the OAP directly is testing the OAP.
 3. Query the `MESH` layer, and project the response with `yq` down to the fields that carry meaning.
 4. Keep image references as `$OAP_REPO` / `$OAP_TAG` style variables so `test/e2e/env` stays the only

docs/contributing/release.md

@@ -169,12 +169,25 @@ irreversible step comes last, and a failure leaves nothing on the remote to clea
 Preflight refuses to start when:
 
 - the host is not Linux;
-- any of `helm`, `gpg`, `shasum`, `svn`, `git`, `make` is missing;
+- any of `helm`, `gpg`, `shasum`, `svn`, `git`, `make`, `tar`, `awk` is missing. All of them are
+  reported in one message — finding them one at a time costs one failed run per package;
+- `helm` is older than 3.8. The chart ships only as an OCI artifact, and `helm push` to an `oci://`
+  registry arrived in 3.8. Helm 4 is accepted: `Chart.yaml` is `apiVersion: v2`, which both majors
+  read;
+- `gpg` holds no secret key. `make release` signs with `gpg --batch`, so without one the run would
+  fail *after* building and packaging everything;
+- `dist/dev/skywalking` cannot be read — a network problem, or svn credentials that are not set up;
 - the working tree is dirty — `release-src` archives the working *tree*, not `HEAD`;
 - a `*.tgz`, `*.tgz.asc` or `*.tgz.sha512` is lying in the repository root. Those are gitignored, so
   `git status` cannot see them, and a leftover from a previous release would be embedded in this
   release's source tarball;
-- `v$VERSION` already exists.
+- `v$VERSION` already exists, or `dist/dev/skywalking/helm/$VERSION` already does. The second catches
+  a re-run after a partial upload, which would otherwise only surface at `svn commit` — after the
+  build, the signing and the tag push.
+
+The point of the capability checks is *when* they fail. A missing signing key or unusable svn
+credentials are both perfectly capable of stopping a release half-way through, with a tag already on
+the remote; preflight is the only place where stopping is free.
 
 `verify_artifacts` checks, for each of the two artifacts, that the file and its `.asc` and `.sha512`
 are present, that `gpg --batch --verify` passes and that `shasum -a 512 -c` passes. It then runs
@@ -344,13 +357,17 @@ as a no *without* aborting, so it walks the whole plan and does none of it.
 
 | Stage | Prompt | Declining |
 | --- | --- | --- |
-| `preflight` | — | fails if `svn` / `gh` / `git` are missing, or if `dist/dev/skywalking/helm/$VERSION` does not exist |
+| `preflight` | — | fails if `svn` / `gh` / `git` are missing, if `gh` is not authenticated, if `dist/release/skywalking` cannot be read, or if `dist/dev/skywalking/helm/$VERSION` does not exist |
 | `promote_artifacts` | `svn mv` from `dist/dev` to `dist/release` | aborts |
 | `remove_previous` | remove everything under `release/helm/` other than `$VERSION` | **skips and continues** — the one exception |
 | `github_release` | `gh release create` — this is what publishes the chart | aborts |
 | `announce_mail` | — | prints the mail, sends nothing |
 | `remaining` | — | prints what is left by hand |
 
+`gh auth status` is checked in preflight rather than left to `github_release`, because
+`github_release` runs *after* `promote_artifacts` — and an `svn mv` into `dist/release` cannot be
+taken back. An unauthenticated `gh` has to stop the run before that, or not at all.
+
 `remove_previous` is the exception because skipping it is survivable: `dist/release` is meant to
 hold only the current version, but a stale sibling breaks nothing. Declining prints
 `remove them by hand before announcing`, and the run continues. It lists what is there and asks

docs/evaluate/requirements.md

@@ -12,7 +12,7 @@ default (Elasticsearch) path requires.
 | `kubectl` | matching your cluster | Used to watch the install (`kubectl get pods -w`) and for all troubleshooting. |
 
 CI exercises the chart on a four-node [kind](https://kind.sigs.k8s.io/) cluster running
-`kindest/node:v1.28.15` (`test/e2e/kind28.yaml`) — all five e2e suites use it — so 1.28 is the
+`kindest/node:v1.28.15` (`test/e2e/kind28.yaml`) — every e2e cell uses it — so 1.28 is the
 version with the most coverage.
 
 The chart adapts to older Ingress APIs (`ui-ingress.yaml` falls back from

docs/evaluate/version-compatibility.md

@@ -111,7 +111,7 @@ Details: [Elasticsearch](../storage/elasticsearch.md), [BanyanDB](../storage/ban
 ## Kubernetes and Helm
 
 - **Helm 3 or newer**, and **3.8 or newer** for the `oci://` install above. Chart `5.0.0` is `apiVersion: v2`, which rules out Helm 2; the chart README and [Quick Start](../install/quick-start.md) both list 3.8+ because the chart is only published as an OCI artifact.
-- All five e2e cells run on one kind config, `test/e2e/kind28.yaml` — **`kindest/node:v1.28.15`**, a control plane and three workers. The chart declares no `kubeVersion` constraint.
+- Every e2e cell runs on one kind config, `test/e2e/kind28.yaml` — **`kindest/node:v1.28.15`**, a control plane and three workers. The chart declares no `kubeVersion` constraint.
 
 ## Before you change a version
 

test/e2e/e2e-config-override.yaml

@@ -16,7 +16,7 @@
 # Covers the third configuration mechanism for both components: a file rendered
 # into a ConfigMap and mounted over the image's own.
 #
-# The other five cells all take the default path -- Horizon configured purely by
+# The other three cells all take the default path -- Horizon configured purely by
 # environment variable, with no ConfigMap created at all -- so without this the
 # opt-in mount would ship untested. It is also the riskier path: mounting over
 # Horizon's /app/horizon.yaml REPLACES it, and the chart has to merge its

tools/releasing/release-passed.sh

@@ -51,7 +51,9 @@ confirm() {
   ${DRY_RUN} && { log "dry run: would $1"; return 1; }
   log "About to $1"
   log "Proceed? [y/N]"
-  read -r reply
+  # `read` exits 1 at EOF. Without this the run reports "declined" when in truth there was no
+  # terminal to ask -- e.g. the script was piped, or run from CI.
+  read -r reply || die "nothing on stdin -- run this script from a terminal"
   [[ "${reply}" == "y" || "${reply}" == "Y" ]] && return 0
   die "declined -- aborting before anything further"
 }
@@ -64,9 +66,20 @@ preflight() {
   TAG="v${VERSION}"
   log "publishing ${VERSION}"
 
+  local missing=""
   for tool in svn gh git; do
-    command -v "${tool}" >/dev/null || die "${tool} is not installed"
+    command -v "${tool}" >/dev/null || missing="${missing} ${tool}"
   done
+  [[ -z "${missing}" ]] || die "not installed:${missing}"
+
+  # gh is used in github_release(), which runs AFTER the svn promotion -- and that
+  # promotion cannot be undone. An unauthenticated gh has to fail here or not at all.
+  gh auth status >/dev/null 2>&1 \
+    || die "gh is not authenticated -- run 'gh auth login'. The GitHub release is created after the
+       svn promotion, which cannot be undone, so this has to be working before anything starts."
+
+  svn ls "${SVN_RELEASE_URL}" >/dev/null 2>&1 \
+    || die "cannot read ${SVN_RELEASE_URL} -- check your network and your ASF svn credentials"
 
   svn ls "${SVN_DEV_URL}/helm/${VERSION}" >/dev/null 2>&1 \
     || die "${SVN_DEV_URL}/helm/${VERSION} does not exist -- was release.sh run?"
@@ -103,7 +116,7 @@ remove_previous() {
   echo
   log "Remove ALL of the above? [y/N]"
   ${DRY_RUN} && { log "dry run: not removing"; return; }
-  read -r reply
+  read -r reply || die "nothing on stdin -- run this script from a terminal"
   if [[ "${reply}" == "y" || "${reply}" == "Y" ]]; then
     local targets=()
     while IFS= read -r entry; do