Submit a release's MSI to the Microsoft Store from a workflow

.github/workflows/ci.yml

@@ -153,6 +153,10 @@ jobs:
       # rejects the shell metacharacters that reach sed.
       - name: test release script helpers
         run: bash scripts/release/lib_test.sh
+      # msstore-submit.sh runs once per release against an API CI cannot
+      # reach, so its test answers in the API's place.
+      - name: test Store submission script
+        run: bash scripts/release/msstore-submit_test.sh
       - name: PSScriptAnalyzer install.ps1
         shell: pwsh
         # Write-Host is intentional here (colored installer UX output, which

.github/workflows/msstore.yml

@@ -0,0 +1,41 @@
+# Submit a release's desktop installer to the Microsoft Store.
+#
+# Run by hand, after the release is published AND the site has been deployed
+# since, because the Store fetches the MSI from netscli.com/download/<tag>/,
+# which only exists once pages.yml has staged it. The script checks that URL
+# first and stops if it is not there yet.
+#
+# Needs the MSSTORE_* secrets from an Entra app linked to Partner Center. See
+# docs/MICROSOFT-STORE.md, "Automating later releases".
+name: Microsoft Store
+
+on:
+  workflow_dispatch:
+    inputs:
+      tag:
+        description: Release tag to submit, for example v0.3.5
+        required: true
+
+permissions:
+  contents: read
+
+concurrency:
+  group: msstore
+  cancel-in-progress: false
+
+jobs:
+  submit:
+    name: Submit ${{ inputs.tag }}
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - name: Submit to the Store
+        env:
+          MSSTORE_TENANT_ID: ${{ secrets.MSSTORE_TENANT_ID }}
+          MSSTORE_CLIENT_ID: ${{ secrets.MSSTORE_CLIENT_ID }}
+          MSSTORE_CLIENT_SECRET: ${{ secrets.MSSTORE_CLIENT_SECRET }}
+          MSSTORE_SELLER_ID: ${{ secrets.MSSTORE_SELLER_ID }}
+          MSSTORE_PRODUCT_ID: ${{ secrets.MSSTORE_PRODUCT_ID }}
+          TAG: ${{ inputs.tag }}
+        run: scripts/release/msstore-submit.sh "$TAG"

docs/MICROSOFT-STORE.md

@@ -77,8 +77,57 @@ days. Each later release needs a new submission with that release's MSI URL.
 The site only picks up a new release's MSI when it is next deployed, so after
 publishing a release run the Deploy GitHub Pages workflow (or merge any site
 change) before giving the Store the new URL.
-That can be automated with the Store submission API once the first one has
-passed.
+Once the first submission has passed, the Microsoft Store workflow does
+this, as below.
+
+### 5. Automating later releases
+
+The Microsoft Store workflow (`.github/workflows/msstore.yml`) points the
+listing at a release's MSI and submits it, through the Store submission API
+(`scripts/release/msstore-submit.sh`). It is run by hand, with the release
+tag, after the site has redeployed. It stops without changing anything if the
+MSI URL is not live yet or a submission is still in certification, and it
+does not wait for certification itself.
+
+It needs an Entra app that Partner Center trusts. One-time setup:
+
+1. In Partner Center, open **Account settings**, then **Tenants**, and
+   associate a Microsoft Entra tenant. An individual account can create a new
+   one there if it has none.
+2. Under **Account settings**, open **User management**, then **Microsoft
+   Entra applications**, and create a new application named `netscli-store`
+   with the **Manager** role. Partner Center creates it in the tenant.
+3. Open that application in Partner Center and copy the tenant ID and client
+   ID. In the Azure portal, under **App registrations**, open it, then
+   **Certificates & secrets**, and create a client secret. Microsoft allows
+   up to 24 months and recommends under 12, so note the expiry date. The
+   workflow fails with "No token from Entra" once it passes.
+4. Set the five repository secrets. Each command asks for the value, so it
+   stays out of shell history.
+
+| Secret | Where it is |
+| --- | --- |
+| `MSSTORE_TENANT_ID` | Step 3 |
+| `MSSTORE_CLIENT_ID` | Step 3 |
+| `MSSTORE_CLIENT_SECRET` | The client secret value from step 3 |
+| `MSSTORE_SELLER_ID` | Partner Center, Account settings, Legal info, Seller ID |
+| `MSSTORE_PRODUCT_ID` | The app's Partner Center product ID |
+
+Microsoft's API documentation calls the last one "the Partner Center ID of
+the product" without saying where it is shown. Not yet checked against the
+live API: try the number in the app's Partner Center address first, and the
+Store ID (9N...) from **Product identity** if the API answers that the
+product does not exist.
+
+```bash
+for s in TENANT_ID CLIENT_ID CLIENT_SECRET SELLER_ID PRODUCT_ID; do gh secret set "MSSTORE_$s"; done
+```
+
+Then, for each release, once the Deploy GitHub Pages workflow has run:
+
+```bash
+gh workflow run msstore.yml -f tag=v0.3.5
+```
 
 ## Listing text
 

scripts/release/msstore-submit.sh

@@ -0,0 +1,93 @@
+#!/usr/bin/env bash
+# Point the Microsoft Store listing at a release's installer and submit it
+# for certification, through the Store submission API.
+#
+# Why: the Store does not pick up new MSI releases by itself. Each release
+# needs a submission whose package URL names that release's MSI, served by
+# netscli.com (see stage-store-downloads.sh). Done by hand in Partner Center
+# that is five screens per release.
+#
+# What it does, in the order the API requires:
+#   1. checks the installer URL answers 200 without a redirect, because the
+#      Store refuses one that redirects and would only say so days later
+#   2. refuses to start while another submission is still in certification
+#   3. rewrites the packageUrl of the draft's packages, keeping every other
+#      field, and commits them
+#   4. waits until the draft reports ready, then submits it
+# It does not wait for certification, which takes days. Partner Center shows
+# the result.
+#
+# Usage: msstore-submit.sh <tag>
+# Needs curl and jq, and these from an Entra app linked to the Partner Center
+# account (docs/MICROSOFT-STORE.md, "Automating later releases"):
+#   MSSTORE_TENANT_ID MSSTORE_CLIENT_ID MSSTORE_CLIENT_SECRET
+#   MSSTORE_SELLER_ID MSSTORE_PRODUCT_ID
+set -euo pipefail
+
+tag="${1:?usage: msstore-submit.sh <tag>}"
+: "${MSSTORE_TENANT_ID:?}" "${MSSTORE_CLIENT_ID:?}" "${MSSTORE_CLIENT_SECRET:?}"
+: "${MSSTORE_SELLER_ID:?}" "${MSSTORE_PRODUCT_ID:?}"
+
+api="https://api.store.microsoft.com/submission/v1/product/${MSSTORE_PRODUCT_ID}"
+package_url="https://netscli.com/download/${tag}/netscli-gui-windows-x86_64.msi"
+ready_attempts="${MSSTORE_READY_ATTEMPTS:-60}"
+
+status=$(curl -s -o /dev/null -w '%{http_code}' --head "${package_url}")
+if [ "${status}" != "200" ]; then
+  echo "${package_url} answered ${status}, not 200. Deploy the site after the release, then retry." >&2
+  exit 1
+fi
+
+token=$(curl -sf "https://login.microsoftonline.com/${MSSTORE_TENANT_ID}/oauth2/v2.0/token" \
+  --data-urlencode "grant_type=client_credentials" \
+  --data-urlencode "client_id=${MSSTORE_CLIENT_ID}" \
+  --data-urlencode "client_secret=${MSSTORE_CLIENT_SECRET}" \
+  --data-urlencode "scope=https://api.store.microsoft.com/.default" | jq -r '.access_token')
+if [ -z "${token}" ] || [ "${token}" = "null" ]; then
+  echo "No token from Entra. Check the MSSTORE_* secrets, and whether the client secret has expired." >&2
+  exit 1
+fi
+
+# Every API answer carries isSuccess and errors. Print the errors and stop on
+# a failure, otherwise print the body for the caller.
+call() {
+  local method="$1" path="$2" response
+  local args=(-s -X "${method}" "${api}${path}"
+    -H "Authorization: Bearer ${token}"
+    -H "X-Seller-Account-Id: ${MSSTORE_SELLER_ID}"
+    -H "Content-Type: application/json")
+  if [ -n "${3:-}" ]; then args+=(--data "$3"); fi
+  response=$(curl "${args[@]}")
+  if [ "$(jq -r '.isSuccess' <<<"${response}")" != "true" ]; then
+    echo "${method} ${path} failed:" >&2
+    jq -r '.errors[]? | "  \(.code): \(.message)"' <<<"${response}" >&2 || echo "${response}" >&2
+    return 1
+  fi
+  echo "${response}"
+}
+
+ongoing=$(call GET /status | jq -r '.responseData.ongoingSubmissionId // ""')
+if [ -n "${ongoing}" ]; then
+  echo "Submission ${ongoing} is still in progress. Wait for it to finish in Partner Center, then retry." >&2
+  exit 1
+fi
+
+packages=$(call GET /packages | jq --arg url "${package_url}" \
+  '{packages: [.responseData.packages[] | .packageUrl = $url]}')
+echo "Packages now:"
+jq -r '.packages[] | "  \(.packageType) \(.architectures | join(",")) \(.packageUrl)"' <<<"${packages}"
+call PUT /packages "${packages}" >/dev/null
+call POST /packages/commit >/dev/null
+
+# The Store downloads and checks the installer before the draft is ready.
+for ((i = 1; ; i++)); do
+  [ "$(call GET /status | jq -r '.responseData.isReady')" = "true" ] && break
+  if [ "${i}" -ge "${ready_attempts}" ]; then
+    echo "The draft was not ready after ${ready_attempts} checks. Look at Packages in Partner Center." >&2
+    exit 1
+  fi
+  sleep 10
+done
+
+submission=$(call POST /submit | jq -r '.responseData.submissionId')
+echo "Submitted ${tag} to the Microsoft Store as submission ${submission}."

scripts/release/msstore-submit_test.sh

@@ -0,0 +1,123 @@
+#!/usr/bin/env bash
+# Tests for msstore-submit.sh against a fake Store API.
+#
+# The script only runs for real once per release, against an API nobody can
+# call from CI without the Store secrets, so a mistake in it would surface
+# at release time as a half-made submission. A fake `curl` on PATH answers
+# like the API does and records each request.
+#
+# Run: bash scripts/release/msstore-submit_test.sh
+
+set -euo pipefail
+
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+TMP="$(mktemp -d)"
+trap 'rm -rf "$TMP"' EXIT
+
+failures=0
+ok() { echo "  ok   - $1"; }
+bad() { echo "  FAIL - $1" >&2; failures=$((failures + 1)); }
+
+mkdir -p "$TMP/bin"
+printf '#!/bin/sh\n' >"$TMP/bin/sleep"
+cat >"$TMP/bin/curl" <<'EOF'
+#!/usr/bin/env bash
+# The draft is ready from the third status check onwards.
+echo "$*" >>"$FAKE/log"
+case "$*" in
+  *--head*) printf '%s' "${FAKE_HEAD:-200}" ;;
+  *oauth2*) echo '{"access_token":"tok"}' ;;
+  *"-X GET"*/status*)
+    n=$(cat "$FAKE/n" 2>/dev/null || echo 0)
+    echo $((n + 1)) >"$FAKE/n"
+    ongoing=""
+    [ "$n" = 0 ] && ongoing="${FAKE_ONGOING:-}"
+    ready=false
+    [ "$n" -ge 3 ] && ready=true
+    echo "{\"isSuccess\":true,\"responseData\":{\"isReady\":$ready,\"ongoingSubmissionId\":\"$ongoing\"}}" ;;
+  *"-X GET"*/packages*)
+    echo '{"isSuccess":true,"responseData":{"packages":[{"packageId":"p1","packageUrl":"https://netscli.com/download/v0.3.4/netscli-gui-windows-x86_64.msi","languages":["en-us"],"architectures":["X64"],"isSilentInstall":true,"packageType":"msi"}]}}' ;;
+  *"-X PUT"*)
+    prev=""
+    for a in "$@"; do [ "$prev" = "--data" ] && printf '%s' "$a" >"$FAKE/put"; prev="$a"; done
+    if [ -n "${FAKE_PUT_FAIL:-}" ]; then
+      echo '{"isSuccess":false,"errors":[{"code":"badrequest","message":"bad package"}]}'
+    else
+      echo '{"isSuccess":true}'
+    fi ;;
+  *"-X POST"*/packages/commit*) echo '{"isSuccess":true}' ;;
+  *"-X POST"*/submit*) echo '{"isSuccess":true,"responseData":{"submissionId":"999"}}' ;;
+  *) echo "fake curl: unexpected request: $*" >&2; exit 9 ;;
+esac
+EOF
+chmod +x "$TMP/bin/curl" "$TMP/bin/sleep"
+
+# Runs the script with the given extra environment. Sets $status and $out,
+# and leaves the request log in $FAKE.
+run() {
+  FAKE="$(mktemp -d "$TMP/run.XXXX")"
+  set +e
+  out=$(env PATH="$TMP/bin:$PATH" FAKE="$FAKE" \
+    MSSTORE_TENANT_ID=t MSSTORE_CLIENT_ID=c MSSTORE_CLIENT_SECRET=s \
+    MSSTORE_SELLER_ID=seller MSSTORE_PRODUCT_ID=prod "$@" \
+    bash "$HERE/msstore-submit.sh" v0.3.5 2>&1)
+  status=$?
+  set -e
+}
+
+echo "msstore-submit.sh"
+
+run
+if [ "$status" = 0 ] && grep -q "submission 999" <<<"$out"; then ok "submits"; else bad "submits: $out"; fi
+want="https://netscli.com/download/v0.3.5/netscli-gui-windows-x86_64.msi"
+if [ "$(jq -r '.packages[0].packageUrl' "$FAKE/put")" = "$want" ]; then
+  ok "points the package at the release's MSI"
+else
+  bad "package URL: $(cat "$FAKE/put")"
+fi
+if [ "$(jq -c '.packages[0] | del(.packageUrl)' "$FAKE/put")" = \
+  '{"packageId":"p1","languages":["en-us"],"architectures":["X64"],"isSilentInstall":true,"packageType":"msi"}' ]; then
+  ok "keeps every other package field"
+else
+  bad "other fields changed: $(cat "$FAKE/put")"
+fi
+if grep -q "X-Seller-Account-Id: seller" "$FAKE/log"; then ok "sends the seller id"; else bad "no seller header"; fi
+order=$(grep -oE -- '-X [A-Z]+ [^ ]+' "$FAKE/log" | sed 's|https://api.store.microsoft.com/submission/v1/product/prod||' | uniq | tr '\n' ' ')
+if [ "$order" = "-X GET /status -X GET /packages -X PUT /packages -X POST /packages/commit -X GET /status -X POST /submit " ]; then
+  ok "calls the API in order, waiting until the draft is ready"
+else
+  bad "call order: $order"
+fi
+
+run FAKE_HEAD=302
+if [ "$status" != 0 ] && ! grep -q "oauth2" "$FAKE/log"; then
+  ok "stops before the API when the installer URL redirects"
+else
+  bad "redirect: status $status, $out"
+fi
+
+run FAKE_ONGOING=123
+if [ "$status" != 0 ] && grep -q "123 is still in progress" <<<"$out" && ! grep -q -- "-X PUT" "$FAKE/log"; then
+  ok "changes nothing while a submission is in certification"
+else
+  bad "ongoing: status $status, $out"
+fi
+
+run FAKE_PUT_FAIL=1
+if [ "$status" != 0 ] && grep -q "badrequest: bad package" <<<"$out" && ! grep -q "/submit" "$FAKE/log"; then
+  ok "prints the API's errors and does not submit"
+else
+  bad "PUT failure: status $status, $out"
+fi
+
+run MSSTORE_READY_ATTEMPTS=2
+if [ "$status" != 0 ] && grep -q "not ready after 2 checks" <<<"$out" && ! grep -q "/submit" "$FAKE/log"; then
+  ok "gives up when the draft never becomes ready"
+else
+  bad "never ready: status $status, $out"
+fi
+
+if [ "$failures" -gt 0 ]; then
+  echo "$failures failed" >&2
+  exit 1
+fi