Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .github/actions/prepare-test-host/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: Prepare isolated DSP test tools
description: Satisfy the DSP's existing trusted-tool rules on a disposable hosted runner
runs:
using: composite
steps:
- name: Require an isolated GitHub-hosted runner
shell: bash
env:
RUNNER_KIND: ${{ runner.environment }}
run: test "$RUNNER_KIND" = github-hosted
- name: Prepare trusted Node and Chrome paths
shell: bash
run: |
# Hosted tool caches may have writable parents. Do not weaken runtime checks.
sudo install -d -o root -g root -m 755 /dispatch-ci /dispatch-ci/bin
sudo install -o root -g root -m 755 "$(command -v node)" /dispatch-ci/bin/node
echo /dispatch-ci/bin >> "$GITHUB_PATH"
test -x /opt/google/chrome/chrome
sudo chown root:root /opt /opt/google
sudo chown -R root:root /opt/google/chrome
sudo chmod go-w /opt /opt/google /opt/google/chrome
sudo chmod -R go-w /opt/google/chrome
sudo chmod 4755 /opt/google/chrome/chrome-sandbox
stat -Lc '%u %a %n' /opt /opt/google /opt/google/chrome /opt/google/chrome/chrome /dispatch-ci/bin/node
echo 'DISPATCH_CHROME_EXECUTABLE=/opt/google/chrome/chrome' >> "$GITHUB_ENV"
command -v Xvfb
command -v setpriv
/usr/bin/python3 -c "import ctypes; ctypes.CDLL('libX11.so.6'); ctypes.CDLL('libXtst.so.6')"
51 changes: 43 additions & 8 deletions .github/workflows/checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,31 +4,66 @@ on:
push:
branches: [main]
workflow_dispatch:
concurrency:
group: dsp-checks-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
platform:
uses: ./.github/workflows/platform-bundle.yml
dsp:
runs-on: ubuntu-latest
needs: platform
runs-on: ubuntu-24.04
timeout-minutes: 20
env:
DISPATCH_PLATFORM_PACKAGES_URL: ${{ vars.DISPATCH_PLATFORM_PACKAGES_URL }}
DISPATCH_PLATFORM_PACKAGES_SHA256: ${{ vars.DISPATCH_PLATFORM_PACKAGES_SHA256 }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/setup-node@v4
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: 22
cache: npm
cache-dependency-path: tooling/frontend/package-lock.json
- run: python3 tooling/fetch-platform.py "$RUNNER_TEMP/platform-packages"
- uses: ./.github/actions/prepare-test-host
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: platform-bundle
path: ${{ runner.temp }}/dependency
- name: Verify and install platform packages
env:
BUNDLE_SHA256: ${{ needs.platform.outputs.sha256 }}
run: |
cd "$RUNNER_TEMP/dependency"
printf '%s platform-bundle.tar.gz\n' "$BUNDLE_SHA256" | sha256sum --check --strict
mkdir "$RUNNER_TEMP/platform-packages"
tar -xzf platform-bundle.tar.gz -C "$RUNNER_TEMP/platform-packages"
- name: Validate GitHub workflows
run: |
curl --fail --silent --show-error --location https://github.com/rhysd/actionlint/releases/download/v1.7.12/actionlint_1.7.12_linux_amd64.tar.gz --output "$RUNNER_TEMP/actionlint.tar.gz"
printf '%s %s\n' '8aca8db96f1b94770f1b0d72b6dddcb1ebb8123cb3712530b08cc387b349a3d8' "$RUNNER_TEMP/actionlint.tar.gz" | sha256sum --check --strict
tar -xzf "$RUNNER_TEMP/actionlint.tar.gz" -C "$RUNNER_TEMP" actionlint
"$RUNNER_TEMP/actionlint"
- run: npm run bootstrap -- "$RUNNER_TEMP/platform-packages"
- name: Verify trusted test executables
run: |
node - <<'JS'
const assert = require('node:assert/strict');
const { execFileSync } = require('node:child_process');
const { resolveRootExecutable, trustedCommandPath } = require('dispatch-protocol/trusted-command-path');
assert(resolveRootExecutable(process.env.DISPATCH_CHROME_EXECUTABLE, []), 'Chrome must satisfy production tool trust rules');
assert(resolveRootExecutable(undefined, ['setpriv']), 'setpriv must be trusted');
const version = execFileSync('/usr/bin/env', ['node', '--version'], { env: { PATH: trustedCommandPath() }, encoding: 'utf8' }).trim();
assert.equal(version, process.version, 'Collectors must use the selected Node version');
console.log('Trusted Node, Chrome and launcher are available');
JS
- run: npm run check
- run: node tooling/ci-browser-smoke.js
- run: npm test
- run: npm run build -- "$RUNNER_TEMP/dsp-candidate"
- uses: actions/upload-artifact@v4
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: dsp-development-candidate
path: ${{ runner.temp }}/dsp-candidate
retention-days: 7
if-no-files-found: error
61 changes: 61 additions & 0 deletions .github/workflows/platform-bundle.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: Verified platform dependency
on:
workflow_call:
inputs:
production:
type: boolean
default: false
outputs:
sha256:
value: ${{ jobs.bundle.outputs.sha256 }}
permissions:
contents: read
jobs:
bundle:
runs-on: ubuntu-latest
timeout-minutes: 10
outputs:
sha256: ${{ steps.archive.outputs.sha256 }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: 22
package-manager-cache: false
- run: python3 -m unittest discover -s tooling -p 'test_*.py'
- id: lock
env:
PRODUCTION: ${{ inputs.production }}
run: |
if [ "$PRODUCTION" = true ]; then
python3 tooling/platform-lock.py --production
else
python3 tooling/platform-lock.py
fi
- if: steps.lock.outputs.mode == 'source'
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: ${{ github.repository_owner }}/${{ steps.lock.outputs.repository }}
ref: ${{ steps.lock.outputs.commit }}
path: platform-source
persist-credentials: false
- if: steps.lock.outputs.mode == 'source'
env:
EXPECTED_COMMIT: ${{ steps.lock.outputs.commit }}
run: |
test "$(git -C platform-source rev-parse HEAD)" = "$EXPECTED_COMMIT"
node platform-source/tooling/platform-packages.js build "$RUNNER_TEMP/platform-packages"
- if: steps.lock.outputs.mode == 'release'
run: python3 tooling/fetch-platform.py "$RUNNER_TEMP/platform-packages"
- id: archive
run: |
tar --sort=name --mtime=@0 --owner=0 --group=0 --numeric-owner -czf "$RUNNER_TEMP/platform-bundle.tar.gz" -C "$RUNNER_TEMP/platform-packages" .
echo "sha256=$(sha256sum "$RUNNER_TEMP/platform-bundle.tar.gz" | cut -d ' ' -f 1)" >> "$GITHUB_OUTPUT"
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: platform-bundle
path: ${{ runner.temp }}/platform-bundle.tar.gz
retention-days: 1
if-no-files-found: error
100 changes: 100 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
name: Publish dsp release
on:
workflow_dispatch:
inputs:
version:
description: Version explicitly selected by the owner (X.Y.Z)
required: true
type: string
expected_main:
description: Verified latest main commit (40 characters)
required: true
type: string
changelog:
description: Reviewed release notes
required: true
type: string
concurrency:
group: dsp-release-publication
cancel-in-progress: false
permissions:
contents: read
jobs:
platform:
if: github.ref == 'refs/heads/main' && inputs.expected_main == github.sha
uses: ./.github/workflows/platform-bundle.yml
with:
production: true
publish:
if: github.ref == 'refs/heads/main' && inputs.expected_main == github.sha
needs: platform
runs-on: ubuntu-24.04
timeout-minutes: 30
permissions:
contents: write
actions: read
id-token: write
attestations: write
env:
RELEASE_VERSION: ${{ inputs.version }}
RELEASE_COMMIT: ${{ inputs.expected_main }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ inputs.expected_main }}
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: 22
cache: npm
cache-dependency-path: tooling/frontend/package-lock.json
- uses: ./.github/actions/prepare-test-host
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: platform-bundle
path: ${{ runner.temp }}/dependency
- name: Verify platform bundle
env:
BUNDLE_SHA256: ${{ needs.platform.outputs.sha256 }}
run: |
cd "$RUNNER_TEMP/dependency"
printf '%s platform-bundle.tar.gz\n' "$BUNDLE_SHA256" | sha256sum --check --strict
mkdir "$RUNNER_TEMP/platform-packages"
tar -xzf platform-bundle.tar.gz -C "$RUNNER_TEMP/platform-packages"
- run: npm run bootstrap -- "$RUNNER_TEMP/platform-packages"
- name: Set release package paths
run: |
echo "RELEASE_NOTES_FILE=$RUNNER_TEMP/release-notes.md" >> "$GITHUB_ENV"
- name: Verify selected main and unused version
env:
GH_TOKEN: ${{ github.token }}
run: node tooling/release.js guard
- name: Verify trusted test executables
run: |
node - <<'JS'
const assert = require('node:assert/strict');
const { execFileSync } = require('node:child_process');
const { resolveRootExecutable, trustedCommandPath } = require('dispatch-protocol/trusted-command-path');
assert(resolveRootExecutable(process.env.DISPATCH_CHROME_EXECUTABLE, []), 'Chrome must satisfy production tool trust rules');
assert(resolveRootExecutable(undefined, ['setpriv']), 'setpriv must be trusted');
const version = execFileSync('/usr/bin/env', ['node', '--version'], { env: { PATH: trustedCommandPath() }, encoding: 'utf8' }).trim();
assert.equal(version, process.version, 'Collectors must use the selected Node version');
console.log('Trusted Node, Chrome and launcher are available');
JS
- run: npm run check
- run: node tooling/ci-browser-smoke.js
- run: npm test
- run: npm run build -- "$RUNNER_TEMP/candidate"
- name: Package the reviewed release
env:
RELEASE_NOTES: ${{ inputs.changelog }}
run: |
printf '%s\n' "$RELEASE_NOTES" > "$RELEASE_NOTES_FILE"
node tooling/release.js package "$RUNNER_TEMP/candidate" "$RUNNER_TEMP/publication"
- uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2
with:
subject-path: ${{ runner.temp }}/publication/*
- name: Publish verified assets to GitHub Releases
env:
GH_TOKEN: ${{ github.token }}
run: node tooling/release.js publish "$RUNNER_TEMP/publication"
8 changes: 5 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Dispatch DSP

This is the future `dispatch-dsp` repository. The current task is local only:
do not initialize Git, publish, or change production. In the development workspace,
read the workspace `AGENTS.md` and `dev/AGENTS.md` as well.
This is the `dispatch-dsp` repository. Develop in feature worktrees,
open PRs, and present verified PR details before requesting merge approval.
Merge only with explicit approval in chat. Publishing requires a user-requested
release and user-selected version. Installation is a separate owner action.
In the Dispatch workspace, also read the workspace `AGENTS.md` and `dev/AGENTS.md`.

- `runtime/`: DSP supervisor, jobs, collector execution, scoped workers and local vault/session handling.
- `plugins/`: Paycom and future plugin source, including each plugin's frontend.
Expand Down
38 changes: 33 additions & 5 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,17 @@
# DSP development

Requires Node 22 or 24 and npm. Obtain the verified platform package bundle from
Core's build/release artifacts, then run from this repository:
Requires Node 22 or 24 and npm. The full test suite also runs real Chrome against
local intercepted fixtures: install root-owned Chrome, `setpriv`, Xvfb, Python 3,
libX11 and libXtst. Set `DISPATCH_CHROME_EXECUTABLE` if Chrome is outside the trusted
command path. No real provider credentials are needed. Create worktrees with
`umask 022`; source and executable parents must not be writable by other users.
Hosted CI prepares trusted tool paths on a disposable runner and runs a blank
native-window smoke check before the full suite. DSP test files run sequentially
(`tooling/tests.json` sets `concurrency: 1`) so independent real-browser fixtures
do not compete for cold-start resources on small runners.

Obtain the verified platform package bundle from Core's build/release artifacts,
then run from this repository:

```sh
npm run bootstrap -- /absolute/build/platform-packages
Expand Down Expand Up @@ -33,10 +43,28 @@ tests require an explicit Core source export installed as a test-only package;
Core's `tooling/integration-package.js` installs that fixture and its platform
dependencies into this project's `node_modules`. Then run `npm run test:integration`.

Public CI requires the platform bundle URL and SHA-256 to be configured after
Core's first package release. Until then, the same checks run locally with the
bundle path above. Hosted checks never have production deployment credentials.
Public CI uses the reviewed dependency lock in `tooling/platform-dependencies.json`.
Before the first Core release, a separate job builds a bundle from an exact Core
commit. Production publication requires a published bundle URL and SHA-256.
Hosted checks never have production deployment credentials.

Each DSP keeps its own installed plugin code, SDK copies, settings, databases,
credentials and browser sessions. Downloading a release does not change them.
See `RELEASES.md` for independent releases and controlled activation.

## GitHub workflow

Use an isolated feature worktree from freshly fetched `origin/main`. Open a draft
PR after the first reviewed commit. Run the applicable local checks and wait for
GitHub checks on the exact PR commit before marking it ready. Report the PR link,
changes and verification in chat. Only merge after the owner explicitly approves;
recheck the approved head and required checks immediately before merging. Automatic
merge is disabled. Main requires PRs, up-to-date checks and resolved conversations.
The owner's chat approval is the human gate; GitHub does not interpret chat.

`tooling/workflow.py pr-details --repo OWNER/REPOSITORY --pr NUMBER` reports PR
facts. This helper is read-only; use normal git/gh commands for branches and PRs.
Keep multiline PR bodies in a file and pass `--body-file`.

The manual release workflow is a separate operation; never dispatch it as part of
ordinary development, merging, testing or retrying CI. See `RELEASES.md`.
52 changes: 47 additions & 5 deletions RELEASES.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,53 @@ must drain processes, snapshot private state, start selected code, verify health
and restore on failure. Recover interrupted operations explicitly before continuing.
The persistent state directory is private and is never included in source exports.

The GitHub feed, signed release provenance, repository configuration, permanent Dev
DSP deployment, separate Core preview, privileged activation hooks and owner Updates
UI are subsequent work. The development builder intentionally cannot publish a
production release. Release versions and the deployment baseline are not assigned
by local repository preparation.
The GitHub feed, permanent Dev DSP deployment, separate Core preview, privileged
activation hooks and owner Updates UI are subsequent work. The development builder
still produces development candidates. Publication uses the separate guarded
workflow below; no release version or production baseline is assigned by setup.

Core's legacy `core/installations/RELEASES.md` documents old native/OCI recovery formats.
It does not authorize or describe the new release workflow.

## GitHub publication

`.github/workflows/release.yml` accepts only a manual dispatch from `main`, the
owner-selected `version`, the verified 40-character `expected_main` commit, and
reviewed `changelog` text. Dispatch it only after the owner requests a release and
supplies its version. Pass inputs as structured JSON; do not interpolate notes
into shell commands. The release worker checks that main still matches and has a
successful main-push checks run. It checks again before publishing.

The SDK's `tooling/release-publication.js` packages a fresh verified development
build, records the source repository/commit and selected version, includes the
notes in its hashed inventory, and produces a deterministic tar archive plus
`release.json`, `release-notes.md` and `SHA256SUMS`. Core also publishes
`platform-packages.tar.gz`, bound by digest in the release manifest. Each asset
receives GitHub build provenance. Publishing verifies the signer workflow, main
source ref, source commit and hosted runner before creating the version tag and
a draft release. It downloads and compares the uploaded assets before publishing.

Existing tags or releases, including drafts, are never overwritten. A failed
publication can leave a reserved tag or draft; inspect it and its verified assets
before recovery. Do not repeatedly dispatch publication or delete history to make
CI pass. Reusing a published SDK/support-package/plugin version with different
installed bytes is rejected; bump that component in a reviewed PR first.

The release job has GitHub publication permissions only. There are no production
SSH credentials, service restarts, deployment hooks or DSP activation steps.
GitHub publication and installed-version verification are separate operations.
The first real publication is still pending an owner-selected version; do not
claim that upload/attestation publication has been exercised by the unit tests.

## Core dependency bootstrap

`tooling/platform-dependencies.json` is a reviewed dependency lock. Development CI
may use `developmentSource` with a full Core commit. A separate job builds that
commit's package bundle; the DSP job receives only the bundle and verifies its
checksum. Every run creates a fresh artifact, so development does not depend on an
expiring artifact from an earlier run. This never activates or publishes a product.

Production publication rejects the development source mode. After the first Core
release, pin that release's `platform-packages.tar.gz` URL and SHA-256 in a DSP PR.
`tooling/fetch-platform.py` verifies the downloaded archive before extraction.
Future SDK upgrades change the lock and declared package versions together.
Loading