Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# MySkills Agent Instructions

Version: 1.0.0
Last updated: 2026-06-19
Last updated: 2026-09-29

## Source Of Truth

Expand Down Expand Up @@ -42,6 +42,7 @@ Run the narrowest check that proves the change, then broaden when touching share
- Disposable Postgres integration gate: `TEST_DATABASE_URL=postgres://myskills_test:myskills_test@localhost:5432/myskills_test npm run test:postgres`
- Release artifact gate: `npm run release:artifacts`
- Production env preflight: `npm run check:prod-env -- --env-file .env.production`
- Full CI, release or CodeQL gate on Linux with Docker: `scripts/local-ci.sh verify|release-check|codeql` (see `docs/LOCAL_CI.md`)

`npm run test:postgres` must use a disposable database whose name includes `test` or `ci`; it resets that schema.

Expand Down
2 changes: 2 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ TEST_DATABASE_URL=postgres://myskills_test:myskills_test@localhost:5432/myskills

`TEST_DATABASE_URL` must point at a disposable database whose name includes `test` or `ci`; the test resets that schema.

To run the full CI, release or CodeQL gates on a Linux host with Docker, use `scripts/local-ci.sh`; see [Local CI](docs/LOCAL_CI.md).

## Pull Request Expectations

Every PR should include:
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ The [MCP guide](apps/mcp/README.md) covers the source-based stdio and HTTP serve
- [Libraries](docs/LIBRARIES.md) — save sources, review changes and curate shared skills.
- [Architecture](docs/ARCHITECTURE.md) — how the registry, API and clients fit together.
- [Contributing](CONTRIBUTING.md) — development setup and pull request guidance.
- [Local CI](docs/LOCAL_CI.md) — run the CI, release and CodeQL gates with `scripts/local-ci.sh`; maintainers dispatch the same entrypoint through their `local-ci` controller.
- [Support](SUPPORT.md) · [GitHub issues](https://github.com/jremick/myskills/issues) — questions, bugs and feature requests.
- [Security policy](SECURITY.md) — report vulnerabilities privately.
- [Changelog](CHANGELOG.md) — user-facing changes and upgrade notes.
Expand Down
2 changes: 1 addition & 1 deletion SUPPORT.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Supported for beta feedback:
- Fresh-clone setup and local self-hosting experiments.
- API, web, CLI, and MCP behavior documented in this repository.
- The example skill package under `examples/skills`.
- Release artifacts generated by the documented release workflow.
- Release artifacts generated by the documented release verification.

Not supported yet:

Expand Down
4 changes: 2 additions & 2 deletions docs/BUSINESS_SAFE_RELEASE_GOAL.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Business-Safe Production Release Goal

Version: 0.1.0-beta.4
Last updated: 2026-06-30
Last updated: 2026-09-29

## Goal

Expand Down Expand Up @@ -74,4 +74,4 @@ Turn the public beta into a business-safe, production-ready open-source release
- Fresh clone and production-like deploy rehearsals pass.
- Security review and threat model are refreshed after the production hardening work.
- All public docs describe the supported and unsupported production posture without stale alpha-only caveats.
- A release candidate tag is cut and the release workflow succeeds.
- A release candidate tag is cut and tagged release verification (`scripts/local-ci.sh release-check`) succeeds.
12 changes: 6 additions & 6 deletions docs/CODEX_CLOUD.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
# Codex Cloud Setup

Version: 0.1.0-beta.17
Last updated: 2026-07-13
Last updated: 2026-09-29

This runbook makes MySkills ready for subscription-based Codex cloud/web tasks while keeping implementation work on GitHub pull requests and avoiding API-billed GitHub Actions agents for now.

## Current Repo Contract

Codex cloud should mirror the existing GitHub CI contract:
Codex cloud should mirror the existing CI contract, defined portably by `scripts/local-ci.sh` (see [Local CI](LOCAL_CI.md)):

- CI installs dependencies with `npm ci`.
- CI runs `npm run check` for the general gate.
- CI runs `npm run test:postgres` in a separate job with disposable Postgres.
- Release verification runs the canonical `npm run release:verify` gate, then builds production Docker targets in the tag workflow.
- Release verification runs the canonical `npm run release:verify` gate, then builds production Docker targets (`scripts/local-ci.sh release-check`).

Do not add a GitHub Actions workflow that invokes a coding agent yet. Use Codex cloud/web to create branches and pull requests, then let the existing CI and human review gates decide whether to merge.
The full entrypoint needs Linux with Docker and is not expected to run inside a Codex cloud task. Do not add a GitHub Actions workflow that invokes a coding agent yet. Use Codex cloud/web to create branches and pull requests, then let the required checks and human review decide whether to merge.

## Codex Environment

Expand Down Expand Up @@ -45,10 +45,10 @@ Inspect the MySkills repository instructions and CI. Do not change runtime behav

Expected behavior:

- The agent reads `AGENTS.md`, `README.md`, `package.json`, and `.github/workflows/ci.yml`.
- The agent reads `AGENTS.md`, `README.md`, `package.json`, and `docs/LOCAL_CI.md`.
- The diff is documentation-only.
- No secrets, deployment variables, GitHub Actions agent workflows, or production deploy changes are added.
- The PR waits for existing GitHub CI and human approval before merge.
- The PR waits for the required checks and human approval before merge.

## Verification Commands For Agents

Expand Down
164 changes: 164 additions & 0 deletions docs/LOCAL_CI.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
# Local CI And Release Checks

`scripts/local-ci.sh` runs the same gates as the GitHub Actions workflows on a Linux host with
Docker. Contributors can use it before opening a pull request. An external runner can call it
after checkout and read its evidence. Release and merge instructions use its results. The GitHub
workflows stay in the repository as a parity reference during the migration. This entrypoint
does not report statuses or publish anything.

## Requirements

- Linux on amd64. The E2E Compose file pins MinIO to `linux/amd64`, so other architectures need
emulation and do not give equivalent evidence.
- Docker Engine with Compose v2 and BuildKit, reachable through the local socket.
- Git, Bash and network access to the npm registry, Docker Hub, GitHub release assets (MinIO
binaries) and the Playwright browser download host.
- Node.js 22 (at least 22.13) and Node.js 24, each with the npm version in `package.json`
`packageManager`. The script checks versions; it never installs npm globally.
- Chromium system libraries for Playwright. CI installs them with `--with-deps`; locally install
them once with `sudo npx playwright install-deps chromium`.

## Usage

Commit first. The script tests the checked-out commit in fresh clones and refuses a dirty tree.

```bash
export LOCAL_CI_RUN_ID="pr-123-$(date +%s)"
export LOCAL_CI_EVIDENCE_DIR="$HOME/local-ci-evidence/$LOCAL_CI_RUN_ID"
export LOCAL_CI_NODE22_BIN=/path/to/node-22/bin
export LOCAL_CI_NODE24_BIN=/path/to/node-24/bin
export LOCAL_CI_SOURCE_SHA="$(git rev-parse HEAD)"
scripts/local-ci.sh verify
scripts/local-ci.sh verify --job check-node22 # one job; the result is marked non-gating
```

Release verification needs the release tag at `HEAD` and a main ref that contains it:

```bash
git fetch origin main --tags
LOCAL_CI_RELEASE_TAG="v$(node -p 'require("./package.json").version')" scripts/local-ci.sh release-check
```

CodeQL needs an official CodeQL CLI bundle:

```bash
LOCAL_CI_CODEQL_BIN=/path/to/codeql/codeql scripts/local-ci.sh codeql
```

| Mode | Jobs |
|---|---|
| `verify` | `check-node22`, `postgres-node22`, `web-e2e-node22`, `check-node24`, `postgres-node24`, `web-e2e-node24`, `railway-images` |
| `release-check` | `release` |
| `codeql` | `codeql-javascript-typescript` |

## Inputs

| Variable | Rule |
|---|---|
| `LOCAL_CI_RUN_ID` | Required. 1-48 lowercase letters, digits or hyphens. It names every container, Compose project, image tag and the workspace. |
| `LOCAL_CI_EVIDENCE_DIR` | Required. Absolute, outside the source tree, not a symbolic link, and absent or empty. Every file in it is scanned and exported, so a populated directory is refused. |
| `LOCAL_CI_SOURCE_SHA` | Optional. Must equal `HEAD`. Without it the result is not gating. |
| `LOCAL_CI_NODE22_BIN`, `LOCAL_CI_NODE24_BIN` | Directories containing `node`, `npm` and `npx`. When unset, `node` on `PATH` is used only for its own major version. |
| `LOCAL_CI_WORK_DIR` | Optional parent for the per-run workspace. Defaults to the OS temporary directory. |
| `DOCKER_HOST`, `DOCKER_CONTEXT` | Jobs that use Docker need a local `unix://` endpoint; a remote endpoint is refused. Set at most one of the two, because `DOCKER_CONTEXT` overrides `DOCKER_HOST`. |
| `LOCAL_CI_RELEASE_TAG`, `LOCAL_CI_MAIN_REF` | `release-check` only. The tag must be `v<package version>` and point at `HEAD`. `HEAD` must be an ancestor of the main ref (default `refs/remotes/origin/main`). The script does not fetch. |
| `LOCAL_CI_CODEQL_BIN`, `LOCAL_CI_CODEQL_CATEGORY` | `codeql` only. The category defaults to `/language:javascript-typescript`. |
| `MYSKILLS_E2E_PORT`, `MYSKILLS_E2E_WEB_PORT`, `MYSKILLS_E2E_MAILPIT_PORT` | Optional loopback ports. Free ports are chosen when unset. |

Jobs receive an allowlisted environment (paths, locale, Docker endpoint, proxy and CA settings,
browser and npm caches) with `CI=true`. Tokens such as `GITHUB_TOKEN` or `NPM_TOKEN` are not
passed to jobs. Do not keep publishing credentials in the account that runs jobs: dependency
scripts and tests run with that account's files and its Docker access.

## Results

Exit status 0 means passed, 1 failed, 2 rejected before any work, and 128 plus the signal number
means cancelled. Read `result.json` rather than relying on the exit status. It is written
atomically and contains:

- `status`: `passed`, `failed`, `rejected` or `cancelled`. `passed` also requires complete
cleanup and clean evidence.
- `gating` and `gatingBlockers`: only a complete job set with a verified `LOCAL_CI_SOURCE_SHA` on a
Linux/amd64 host can gate a commit. Other hosts report `unsupported-host-platform`.
- `contexts`: for a complete `verify` run, the protected-branch contexts `check`, `web-e2e` and
`postgres-integration`. Partial runs report `null`.
- `jobs`: status, reason, steps, exit codes, timings and a hashed log for each job.
- `cleanup` and `artifacts`: cleanup outcome and the SHA-256 of every evidence file.

Other evidence: `logs/<job>.log`, `resources.json`, `environment.json`,
`browser-evidence/<job>/` (the reviewed summaries and screenshots from
`scripts/collect-browser-evidence.mjs`), `release/` (verified artifacts and
`verification.json`) and `codeql/` (SARIF and a summary).

Credential-shaped output is replaced with `[redacted]` in logs and fails the run. The full-stack
runner also redacts its generated credentials. Raw Playwright reports, traces and videos stay in
the job clone and are deleted with it.

## Isolation And Cleanup

Each job runs in its own clone of the pinned commit inside
`<work dir>/myskills-local-ci-<run id>`. Before that, the run ID is reserved by creating
`/var/tmp/myskills-local-ci-locks/<run id>` exclusively, whatever `TMPDIR` or the work directory is,
so two runs on one host cannot share a run ID. The lock directory must be owned by the running user
and not writable by others. The reservation covers only this host's Docker daemon, which is why
remote Docker endpoints are refused. A reservation is never taken over. The script records each container, Compose project and image in
`resources.json` before or as it creates it, and removes only those exact names. Compose cleanup
matches the exact `com.docker.compose.project` label. A container whose creation failed, for
example because of a name conflict, is never removed. The script never prunes and never matches
name prefixes. Shared npm, Playwright and Docker build caches are kept.

Each step runs in its own process group. `SIGTERM` stops the current step, cleans up and writes a
cancelled result; allow about 60 seconds. The reservation is released only after complete cleanup.
If cleanup fails, or after `SIGKILL`, the reservation stays and the run ID is refused. To recover,
remove the resources listed in that run's `resources.json`. Then remove
`/var/tmp/myskills-local-ci-locks/<run id>` only if its `owner.json` `owner` equals the
`run-id-reservation` entry's `owner` in the same `resources.json`.

## Maintainer Controller

Maintainers dispatch this entrypoint to a trusted Linux host with a private `local-ci` controller.
The controller snapshots the exact commit, runs the entrypoint, collects `result.json` and the
evidence, and reports the protected contexts. Contributors do not need it. Dispatch only trusted
changes. The syntax below is current; final paths and configuration may still change.

```bash
local-ci submit --app myskills --job verify --commit <sha>
local-ci submit --app myskills --job release-check --commit <sha> --tag v<version>
local-ci submit --app myskills --job codeql --commit <sha>
local-ci wait <run-id>
local-ci report --app myskills --commit <sha> # dry run; --execute posts statuses
local-ci sarif upload <run-id> # dry run; --execute uploads SARIF
```

## Mapping From GitHub Actions

| Workflow gate | Local equivalent | Difference |
|---|---|---|
| CI `Check / Node 22.x`, `Check / Node 24.x`: `npm ci`, `npm run check` | `check-node22`, `check-node24` | Node patch versions come from the supplied toolchains, not the latest `22.x`/`24.x`. The npm version is checked, not installed. |
| CI `check` aggregate | `contexts.check` (all seven `verify` jobs) | None. |
| CI `Web E2E / Node 22.x`, `Web E2E / Node 24.x` (15-minute timeout): browser install, workspace build, mocked browser run, evidence collection, full-stack run, evidence collection, evidence upload | `web-e2e-node22`, `web-e2e-node24` with the same steps, conditions and 15-minute limit | Browser system libraries come from host setup. Evidence is exported to the evidence directory instead of a 7-day artifact. A job without exported evidence fails, as with `if-no-files-found: error`. |
| CI `web-e2e` aggregate | `contexts["web-e2e"]` | None. |
| CI `Railway images`: `Dockerfile.api`, `Dockerfile.web`, `Dockerfile.backup` and two credential-free `--network none` smoke runs | `railway-images` | Builds use `--pull` and run-scoped tags. Image IDs are recorded and the images are removed afterwards. |
| CI `Postgres / Node 22.x`, `Postgres / Node 24.x` with a `postgres:17-alpine` service | `postgres-node22`, `postgres-node24` | Same image, credentials and health check on a random loopback port. |
| CI `postgres-integration` aggregate | `contexts["postgres-integration"]` | None. |
| Release `Verify tag and main ancestry` | `release-check` input validation | The runner supplies full history, the tag and a current main ref; the script does not fetch. |
| Release `postgres:17` service, `npm ci`, browser install, `npm run release:verify` with tag enforcement | `release` job steps | Same commands and environment. |
| Release image builds: root `Dockerfile` `api`, `mcp-http`, `web`; Railway API and web; backup image and smoke runs | `release` job `build-*` and `smoke-*` steps | Run-scoped tags; nothing is pushed. |
| Release `Upload release artifacts` | `verify-release-artifacts` step and `release/artifacts/` | Also checks that the artifact set is exact, that `SHA256SUMS` and the metadata match, and that the source archive rebuilds byte for byte from the pinned commit. |
| CodeQL `Analyze JavaScript and TypeScript`: `.github/codeql/codeql-config.yml` with `security-extended` | `codeql` mode | Uses the repository config plus `queries: - uses: security-extended`, and fails if an excluded query still reports. The number of findings does not gate; GitHub alert state, including dismissals, stays authoritative after upload. |

## Not Covered Here

The entrypoint does not provide these functions. Until cutover, GitHub Actions and repository
settings provide them; afterwards the maintainers' controller and GitHub settings do:

- Pull request, push, tag and weekly CodeQL triggers, and release concurrency.
- Reporting `local-ci/check`, `local-ci/web-e2e` and `local-ci/postgres-integration` after the matching branch-protection cutover. These distinct names replace the Actions contexts `check`, `web-e2e` and `postgres-integration`.
- SARIF upload and the code scanning merge rule.
- Artifact retention.
- Dependabot, which is not an Actions workflow.

The workflow files stay as the parity reference, so `scripts/check-structure.mjs` still requires
them and `scripts/check-prerelease.mjs` still checks their static contract. Remove those checks
only together with the files. The full-stack Compose run builds from cached base images without
`--pull`, so the host cache can differ from a fresh GitHub runner until it is refreshed.
6 changes: 3 additions & 3 deletions docs/RAILWAY_DEPLOYMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -500,11 +500,11 @@ curl --doh-url https://cloudflare-dns.com/dns-query https://api.myskills.sh/read

The current live project is intentionally manual but can be made easier without changing hosting providers:

1. Keep feature work on a branch and require GitHub CI to pass.
2. Merge or fast-forward the Railway-connected branch after the rendered checks pass. Verify required CI for the exact merged source before promotion, and capture a current database-and-artifact recovery point.
1. Keep feature work on a branch and require the protected checks (`local-ci/check`, `local-ci/web-e2e` and `local-ci/postgres-integration` after cutover) to pass; see [Local CI](LOCAL_CI.md).
2. Merge or fast-forward the Railway-connected branch after the rendered checks pass. Verify the required checks for the exact merged source before promotion, and capture a current database-and-artifact recovery point.
3. When changing artifact publication or cleanup coordination, remove incompatible API writers and cleanup workers before starting the replacement. For Libraries beta.8, drain beta.7 API instances and workers before accepting library writes; older code does not enforce private-attestation and library-binding guards. Follow the [Libraries rollback boundary](RELEASE.md#libraries-beta8-compatibility-boundary). Account for the resulting API interruption in the rollout plan.
4. Deploy `api` from the approved commit and wait for Railway success and direct `/ready` before uploading `web` from the same commit. The web proxy must start after the healthy API so it does not retain an address for a retiring private instance.
5. Compare direct API, web, and proxy `/version.json` with the approved source. Verify web health and same-origin `/api/health` and `/api/ready`.
6. Complete staging's real browser/CLI journey before production. After production promotion, verify HTML revalidation in an existing browser cache, existing-session auth, authorized private package delivery, anonymous denial, rendered package text and navigation, and recent logs. Use a fresh context for anonymous checks and preserve existing user sessions during verification. Use read requests for production checks; package access still writes its normal audit events.

The release workflow is intentionally verification-only and does not deploy Railway. Follow the staging, production approval, and rollback boundary in [Release Process](RELEASE.md). Any future deploy automation must use scoped project credentials, preserve a separate staging/user-test step, require explicit production approval, deploy API and web from the same commit in API-ready-then-web order, and report resulting deployment IDs plus direct and same-origin health/browser readback.
Release verification (`scripts/local-ci.sh release-check`, and the tag workflow while it remains) is intentionally verification-only and does not deploy Railway. Follow the staging, production approval, and rollback boundary in [Release Process](RELEASE.md). Any future deploy automation must use scoped project credentials, preserve a separate staging/user-test step, require explicit production approval, deploy API and web from the same commit in API-ready-then-web order, and report resulting deployment IDs plus direct and same-origin health/browser readback.
Loading
Loading