diff --git a/.claude/skills/openship-config/references/fields.md b/.claude/skills/openship-config/references/fields.md index 3f8cc6a7d..f49b67684 100644 --- a/.claude/skills/openship-config/references/fields.md +++ b/.claude/skills/openship-config/references/fields.md @@ -14,6 +14,7 @@ value. Validated by `openship config validate` (same parser the deploy uses). | `installCommand` | string | Dependency install command. | | `buildCommand` | string | Build command. | | `startCommand` | string | Production start command. | +| `releaseCommands` | string[] | Commands run ONCE per deploy, after the build and before the new version goes live (migrations, cache warms). Each runs with the start command's env; a non-zero exit fails the deploy and the previous version keeps serving. Laravel: `["php artisan migrate --force", "php artisan optimize", "php artisan storage:link", "php artisan reload"]`. Omit for no release phase (the default — nothing is auto-injected). Skipped with a logged warning on Openship Cloud, static sites and compose projects. | | `outputDirectory` | string | Build output dir (`dist`, `.next`, `build`, `out`, …). | | `buildImage` | string | Build Docker image (e.g. `node:22`). | | `productionPaths` | string[] | Paths shipped as the production artifact. | diff --git a/TODO.md b/TODO.md index 62a3f42ca..3223a00e2 100644 --- a/TODO.md +++ b/TODO.md @@ -382,23 +382,41 @@ the JS asset stage also landed; the storage section below has what's left of tho ### A generic release phase Commands that run ONCE per deploy, after build and before cutover, failing the -deploy on error. `queue:work`, `schedule:run`, `migrate --force`, `optimize` and -`storage:link` appear nowhere in the tree today, so migrations, scheduled tasks -and queued jobs silently never run. - -- [ ] Add release commands to the project + `openship.json`, snapshot them onto - the deployment, and run them from the deploy pipeline between build and - activate (`apps/api/src/modules/deployments/build-pipeline.ts` — the same - seam `deployConfig` is assembled in). -- [ ] Laravel's set for 13.x: `migrate --force`, `optimize` (config/events/routes/ - views), `storage:link`, and `reload` (13's umbrella for cycling long-running - services — supersedes `queue:restart` for deploys, also covers Reverb and - Octane). +deploy on error. + +**Shipped (v1).** `openship.json` `releaseCommands` (a LIST — a framework's +release set is several independent steps, each with its own log marker and its +own attributable failure) → `project.release_commands` → frozen on the +deployment snapshot → run in `executeServerDeploy` +(`apps/api/src/modules/deployments/build-pipeline.ts`) right after `deployConfig` +is assembled and before the first domain row or `runDeployPipeline`, so a failed +migration leaves the previous version running and routed. Docker runs each +command in a throwaway container off the new image (deploy env + volumes + +project network, no published port, no restart policy); bare runs it in the +staged release dir through a login shell. Ordering / fail-fast / skip live in +`deployments/release-phase.ts`. Nothing is auto-injected per framework. + +- [ ] Laravel's set for 13.x is DOCUMENTED, not injected: `migrate --force`, + `optimize` (config/events/routes/views), `storage:link`, and `reload` (13's + umbrella for cycling long-running services — supersedes `queue:restart` for + deploys, also covers Reverb and Octane). Auto-injection on detection is the + open question. +- [ ] No dashboard field yet — the phase is reachable through `openship.json`, + `POST /projects` / `PATCH /projects/:id` and `POST /:id/options` only. +- [ ] Compose/services projects skip it with a logged warning: a release command + there would have to name a SERVICE to run in, which v1 doesn't model. + Openship Cloud skips it too (no one-off execution primitive), as do static + deploys (no runtime). +- [ ] Bare runs the command BEFORE `linkPersistentPaths`, so a path that becomes a + `shared/` symlink is still a plain dir at release time. Migrating a DATABASE + is unaffected; a command that writes a file expected to survive the release + swap (a SQLite file under `storage/`) is not. +- [ ] A rollback/redeploy replays the TARGET release's frozen commands. Fine for + idempotent migrations, an open question for anything else. - [ ] Not the same thing as `#206` deploy hooks: those are an inbound trigger that STARTS a deploy; this runs DURING one. -- [ ] Until this exists, a stock SQLite Laravel app still needs its migrations run - by hand (the service terminal can do it) — a persistent volume stops data - LOSS, it doesn't bootstrap a schema. +- [ ] `queue:work` and `schedule:run` are still unexpressible — they're roles, not + release steps. See multi-role stacks below. ### Multi-role stacks diff --git a/apps/api/src/modules/deployments/build-pipeline.ts b/apps/api/src/modules/deployments/build-pipeline.ts index 6ce529485..4acfc6dbe 100644 --- a/apps/api/src/modules/deployments/build-pipeline.ts +++ b/apps/api/src/modules/deployments/build-pipeline.ts @@ -89,6 +89,7 @@ import { onFailure, onSuccess, onCancelled, reportPipelineError, setDeploymentSt import { auditPorts } from "./port-audit.service"; import { verifyDeployedContainers } from "./stability-audit.service"; import { resolveReadinessGate, runReadinessGate, type ResolvedReadinessGate } from "./readiness-gate"; +import { RELEASE_COMMAND_TIMEOUT_MS, resolveReleaseCommands, runReleasePhase } from "./release-phase"; import { auditStaticOutput, staticOutputTargets } from "./output-audit.service"; import { createBuildConfig } from "./build-config"; import { pinnedAppImage, pinnedStaticDir, snapshotNeedsGitSource } from "./pinned-artifacts"; @@ -913,6 +914,15 @@ async function executeBuildAndDeploy(project: Project, dep: Deployment, buildSes }); } + // A compose project's release phase would have to name a SERVICE to run in + // (there is no single app image), which v1 doesn't model — so say so + // instead of dropping the declaration on the floor. + await runReleasePhase({ + commands: resolveReleaseCommands(snapshot.releaseCommands), + unsupportedReason: "Release commands aren't supported on services/compose projects yet", + log: (message, level) => logger.log(`${message}\n`, level), + }); + // Clone-on-server for compose: open one repo-pinned relay for the whole // fan-out (all services share the same repo), thread its helper path into // every service buildConfig, and close it once the pipeline settles. @@ -1128,6 +1138,14 @@ async function executeStaticEdgeDeploy( ): Promise { const { ctx, project, dep, snapshot, buildSessionId, routeState, buildResult, envMap, prodResources, logger } = phase; + // Pages is a file upload, not a workload — there is nothing to run a command + // in. Declared commands are named in the log rather than silently dropped. + await runReleasePhase({ + commands: resolveReleaseCommands(snapshot.releaseCommands), + unsupportedReason: "A static edge (Pages) deploy has no runtime to run release commands in", + log: (message, level) => logger.log(`${message}\n`, level), + }); + logger.step("deploy", "running", "Deploying to edge (static)..."); const staticResult = await runtime.deployStatic({ @@ -1735,8 +1753,44 @@ async function executeServerDeploy(phase: DeployPhaseInputs): Promise { // Bare uses this to hard-link identical files across releases. // Other runtimes ignore it. previousDeploymentId: project.activeDeploymentId ?? undefined, + // Frozen on the snapshot, so a redeploy of an old deployment replays the + // commands THAT release declared. Absent on pre-release-phase rows. + releaseCommands: snapshot.releaseCommands, }; + // ── Release phase: after the build, before ANY cutover ──────────────────── + // Placed here on purpose. Nothing below has touched the live deployment yet — + // the previous version is running and routed, no domain row has been created, + // runDeployPipeline hasn't been entered — so a failed migration aborts with the + // old version still serving and nothing to roll back. It is the same boundary + // for every runtime this path serves: docker/bare/static all activate inside + // runDeployPipeline's `activate` step, which is strictly after this. + const releaseCommands = resolveReleaseCommands(deployConfig.releaseCommands); + if (releaseCommands.length > 0) { + try { + await runReleasePhase({ + commands: releaseCommands, + // A static file-serve has no process and no image to run a command in, + // and cloud has no one-off execution primitive at all — both fall through + // to the logged skip rather than pretending the commands ran. + run: + !isStaticFileServe && runtime.supports("releaseCommand") && runtime.runReleaseCommand + ? (command) => + runtime.runReleaseCommand!(deployConfig, command, logger.callback, { + timeoutMs: RELEASE_COMMAND_TIMEOUT_MS, + }) + : undefined, + unsupportedReason: isStaticFileServe + ? "A static site has no runtime to run release commands in" + : `The "${runtime.name}" runtime can't run release commands yet`, + log: (message, level) => logger.log(`${message}\n`, level), + }); + } catch (err) { + await onFailure(ctx, safeErrorMessage(err), buildResult.durationMs); + return; + } + } + // Resolve the previous deployment + its runtime so we can deactivate it cleanly. const prevDep = project.activeDeploymentId ? await repos.deployment.findById(project.activeDeploymentId) diff --git a/apps/api/src/modules/deployments/build.service.ts b/apps/api/src/modules/deployments/build.service.ts index 2a915124f..84a8beb61 100644 --- a/apps/api/src/modules/deployments/build.service.ts +++ b/apps/api/src/modules/deployments/build.service.ts @@ -186,6 +186,14 @@ export interface DeploymentConfigSnapshot { rootDirectory: string; port: number; startCommand: string; + /** + * Commands run once between this deployment's build and its cutover. Frozen + * here for the same reason `volumes` is: a redeploy of an OLD deployment must + * replay the commands THAT release declared, not what the project says today. + * Absent on every row written before the field existed — and absent means no + * release phase, so those redeploy exactly as they always did. + */ + releaseCommands?: string[]; resources: ResourceConfig | null; buildResources: ResourceConfig | null; /** Whether the project needs a running server (false = static, deploy via Pages) */ @@ -377,6 +385,11 @@ export function buildConfigSnapshot( rootDirectory: project.rootDirectory || "", port: project.port ?? 3000, startCommand: project.startCommand!, + // Only carried when the project declared some: an absent key keeps the + // snapshot byte-identical to a pre-release-phase one. + ...(Array.isArray(project.releaseCommands) && project.releaseCommands.length > 0 + ? { releaseCommands: project.releaseCommands as string[] } + : {}), resources: (project.resources as ResourceConfig) || null, buildResources: (project.buildResources as ResourceConfig) || null, hasServer: project.hasServer ?? !!project.startCommand?.trim(), diff --git a/apps/api/src/modules/deployments/prepare.service.ts b/apps/api/src/modules/deployments/prepare.service.ts index 97ac19ab1..a96d91438 100644 --- a/apps/api/src/modules/deployments/prepare.service.ts +++ b/apps/api/src/modules/deployments/prepare.service.ts @@ -248,6 +248,11 @@ export interface ProjectInfo { * the pipeline does when the project has no `readiness`. */ readiness?: OpenshipReadiness; + /** + * Declared release commands — run once per deploy between build and cutover. + * Absent means no release phase, which is the default for every project. + */ + releaseCommands?: string[]; } /** A `domains[]` entry normalized to the `CreateProjectBody.publicEndpoints` shape. */ @@ -436,6 +441,8 @@ function applyOpenshipOverlay(info: ProjectInfo, config: OpenshipConfig | undefi } if (config.resources) info.resources = config.resources; if (config.readiness) info.readiness = config.readiness; + // Declared `[]` is meaningful (release phase off), so test for presence. + if (config.releaseCommands) info.releaseCommands = config.releaseCommands; // Declared compose services replace detection: the project IS a services // project. runtimeMode="docker" then falls out of buildProductionProjectInput's @@ -500,6 +507,7 @@ export function projectInfoToScanResponse(result: ProjectInfo) { ...(result.publicEndpoints && { publicEndpoints: result.publicEndpoints }), ...(result.resources && { resources: result.resources }), ...(result.readiness && { readiness: result.readiness }), + ...(result.releaseCommands && { releaseCommands: result.releaseCommands }), ...(result.rootEnv && Object.keys(result.rootEnv).length > 0 && { rootEnv: maskEnv(result.rootEnv) }), ...(result.routing && { routing: result.routing }), ...(result.monorepoWorkspace && { monorepoWorkspace: result.monorepoWorkspace }), diff --git a/apps/api/src/modules/deployments/release-phase.ts b/apps/api/src/modules/deployments/release-phase.ts new file mode 100644 index 000000000..527c7b332 --- /dev/null +++ b/apps/api/src/modules/deployments/release-phase.ts @@ -0,0 +1,92 @@ +/** + * The deploy-time release phase — commands that run ONCE per deploy, after the + * build and before the new version is activated. + * + * OFF IS THE DEFAULT: a project with no `releaseCommands` runs nothing here and + * its deploy is byte-identical to what it was before this module existed. What + * it buys the projects that do declare them is the thing an app stack could not + * express at all until now — an app declares exactly one start command, so + * `php artisan migrate --force`, `rails db:migrate` and `manage.py migrate` + * never ran, and a stock Laravel app's schema had to be bootstrapped by hand + * through the service terminal. + * + * Unlike the readiness gate, a failure here has no warn/fail choice: a release + * command exists to be a precondition of serving the new version, so a non-zero + * exit FAILS the deploy. That is the safe direction — the previous version is + * still running and still routed at this point in the pipeline, so a failed + * migration leaves the old app serving instead of cutting over to one that + * cannot talk to its database. + */ + +import { SYSTEM, safeErrorMessage } from "@repo/core"; + +/** + * Normalize a project's / snapshot's declared commands into the list the phase + * actually runs. Blank entries are dropped (a settings form's empty row), so + * `[" "]` resolves to "nothing to run" rather than a shell no-op that still + * costs a container. + */ +export function resolveReleaseCommands(declared: string[] | null | undefined): string[] { + if (!Array.isArray(declared)) return []; + return declared.map((command) => command.trim()).filter((command) => command.length > 0); +} + +/** + * Run the release phase. + * + * `run` is injected — it's `runtime.runReleaseCommand` bound to the deploy + * config at the call site — so the ordering, the log markers, the fail-fast and + * the unsupported-runtime skip are all testable without a runtime, a container + * or a daemon. + * + * Returns nothing and throws on the first failure: the caller turns that into a + * failed deployment. `run: undefined` means the runtime has no such primitive + * (cloud) — the phase then logs a warning naming what it skipped and returns, + * because refusing the deploy outright would break every existing cloud project + * the moment someone added a release command for their self-hosted target. + */ +export async function runReleasePhase(opts: { + commands: string[]; + /** Runs ONE command; rejects with the command's output on a non-zero exit. */ + run?: (command: string) => Promise; + /** Why `run` is missing — logged so a skipped migration is never silent. */ + unsupportedReason?: string; + log: (message: string, level?: "info" | "warn" | "error") => void; +}): Promise { + const { commands, run, unsupportedReason, log } = opts; + if (commands.length === 0) return; + + if (!run) { + log( + `${unsupportedReason ?? "This runtime cannot run release commands"} — skipping ` + + `${commands.length} release command${commands.length === 1 ? "" : "s"}: ` + + `${commands.join(", ")}. Run them by hand before this version serves traffic.`, + "warn", + ); + return; + } + + log( + `Release phase: running ${commands.length} command${commands.length === 1 ? "" : "s"} ` + + `before this version goes live (a failure here fails the deploy and leaves the ` + + `previous version serving).`, + ); + + for (const [index, command] of commands.entries()) { + const marker = `[release ${index + 1}/${commands.length}]`; + log(`${marker} $ ${command}`); + try { + await run(command); + } catch (err) { + const message = safeErrorMessage(err); + log(`${marker} failed: ${message}`, "error"); + throw new Error(`Release command failed: ${command}\n${message}`); + } + log(`${marker} done.`); + } + + log("Release phase complete."); +} + +/** Per-command budget. Re-exported so the pipeline and its tests name one number. */ +export const RELEASE_COMMAND_TIMEOUT_MS = SYSTEM.DEPLOYMENTS.RELEASE_COMMAND_TIMEOUT_MS; diff --git a/apps/api/src/modules/projects/project-crud.service.ts b/apps/api/src/modules/projects/project-crud.service.ts index 033b1b5fa..a4d127512 100644 --- a/apps/api/src/modules/projects/project-crud.service.ts +++ b/apps/api/src/modules/projects/project-crud.service.ts @@ -501,6 +501,9 @@ function buildProductionProjectInput( rootDirectory: data.rootDirectory, composePath: normalizeComposePath(data.composePath), startCommand: data.startCommand, + // undefined (not declared) leaves the column NULL = no release phase, which + // is what every project did before the phase existed. + releaseCommands: data.releaseCommands ?? null, buildImage: data.buildImage, productionMode: workload.productionMode, port: data.port ?? 3000, @@ -1081,6 +1084,7 @@ export async function ensureProject( if (data.rootDirectory !== undefined) update.rootDirectory = data.rootDirectory; if (data.composePath !== undefined) update.composePath = normalizeComposePath(data.composePath); if (data.startCommand !== undefined) update.startCommand = data.startCommand; + if (data.releaseCommands !== undefined) update.releaseCommands = data.releaseCommands; if (data.buildImage !== undefined) update.buildImage = data.buildImage; if (data.port !== undefined) update.port = data.port; // Workload axis (workloadType / hasServer / productionMode) — one choke @@ -1650,6 +1654,7 @@ export async function createProjectEnvironment( rootDirectory: base.rootDirectory, composePath: base.composePath, startCommand: base.startCommand, + releaseCommands: base.releaseCommands, buildImage: base.buildImage, productionMode: base.productionMode, port: base.port, @@ -2121,6 +2126,15 @@ export async function updateOptions( update.composePath = normalizeComposePath(composePath); } if (options.startCommand !== undefined) update.startCommand = options.startCommand; + // Array-or-null only, same rule as `volumes`: a bare string here would run as + // one nonsense command between build and cutover, and a failure there fails the + // deploy. null/[] turns the release phase off. + if (options.releaseCommands !== undefined) { + if (options.releaseCommands !== null && !Array.isArray(options.releaseCommands)) { + throw new ValidationError("releaseCommands must be an array of commands, or null"); + } + update.releaseCommands = options.releaseCommands; + } if (options.productionPort !== undefined) update.port = options.productionPort; if (options.packageManager !== undefined) update.packageManager = options.packageManager; if (options.buildImage !== undefined) update.buildImage = options.buildImage; diff --git a/apps/api/src/modules/projects/project.schema.ts b/apps/api/src/modules/projects/project.schema.ts index 2326d0148..cd50a5818 100644 --- a/apps/api/src/modules/projects/project.schema.ts +++ b/apps/api/src/modules/projects/project.schema.ts @@ -364,6 +364,15 @@ export const CreateProjectBody = Type.Object({ */ composePath: Type.Optional(Type.String({ maxLength: 300 })), startCommand: Type.Optional(Type.String({ maxLength: 500 })), + /** + * Commands run ONCE per deploy, between the build and the cutover — migrations, + * cache warms. A non-zero exit fails the deploy. Omit to keep the current value; + * send `[]` (or null) to turn the release phase off. Mirrors `openship.json`'s + * `releaseCommands`. + */ + releaseCommands: Type.Optional( + Type.Union([Type.Null(), Type.Array(Type.String({ maxLength: 1000 }), { maxItems: 20 })]), + ), buildImage: Type.Optional(Type.String({ maxLength: 200 })), productionMode: Type.Optional( Type.Union([Type.Literal("host"), Type.Literal("static"), Type.Literal("standalone")]), @@ -627,6 +636,10 @@ export const SetOptionsBody = Type.Object( /** Compose file location; `null` clears it and restores root detection. */ composePath: Type.Optional(Type.Union([Type.String({ maxLength: 300 }), Type.Null()])), startCommand: Type.Optional(Type.String()), + /** Deploy-time release commands; `null`/`[]` turns the release phase off. */ + releaseCommands: Type.Optional( + Type.Union([Type.Array(Type.String({ maxLength: 1000 }), { maxItems: 20 }), Type.Null()]), + ), productionPort: Type.Optional(Type.Union([Type.Number(), Type.String()])), packageManager: Type.Optional(Type.String()), buildImage: Type.Optional(Type.String()), diff --git a/apps/api/test/modules/deployments/build.service.test.ts b/apps/api/test/modules/deployments/build.service.test.ts index 36e561147..7b8f79d4f 100644 --- a/apps/api/test/modules/deployments/build.service.test.ts +++ b/apps/api/test/modules/deployments/build.service.test.ts @@ -84,6 +84,7 @@ vi.mock("../../../src/modules/deployments/smart-route", () => ({ })); import { + buildConfigSnapshot, requestBuildAccess, resolveSnapshotTarget, triggerDeployment, @@ -253,6 +254,33 @@ describe("resolveSnapshotTarget", () => { }); }); +/** + * Release commands are FROZEN onto the deployment, for the same reason `volumes` + * is: a redeploy of an old deployment must replay the commands that release + * declared, not whatever the project says today. The absent case matters just as + * much — the key must not appear at all, so a snapshot from a project with no + * release phase is byte-identical to one written before the field existed (and a + * redeploy of a pre-existing deployment stays a no-op). + */ +describe("buildConfigSnapshot — release commands", () => { + it("freezes the project's declared commands onto the snapshot", () => { + const snapshot = buildConfigSnapshot( + baseProject({ releaseCommands: ["php artisan migrate --force", "php artisan optimize"] }) as any, + ); + expect(snapshot.releaseCommands).toEqual([ + "php artisan migrate --force", + "php artisan optimize", + ]); + }); + + it("omits the key entirely for null, [] and a project that never declared any", () => { + for (const releaseCommands of [null, [], undefined]) { + const snapshot = buildConfigSnapshot(baseProject({ releaseCommands }) as any); + expect("releaseCommands" in snapshot).toBe(false); + } + }); +}); + describe("triggerDeployment", () => { beforeEach(() => { vi.clearAllMocks(); diff --git a/apps/api/test/modules/deployments/release-phase-wiring.test.ts b/apps/api/test/modules/deployments/release-phase-wiring.test.ts new file mode 100644 index 000000000..004ee414e --- /dev/null +++ b/apps/api/test/modules/deployments/release-phase-wiring.test.ts @@ -0,0 +1,72 @@ +import { readFileSync } from "node:fs"; +import { describe, expect, it } from "vitest"; + +/** + * WHERE the release phase sits in the deploy pipeline is the safety property, + * and it is not observable from `release-phase.ts` alone: that module runs + * commands and throws, but only its call site decides whether a failure costs + * the user their running app. + * + * The phase must run after the build has produced an artifact and BEFORE + * anything cuts over — before any domain row is created, before + * `runDeployPipeline` (whose `activate` step is where every runtime on this path + * starts the new workload). Then a failed migration aborts with the previous + * version still running and still routed, and with nothing to roll back. + * + * `executeServerDeploy` is a private function inside a 2,000-line module with a + * platform, a runtime and a live DB behind it, so this pins the wiring at the + * source level — the same approach `test/lib/proxy-settings-wiring.test.ts` + * takes for route payloads. The behaviour of the phase itself (ordering, + * fail-fast, skips) is covered in `release-phase.test.ts`. + */ +const PIPELINE = readFileSync( + new URL("../../../src/modules/deployments/build-pipeline.ts", import.meta.url), + "utf8", +); + +/** Index of `needle` inside `executeServerDeploy`, or -1. */ +function atInServerDeploy(needle: string): number { + const start = PIPELINE.indexOf("async function executeServerDeploy("); + expect(start, "executeServerDeploy was renamed — re-anchor this test").toBeGreaterThan(-1); + const found = PIPELINE.indexOf(needle, start); + return found === -1 ? -1 : found - start; +} + +describe("release phase — pipeline wiring", () => { + it("runs after the deploy config is assembled and before runDeployPipeline", () => { + const config = atInServerDeploy("const deployConfig: DeployConfig = {"); + const release = atInServerDeploy("await runReleasePhase({"); + const pipeline = atInServerDeploy("await runDeployPipeline("); + expect(config).toBeGreaterThan(-1); + expect(release).toBeGreaterThan(config); + expect(pipeline).toBeGreaterThan(release); + }); + + // A deploy that fails in the release phase must not leave orphan domain rows + // behind — which is only true while the phase runs before they are created. + it("runs before any domain record is created", () => { + const release = atInServerDeploy("await runReleasePhase({"); + const domains = atInServerDeploy("await ensureRouteDomainRecord({"); + expect(release).toBeGreaterThan(-1); + expect(domains).toBeGreaterThan(release); + }); + + // The failure path: report the deploy failed and RETURN. Falling through would + // activate the new version anyway, which is the exact outcome the phase exists + // to prevent. + it("fails the deployment and returns instead of continuing to activate", () => { + const release = atInServerDeploy("await runReleasePhase({"); + const tail = PIPELINE.slice( + PIPELINE.indexOf("async function executeServerDeploy(") + release, + ); + const catchBlock = tail.slice(tail.indexOf("} catch (err) {"), tail.indexOf("// Resolve the previous deployment")); + expect(catchBlock).toContain("await onFailure(ctx,"); + expect(catchBlock).toContain("return;"); + }); + + // Snapshotted, not read live: a redeploy of an old deployment replays the + // commands that release declared. + it("takes its commands from the deployment snapshot, not the project row", () => { + expect(PIPELINE).toContain("releaseCommands: snapshot.releaseCommands"); + }); +}); diff --git a/apps/api/test/modules/deployments/release-phase.test.ts b/apps/api/test/modules/deployments/release-phase.test.ts new file mode 100644 index 000000000..5fceaa16f --- /dev/null +++ b/apps/api/test/modules/deployments/release-phase.test.ts @@ -0,0 +1,106 @@ +import { describe, it, expect, vi } from "vitest"; +import { + RELEASE_COMMAND_TIMEOUT_MS, + resolveReleaseCommands, + runReleasePhase, +} from "../../../src/modules/deployments/release-phase"; + +/** + * The release phase is what finally lets a deploy run `php artisan migrate + * --force` / `rails db:migrate` — commands an app stack could not express, + * because it declares exactly one start command and nothing else. + * + * Two properties carry the feature and are asserted here rather than in the + * 2,000-line pipeline: a project that declares NOTHING must deploy exactly as it + * did before the phase existed, and a command that fails must fail the deploy + * BEFORE anything cuts over — so the previous version keeps serving instead of a + * new one coming up against a schema it can't use. + */ +describe("resolveReleaseCommands", () => { + it("is empty for null, undefined, [] and blank entries", () => { + for (const value of [null, undefined, [], ["", " "]]) { + expect(resolveReleaseCommands(value)).toEqual([]); + } + }); + + it("keeps declared order and trims", () => { + expect( + resolveReleaseCommands([" php artisan migrate --force ", "php artisan optimize"]), + ).toEqual(["php artisan migrate --force", "php artisan optimize"]); + }); +}); + +describe("runReleasePhase", () => { + // The absent-field contract: no runner is even consulted, and not one line is + // logged — a deploy with no release commands is byte-identical to before. + it("does nothing at all when no commands are declared", async () => { + const run = vi.fn(); + const log = vi.fn(); + await runReleasePhase({ commands: [], run, log }); + expect(run).not.toHaveBeenCalled(); + expect(log).not.toHaveBeenCalled(); + }); + + it("runs every command in order and logs a marker for each", async () => { + const ran: string[] = []; + const lines: string[] = []; + await runReleasePhase({ + commands: ["php artisan migrate --force", "php artisan optimize"], + run: async (command) => { + ran.push(command); + }, + log: (message) => lines.push(message), + }); + expect(ran).toEqual(["php artisan migrate --force", "php artisan optimize"]); + expect(lines.join("\n")).toContain("[release 1/2] $ php artisan migrate --force"); + expect(lines.join("\n")).toContain("[release 2/2] $ php artisan optimize"); + expect(lines.at(-1)).toBe("Release phase complete."); + }); + + // The whole point of the phase: a failing migration must stop the deploy, and + // must stop it BEFORE the commands after it run. + it("fails the deploy on a non-zero exit and does not run later commands", async () => { + const ran: string[] = []; + const lines: Array<[string, string | undefined]> = []; + const failing = runReleasePhase({ + commands: ["php artisan migrate --force", "php artisan optimize"], + run: async (command) => { + ran.push(command); + throw new Error("SQLSTATE[42S02]: Base table or view not found"); + }, + log: (message, level) => lines.push([message, level]), + }); + await expect(failing).rejects.toThrow(/Release command failed: php artisan migrate --force/); + // The command's own output has to reach the operator, or a failed deploy is + // just "deploy failed" with the reason buried on a host they can't see. + await expect(failing).rejects.toThrow(/SQLSTATE\[42S02\]/); + expect(ran).toEqual(["php artisan migrate --force"]); + expect(lines.some(([message, level]) => level === "error" && message.includes("[release 1/2]"))).toBe(true); + }); + + // Cloud has no one-off execution primitive (and a static site has no runtime at + // all). Skipping is the deliberate choice — but it must be LOUD, naming the + // commands, because a silently-unrun migration is the failure mode this whole + // feature exists to remove. + it("warns and skips, naming the commands, when the runtime can't run them", async () => { + const lines: Array<[string, string | undefined]> = []; + await runReleasePhase({ + commands: ["php artisan migrate --force"], + run: undefined, + unsupportedReason: 'The "cloud" runtime can\'t run release commands yet', + log: (message, level) => lines.push([message, level]), + }); + expect(lines).toHaveLength(1); + const [message, level] = lines[0]!; + expect(level).toBe("warn"); + expect(message).toContain('The "cloud" runtime can\'t run release commands yet'); + expect(message).toContain("php artisan migrate --force"); + }); + + // Bounded by construction: an unbounded release command would hold a deploy + // open forever with the old version still serving. + it("exports a bounded per-command budget", () => { + expect(RELEASE_COMMAND_TIMEOUT_MS).toBeGreaterThan(0); + expect(RELEASE_COMMAND_TIMEOUT_MS).toBeLessThanOrEqual(30 * 60 * 1000); + }); +}); diff --git a/apps/web/content/docs/guides/openship-json.mdx b/apps/web/content/docs/guides/openship-json.mdx index 9b5a82e0b..99d167e8f 100644 --- a/apps/web/content/docs/guides/openship-json.mdx +++ b/apps/web/content/docs/guides/openship-json.mdx @@ -43,6 +43,10 @@ A typical server app: - **Secrets** — wrap the value in `{ "value": "…", "secret": true }` so it's encrypted at rest. - **Custom domain** — a dotted hostname is treated as custom; a bare label is a free subdomain. +- **Migrations** — add + [`releaseCommands`](/docs/reference/openship-json#release-commands) (e.g. + `["php artisan migrate --force"]`) to run them once per deploy, after the build and before + the new version goes live. A failure there fails the deploy and the old version keeps serving. - **Leave out** anything detection already gets right (most `installCommand`s, `outputDirectory` for known frameworks, etc.). diff --git a/apps/web/content/docs/reference/openship-json.mdx b/apps/web/content/docs/reference/openship-json.mdx index f9ef62130..ec608d1be 100644 --- a/apps/web/content/docs/reference/openship-json.mdx +++ b/apps/web/content/docs/reference/openship-json.mdx @@ -46,6 +46,7 @@ It's **JSON, not JSONC** — no comments, no trailing commas. installCommand: { type: 'string', description: 'Command that installs dependencies.' }, buildCommand: { type: 'string', description: 'Command that builds the app.' }, startCommand: { type: 'string', description: 'Command that starts the server in production.' }, + releaseCommands: { type: 'string[]', description: 'Commands run once per deploy, after the build and before the new version goes live. A failure fails the deploy. See Release commands.' }, outputDirectory: { type: 'string', description: 'Directory the build writes to (dist, .next, build, out).' }, buildImage: { type: 'string', description: 'Docker image used for the build (e.g. "node:22").' }, productionPaths: { type: 'string[]', description: 'Paths shipped to the runtime as the production artifact.' }, @@ -63,6 +64,44 @@ It's **JSON, not JSONC** — no comments, no trailing commas. }} /> +## Release commands + +`releaseCommands` runs once per deploy, **after the build and before the new version goes +live** — the place migrations belong. Each command runs in order, its output streams into +the deploy log, and a non-zero exit **fails the deploy**: nothing cuts over, and the +version that was already serving keeps serving. + +```json +{ + "releaseCommands": [ + "php artisan migrate --force", + "php artisan optimize", + "php artisan storage:link", + "php artisan reload" + ] +} +``` + +Nothing is injected for you — a framework's release set is a choice, so an app with no +`releaseCommands` runs no release phase at all. The equivalents elsewhere: + +| Stack | Typical release commands | +| --- | --- | +| Laravel | `php artisan migrate --force`, `php artisan optimize`, `php artisan storage:link`, `php artisan reload` | +| Rails | `bundle exec rails db:migrate` | +| Django | `python manage.py migrate --noinput`, `python manage.py collectstatic --noinput` | + +Each command gets the same environment variables as your start command, and a bounded +10-minute budget. On the Docker runtime it runs in a throwaway container built from the new +image, with the project's volumes mounted; on the bare runtime it runs in the staged +release directory. Openship Cloud has no one-off execution primitive, and a static site has +no runtime at all — on those the commands are reported in the deploy log and skipped rather +than silently dropped. + + +This runs **during** a deploy. It is not an inbound trigger that starts one. + + ## Persistent storage Everything a container writes is discarded when the next version replaces it. `volumes` @@ -252,6 +291,7 @@ dashboard for now. |---|---| | `framework`, `packageManager`, `*Command`, `outputDirectory`, `buildImage`, `productionPaths`, `rootDirectory` | Build settings | | `port`, `productionMode`, `runtime` | Runtime settings (`hasServer`, runtime isolation) | +| `releaseCommands` | Project release commands (`PATCH /projects/:id` → `releaseCommands`) | | `volumes` | Configuration → Persistent storage | | `env` | Environment variables (seeded as editable rows) | | `domains` | Public endpoints / custom domains | diff --git a/apps/web/public/openship.schema.json b/apps/web/public/openship.schema.json index 16aa79bcc..eb60ea55f 100644 --- a/apps/web/public/openship.schema.json +++ b/apps/web/public/openship.schema.json @@ -27,6 +27,11 @@ "installCommand": { "description": "Command that installs dependencies.", "type": "string" }, "buildCommand": { "description": "Command that builds the app.", "type": "string" }, "startCommand": { "description": "Command that starts the server in production.", "type": "string" }, + "releaseCommands": { + "description": "Commands run ONCE per deploy, after the build and before the new version goes live — migrations, cache warms. A non-zero exit fails the deploy and the previous version keeps serving. Laravel: [\"php artisan migrate --force\", \"php artisan optimize\", \"php artisan storage:link\", \"php artisan reload\"]. Omit for no release phase (the default).", + "type": "array", + "items": { "type": "string" } + }, "outputDirectory": { "description": "Directory the build writes to (e.g. \".next\", \"dist\").", "type": "string" diff --git a/packages/adapters/src/runtime/bare-release-command.test.ts b/packages/adapters/src/runtime/bare-release-command.test.ts new file mode 100644 index 000000000..5565fe5bd --- /dev/null +++ b/packages/adapters/src/runtime/bare-release-command.test.ts @@ -0,0 +1,99 @@ +import { describe, expect, it, vi } from "vitest"; +import { BareRuntime } from "./bare"; +import type { CommandExecutor, DeployConfig } from "../types"; + +/** + * A bare release command runs in the STAGED artifact directory — the tree + * `deploy` is about to promote — with the deploy's env exported, so a migration + * sees the code it is migrating for and resolves its DSN exactly as the app + * will. It runs BEFORE anything is promoted or started, so a non-zero exit must + * surface as a throw: that is what fails the deploy while the previous release + * is still the one serving. + */ +function makeExecutor(result: { code: number; output: string }) { + const commands: string[] = []; + const executor = { + exec: vi.fn(async () => ""), + streamExec: vi.fn(async (command: string) => { + commands.push(command); + return result; + }), + writeFile: vi.fn(async () => {}), + readFile: vi.fn(async () => ""), + exists: vi.fn(async () => false), + mkdir: vi.fn(async () => {}), + rm: vi.fn(async () => {}), + dispose: vi.fn(async () => {}), + } as unknown as CommandExecutor; + return { executor, commands }; +} + +/** imageRef = the staged build directory, which is what bare hands `deploy`. */ +function config(): DeployConfig { + return { + projectId: "proj_1", + deploymentId: "dep_1", + buildSessionId: "bs_1", + imageRef: "/opt/openship/.builds/bs_1", + environment: "production", + port: 8000, + envVars: { DATABASE_URL: "postgres://u:p@db/app", "not-an-ident": "x" }, + resources: { cpuCores: 0, memoryMb: 0, diskMb: 0 }, + } as unknown as DeployConfig; +} + +describe("BareRuntime.runReleaseCommand", () => { + it("declares the capability", () => { + expect(new BareRuntime({ executor: makeExecutor({ code: 0, output: "" }).executor }).supports("releaseCommand")).toBe(true); + }); + + it("runs in the staged release dir with the start command's env", async () => { + const { executor, commands } = makeExecutor({ code: 0, output: "" }); + const runtime = new BareRuntime({ executor }); + await runtime.runReleaseCommand(config(), "php artisan migrate --force", () => {}); + + expect(commands).toHaveLength(1); + const command = commands[0]!; + expect(command).toContain("cd '/opt/openship/.builds/bs_1' && php artisan migrate --force"); + expect(command).toContain("export DATABASE_URL='postgres://u:p@db/app'"); + // PORT/NODE_ENV are part of the start command's env, so they're part of this one. + expect(command).toContain("export PORT='8000'"); + expect(command).toContain("export NODE_ENV='production'"); + // A key that isn't a shell identifier would break the export prefix outright. + expect(command).not.toContain("not-an-ident"); + }); + + // The whole gate: without this throw a failed migration deploys anyway. + it("throws with the command's own output when it exits non-zero", async () => { + const { executor } = makeExecutor({ code: 1, output: "SQLSTATE[42S02]: table not found" }); + const runtime = new BareRuntime({ executor }); + await expect( + runtime.runReleaseCommand(config(), "php artisan migrate --force", () => {}), + ).rejects.toThrow(/exit code 1[\s\S]*SQLSTATE\[42S02\]/); + }); + + // Bounded: an unbounded release command holds the deploy open forever with the + // old version still serving. The abort's exit code must not be reported as the + // failure — the timeout is the story. + it("aborts and reports a timeout rather than the killed child's exit code", async () => { + const executor = { + exec: vi.fn(async () => ""), + streamExec: vi.fn( + (_command: string, _onLog: unknown, opts?: { signal?: AbortSignal }) => + new Promise<{ code: number; output: string }>((resolve) => { + opts?.signal?.addEventListener("abort", () => resolve({ code: 143, output: "" })); + }), + ), + writeFile: vi.fn(async () => {}), + readFile: vi.fn(async () => ""), + exists: vi.fn(async () => false), + mkdir: vi.fn(async () => {}), + rm: vi.fn(async () => {}), + dispose: vi.fn(async () => {}), + } as unknown as CommandExecutor; + const runtime = new BareRuntime({ executor }); + await expect( + runtime.runReleaseCommand(config(), "sleep 999", () => {}, { timeoutMs: 20 }), + ).rejects.toThrow(/timed out after/); + }); +}); diff --git a/packages/adapters/src/runtime/bare.ts b/packages/adapters/src/runtime/bare.ts index 86721884f..2c1020d3b 100644 --- a/packages/adapters/src/runtime/bare.ts +++ b/packages/adapters/src/runtime/bare.ts @@ -32,7 +32,7 @@ import type { import { LocalExecutor, wrapLocalBuildCommand } from "../system/executor"; import { ensureOwnedDir } from "../system/elevated-executor"; import { execReliable } from "../system/remote-journal"; -import { STACKS, appVolumeTargets, buildOutputTransferExcludes, safeErrorMessage, missingOutputDirectoryMessage, packageManagerEnsureCommand, type StackId, type StackDefinition } from "@repo/core"; +import { SYSTEM, STACKS, appVolumeTargets, buildOutputTransferExcludes, safeErrorMessage, missingOutputDirectoryMessage, packageManagerEnsureCommand, type StackId, type StackDefinition } from "@repo/core"; import { checkToolchainForStack, installTools } from "../toolchain"; import type { RuntimeAdapter, @@ -128,6 +128,7 @@ export class BareRuntime implements RuntimeAdapter { // past release really is an in-place unit swap (see makeActive). "unitRestore", "inContainerExec", + "releaseCommand", ]); private readonly workDir: string; @@ -680,6 +681,80 @@ export class BareRuntime implements RuntimeAdapter { }; } + /** + * Run one release command in the STAGED artifact directory, before it is + * promoted to a release and before the supervisor starts anything. + * + * That directory is the exact tree `deploy` promotes seconds later, so the + * command sees the code it is migrating for. It runs through a login shell + * (`sh -lc`, via the same wrap the build steps use) with the deploy's env + * exported — the closest match available to what `ExecStart=/bin/sh -lc` gives + * the start command. + * + * One difference worth knowing: `linkPersistentPaths` has not run yet, so a + * path that will become a symlink into `shared/` (Laravel's `storage/`) is + * still a plain directory here. A command that MIGRATES A DATABASE is + * unaffected; one that writes files it expects to survive the release swap + * (an SQLite file under `storage/`) would write into the release copy that + * `shared/` is about to be seeded FROM on a first deploy, and into the release + * copy alone on later ones. + */ + async runReleaseCommand( + config: DeployConfig, + command: string, + onLog: LogCallback, + opts?: { timeoutMs?: number }, + ): Promise { + const workDir = config.imageRef ?? this.projectDir(config.projectId); + const timeoutMs = opts?.timeoutMs ?? SYSTEM.DEPLOYMENTS.RELEASE_COMMAND_TIMEOUT_MS; + + // Same env the supervisor gives the start command (see deploy below). + // Non-identifier keys are dropped rather than breaking the `export` prefix, + // matching the build pipeline's env handling. + const env: Record = { + ...Object.fromEntries( + Object.entries(config.envVars ?? {}).map(([k, v]) => [k, String(v)]), + ), + PORT: String(config.port), + NODE_ENV: config.environment === "production" ? "production" : "development", + }; + const envPrefix = Object.entries(env) + .filter(([k]) => /^[A-Za-z_][A-Za-z0-9_]*$/.test(k)) + .map(([k, v]) => `export ${k}=${sq(v)}`) + .join(" && "); + const full = `${envPrefix ? `${envPrefix} && ` : ""}cd ${sq(workDir)} && ${command}`; + // Login-shell wrap for a LOCAL target only — same rule buildOnTarget applies, + // and the reason a version-managed toolchain (nvm, rbenv) is on PATH at all. + const effective = this.executor instanceof LocalExecutor ? wrapLocalBuildCommand(full) : full; + + const abort = new AbortController(); + let timedOut = false; + const timer = setTimeout(() => { + timedOut = true; + abort.abort(); + }, timeoutMs); + let result: { code: number; output: string }; + try { + result = await this.executor.streamExec(effective, onLog, { signal: abort.signal }); + } finally { + clearTimeout(timer); + } + + // Checked BEFORE the exit code: an aborted child's code is whatever the kill + // produced, and reporting that as the failure would hide the real cause. + if (timedOut) { + throw new Error( + `Release command timed out after ${Math.round(timeoutMs / 1000)}s: ${command}`, + ); + } + if (result.code !== 0) { + const tail = result.output.trim().slice(-1000); + throw new Error( + `Release command failed with exit code ${result.code}: ${command}${tail ? `\n${tail}` : ""}`, + ); + } + } + async deployStatic(config: DeployConfig & { outputDirectory: string }): Promise { const stagedDir = config.imageRef ?? this.projectDir(config.projectId); const workDir = config.imageRef diff --git a/packages/adapters/src/runtime/docker-release-command.test.ts b/packages/adapters/src/runtime/docker-release-command.test.ts new file mode 100644 index 000000000..4323975a1 --- /dev/null +++ b/packages/adapters/src/runtime/docker-release-command.test.ts @@ -0,0 +1,130 @@ +import { PassThrough } from "node:stream"; +import { describe, expect, it } from "vitest"; + +import { DockerRuntime } from "./docker"; +import type { DeployConfig } from "../types"; + +/** + * The docker release phase runs each command in a THROWAWAY container off the + * freshly-built image, before anything is activated. + * + * Why a one-off container and not an exec into the deployment: at this point the + * new version isn't running and the OLD one still is — an exec would run the new + * release's migrations inside the old image, and a failure would take a healthy + * container down with it. The call shape below is what encodes that: no + * published port (it must never contend with the running app for the loopback + * pin), no restart policy (a command that exits non-zero must fail, not bounce), + * and the container is removed either way. + */ +function fakeDaemon(opts: { statusCode?: number; log?: string }) { + const created: Array> = []; + const removed: Array | undefined> = []; + const stream = new PassThrough(); + const docker = { + createContainer: async (args: Record) => { + created.push(args); + return { + id: "release-container-1", + start: async () => {}, + logs: async () => { + setImmediate(() => { + if (opts.log) stream.write(Buffer.from(opts.log, "utf8")); + stream.end(); + }); + return stream; + }, + wait: async () => ({ StatusCode: opts.statusCode ?? 0 }), + stop: async () => {}, + remove: async (o?: Record) => { + removed.push(o); + }, + }; + }, + }; + return { docker, created, removed }; +} + +function config(overrides: Partial = {}): DeployConfig { + return { + projectId: "proj_1", + deploymentId: "dep_1", + buildSessionId: "bs_1", + imageRef: "openship/proj_1:dep_1", + environment: "production", + port: 3000, + hostPort: 41234, + envVars: { DATABASE_URL: "postgres://u:p@db/app" }, + resources: { cpuCores: 0, memoryMb: 0, diskMb: 0 }, + restartPolicy: "always", + slug: "my-app", + volumes: ["storage:/app/storage"], + ...overrides, + } as unknown as DeployConfig; +} + +async function runtimeWith(docker: unknown): Promise { + const runtime = await DockerRuntime.create({ + dockerSocketPath: "/tmp/openship-test-absent.sock", + }); + (runtime as unknown as { _docker: unknown })._docker = docker; + return runtime; +} + +describe("DockerRuntime.runReleaseCommand", () => { + it("declares the capability", async () => { + const runtime = await runtimeWith(fakeDaemon({}).docker); + expect(runtime.supports("releaseCommand")).toBe(true); + }); + + it("runs the command in a one-off container off the new image, with the deploy's env and volumes", async () => { + const { docker, created, removed } = fakeDaemon({ log: "Migrating...\n" }); + const lines: string[] = []; + await (await runtimeWith(docker)).runReleaseCommand( + config(), + "php artisan migrate --force", + (entry) => lines.push(entry.message), + ); + + expect(created).toHaveLength(1); + const args = created[0]!; + expect(args.Image).toBe("openship/proj_1:dep_1"); + // Entrypoint override: a base image's docker-entrypoint.sh would swallow Cmd. + expect(args.Entrypoint).toEqual(["/bin/sh", "-c"]); + expect(args.Cmd).toEqual(["php artisan migrate --force"]); + expect(args.Env).toContain("DATABASE_URL=postgres://u:p@db/app"); + expect(args.Env).toContain("NODE_ENV=production"); + // Same mounts the app gets, project-scoped — a migration has to write to the + // volume the app will read from. + const hostConfig = args.HostConfig as Record; + expect(hostConfig.Binds).toEqual(["openship-my-app-storage:/app/storage"]); + // The two things it must NOT inherit from the deploy. + expect(hostConfig.PortBindings).toBeUndefined(); + expect(hostConfig.RestartPolicy).toBeUndefined(); + // Output reaches the deploy log, and the container doesn't leak. + expect(lines.join("")).toContain("Migrating..."); + expect(removed).toHaveLength(1); + }); + + it("fails the deploy on a non-zero exit, with the command's output in the message", async () => { + const { docker, removed } = fakeDaemon({ + statusCode: 1, + log: "SQLSTATE[42S02]: Base table or view not found", + }); + await expect( + (await runtimeWith(docker)).runReleaseCommand(config(), "php artisan migrate --force", () => {}), + ).rejects.toThrow(/exit code 1[\s\S]*SQLSTATE\[42S02\]/); + // Removed on the failure path too — a failed release must not leave a container. + expect(removed).toHaveLength(1); + }); + + it("refuses without a built image rather than running against nothing", async () => { + const { docker } = fakeDaemon({}); + await expect( + (await runtimeWith(docker)).runReleaseCommand( + config({ imageRef: undefined }), + "php artisan migrate --force", + () => {}, + ), + ).rejects.toThrow(/imageRef/); + }); +}); diff --git a/packages/adapters/src/runtime/docker.ts b/packages/adapters/src/runtime/docker.ts index 5a5179819..9f957ef1a 100644 --- a/packages/adapters/src/runtime/docker.ts +++ b/packages/adapters/src/runtime/docker.ts @@ -139,6 +139,7 @@ import { transferLocalDirectory } from "./transfer"; import { ownsNetworkEndpoint, safeErrorMessage, + SYSTEM, type ComposeAdvanced, type ComposeHealthcheck, } from "@repo/core"; @@ -903,6 +904,7 @@ export class DockerRuntime implements RuntimeAdapter { // A docker exec lands in the container's own namespaces, so a command run // through it cannot reach the host. Bare deliberately does NOT declare this. "isolatedExec", + "releaseCommand", ]); /** Docker honors every extended compose key we currently support. */ @@ -2645,6 +2647,149 @@ export class DockerRuntime implements RuntimeAdapter { }; } + /** + * Run one release command in a THROWAWAY container off the freshly-built + * image, before anything is activated. + * + * A one-off container, not an exec into the running deployment: at this point + * in the pipeline the new version isn't running yet and the old one is still + * serving — `exec`ing there would run the new release's migrations inside the + * OLD image, and a failure would take a healthy container down with it. + * + * Env / mounts / network mirror `deploy` above so a migration reaches the same + * database and writes to the same volume the app will read from. Deliberately + * NOT mirrored: the published port (a release command must never contend with + * the running app for the loopback pin) and the restart policy (a one-off + * command that exits non-zero must fail, not bounce). + */ + async runReleaseCommand( + config: DeployConfig, + command: string, + onLog: LogCallback, + opts?: { timeoutMs?: number }, + ): Promise { + const imageRef = config.imageRef; + if (!imageRef) { + throw new Error("Release commands require an imageRef (built image tag)"); + } + const timeoutMs = opts?.timeoutMs ?? SYSTEM.DEPLOYMENTS.RELEASE_COMMAND_TIMEOUT_MS; + + // Same env the start command gets (see deploy above) — a migration that + // resolves its DSN differently from the app is worse than no migration. + const env = [ + `PORT=${config.port}`, + `NODE_ENV=${config.environment === "production" ? "production" : "development"}`, + ...Object.entries(config.envVars).map(([k, v]) => `${k}=${v}`), + ]; + const scopedBinds = scopeVolumeBinds( + config.slug || config.runtimeName || config.projectId, + config.volumes ?? [], + true, + ); + // Best-effort, exactly as in deploy: no network just means a release command + // can't reach a linked project by alias, which is not a reason to refuse. + let networkId: string | undefined; + if (config.networkAlias) { + networkId = await this.ensureNetwork( + config.slug || config.runtimeName || config.projectId, + ).catch(() => undefined); + } + + const container = await this.docker.createContainer({ + name: `openship-release-${config.deploymentId}-${Date.now().toString(36)}`, + Image: imageRef, + // Override the ENTRYPOINT for the same reason the static extract does: + // a base image's docker-entrypoint.sh would swallow this Cmd. + Entrypoint: ["/bin/sh", "-c"], + Cmd: [command], + Env: env, + Labels: this.labels({ + deploymentId: config.deploymentId, + projectId: config.projectId, + }), + HostConfig: { + Binds: scopedBinds.length > 0 ? scopedBinds : undefined, + ...(networkId ? { NetworkMode: networkId } : {}), + ...dockerResourceLimits(config.resources), + }, + }); + + try { + await container.start(); + + // Follow from the start of the container's life — `tail` is deliberately + // omitted, since the daemon's default is the whole log, so opening the + // stream after `start()` can't lose the first lines. + const stream = (await container.logs({ + stdout: true, + stderr: true, + follow: true, + })) as unknown as NodeJS.ReadableStream; + + // Tail kept for the failure message: the operator has to be able to read + // WHY a migration failed without going to find the deploy log. + let tail = ""; + let buffer = ""; + stream.on("data", (chunk: Buffer) => { + const text = stripDockerChunkHeader(chunk).toString("utf-8"); + tail = (tail + text).slice(-4000); + buffer += text; + const lines = buffer.split("\n"); + buffer = lines.pop() ?? ""; + for (const line of lines) { + onLog({ timestamp: new Date().toISOString(), message: `${line}\n`, level: parseLogLevel(line) }); + } + }); + + // Resolves when the daemon closes the follow stream, which it does when the + // container exits. Awaited alongside `wait` so the last lines the command + // wrote before exiting aren't lost to the race between the two. + const streamDone = new Promise((resolve) => { + stream.on("end", () => resolve()); + stream.on("close", () => resolve()); + stream.on("error", () => resolve()); + }); + + let timedOut = false; + const timer = setTimeout(() => { + timedOut = true; + (stream as unknown as { destroy?: () => void }).destroy?.(); + // Stop rather than remove: the finally below removes it, and stopping is + // what unblocks the `wait` this race is holding. + container.stop({ t: 5 }).catch(() => { /* best effort */ }); + }, timeoutMs); + + let status: { StatusCode: number }; + try { + status = await container.wait(); + // Bounded: a daemon that never closes the stream after the container is + // gone must cost a couple of seconds, not the whole release budget (which + // would then report a finished command as a timeout). + await Promise.race([streamDone, new Promise((r) => setTimeout(r, 2_000))]); + } finally { + clearTimeout(timer); + (stream as unknown as { destroy?: () => void }).destroy?.(); + } + if (buffer) { + onLog({ timestamp: new Date().toISOString(), message: `${buffer}\n`, level: parseLogLevel(buffer) }); + } + + if (timedOut) { + throw new Error( + `Release command timed out after ${Math.round(timeoutMs / 1000)}s: ${command}`, + ); + } + if (status.StatusCode !== 0) { + throw new Error( + `Release command failed with exit code ${status.StatusCode}: ${command}` + + (tail.trim() ? `\n${tail.trim().slice(-1000)}` : ""), + ); + } + } finally { + await container.remove({ force: true }).catch(() => { /* best effort */ }); + } + } + async stop(containerId: string): Promise { const container = this.docker.getContainer(containerId); await container.stop(); diff --git a/packages/adapters/src/runtime/types.ts b/packages/adapters/src/runtime/types.ts index 92ecbf07e..ab09025c1 100644 --- a/packages/adapters/src/runtime/types.ts +++ b/packages/adapters/src/runtime/types.ts @@ -126,6 +126,14 @@ export type RuntimeCapability = * "inContainerExec". Omitted ⇒ not confined, so a new runtime fails closed. */ | "isolatedExec" + /** + * Runtime can run a one-off RELEASE COMMAND against a freshly-built artifact, + * between the build and the cutover — `runReleaseCommand`. Docker runs it in a + * throwaway container off the new image; Bare runs it in the staged release + * directory. Cloud has no such primitive, so a project that declares release + * commands gets a logged skip there rather than a silently-unrun migration. + */ + | "releaseCommand" /** * Runtime can report a container's RESTART HISTORY and health, not just a * point-in-time status — the readings the post-deploy stabilization watch @@ -207,6 +215,24 @@ export interface RuntimeAdapter { /** Start a container/process from a completed build */ deploy(config: DeployConfig, onLog?: LogCallback): Promise; + /** + * Run ONE release command against the freshly-built artifact named by + * `config.imageRef`, before anything is activated. Streams the command's + * output through `onLog` and REJECTS on a non-zero exit (or on the timeout) + * with that output in the message, which is what fails the deploy. + * + * Must not touch the running deployment: this is a throwaway execution + * context (a one-off container / the not-yet-promoted release directory), so + * a failure leaves the previous version untouched and still serving. + * Only present when `supports("releaseCommand")`. + */ + runReleaseCommand?( + config: DeployConfig, + command: string, + onLog: LogCallback, + opts?: { timeoutMs?: number }, + ): Promise; + /** Stop a running container/process (preserves state) */ stop(containerId: string): Promise; diff --git a/packages/adapters/src/types.ts b/packages/adapters/src/types.ts index ab6078d7e..f937f6780 100644 --- a/packages/adapters/src/types.ts +++ b/packages/adapters/src/types.ts @@ -282,6 +282,13 @@ export interface DeployConfig { hostPort?: number; /** Shell command to start the application (e.g. "npm start", "node server.js") */ startCommand?: string; + /** + * Commands run ONCE per deploy, after the build and BEFORE this config is + * activated — migrations, cache warms. Each runs with the same env the start + * command gets; a non-zero exit fails the deploy. Empty/absent ⇒ no release + * phase, the default. Consumed by `runReleaseCommand`, never by `deploy`. + */ + releaseCommands?: string[]; /** Detected framework / stack (e.g. "nextjs", "express") */ stack?: string; /** Environment variables injected at runtime */ diff --git a/packages/core/src/openship-config/parse.test.ts b/packages/core/src/openship-config/parse.test.ts index 89644977a..4077a4892 100644 --- a/packages/core/src/openship-config/parse.test.ts +++ b/packages/core/src/openship-config/parse.test.ts @@ -165,6 +165,46 @@ describe("parseOpenshipConfig", () => { }); }); + describe("releaseCommands", () => { + // Laravel 13's canonical set — the case the release phase exists for. + it("round-trips a list of commands in declared order", () => { + const releaseCommands = [ + "php artisan migrate --force", + "php artisan optimize", + "php artisan storage:link", + "php artisan reload", + ]; + const { config, errors, warnings } = parseOpenshipConfig({ releaseCommands }); + expect(errors).toEqual([]); + expect(warnings).toEqual([]); + expect(config?.releaseCommands).toEqual(releaseCommands); + }); + + it("rejects a bare string and a non-string entry", () => { + expect(parseOpenshipConfig({ releaseCommands: "php artisan migrate" }).errors).toEqual([ + "releaseCommands: must be an array of strings", + ]); + expect(parseOpenshipConfig({ releaseCommands: ["ok", 7] }).errors).toEqual([ + "releaseCommands[1]: must be a string", + ]); + }); + + // Absent must stay distinguishable from `[]` all the way to the column: the + // phase is opt-in, and an undeclared field has to behave exactly as it did + // before the field existed. + it("is absent (not undefined-valued) when undeclared", () => { + const { config, errors } = parseOpenshipConfig({ framework: "nextjs" }); + expect(errors).toEqual([]); + expect(config && "releaseCommands" in config).toBe(false); + }); + + it("keeps an explicit empty list as a declared opt-out", () => { + const { config, errors } = parseOpenshipConfig({ releaseCommands: [] }); + expect(errors).toEqual([]); + expect(config?.releaseCommands).toEqual([]); + }); + }); + describe("readiness", () => { it("round-trips every field", () => { const readiness = { diff --git a/packages/core/src/openship-config/parse.ts b/packages/core/src/openship-config/parse.ts index b1762e680..ce9230b3f 100644 --- a/packages/core/src/openship-config/parse.ts +++ b/packages/core/src/openship-config/parse.ts @@ -39,6 +39,7 @@ const TOP_LEVEL_KEYS = new Set([ "installCommand", "buildCommand", "startCommand", + "releaseCommands", "outputDirectory", "buildImage", "productionPaths", @@ -428,6 +429,7 @@ export function parseOpenshipConfig(raw: unknown): ParseResult { installCommand: ctx.str(raw.installCommand, "installCommand"), buildCommand: ctx.str(raw.buildCommand, "buildCommand"), startCommand: ctx.str(raw.startCommand, "startCommand"), + releaseCommands: ctx.strArray(raw.releaseCommands, "releaseCommands"), outputDirectory: ctx.str(raw.outputDirectory, "outputDirectory"), buildImage: ctx.str(raw.buildImage, "buildImage"), productionPaths: ctx.strArray(raw.productionPaths, "productionPaths"), diff --git a/packages/core/src/openship-config/schema.ts b/packages/core/src/openship-config/schema.ts index 7686144ac..a49a02736 100644 --- a/packages/core/src/openship-config/schema.ts +++ b/packages/core/src/openship-config/schema.ts @@ -162,6 +162,20 @@ export interface OpenshipConfig { installCommand?: string; buildCommand?: string; startCommand?: string; + /** + * Commands run ONCE per deploy, after the build and before the new version is + * activated — migrations, cache warms, symlink setup. A non-zero exit FAILS the + * deploy, so the old version keeps serving rather than a new one coming up + * against a schema it can't use. + * + * A LIST, not one `&&`-chained string (the shape `startCommand` and friends + * take): a framework's release set is several independent steps — Laravel 13's + * is `migrate --force`, `optimize`, `storage:link`, `reload` — and each gets + * its own log marker and its own attributable failure. Absent ⇒ no release + * phase runs at all, which is the default for every project; `[]` says the same + * thing out loud. Nothing is auto-injected per framework. + */ + releaseCommands?: string[]; outputDirectory?: string; buildImage?: string; productionPaths?: string[]; diff --git a/packages/core/src/system.ts b/packages/core/src/system.ts index fafe2fc3b..e24850fc6 100644 --- a/packages/core/src/system.ts +++ b/packages/core/src/system.ts @@ -71,6 +71,14 @@ export const SYSTEM = { READINESS_TIMEOUT_MS: 45_000, /** Readiness-probe poll interval. */ READINESS_INTERVAL_MS: 1_000, + /** + * Per-command budget for the release phase (migrations, cache warms) that + * runs between build and cutover. Same 10 minutes a bare build step gets + * (BareRuntimeOptions.buildTimeout) — a migration is build-shaped work, and + * an unbounded one would hold the deploy open forever with the old version + * still serving. Only reached by a project that declares releaseCommands. + */ + RELEASE_COMMAND_TIMEOUT_MS: 10 * 60 * 1000, }, // ── SSE / Build Streaming ──────────────────────────────────────────── diff --git a/packages/db/drizzle/0109_project_release_commands.sql b/packages/db/drizzle/0109_project_release_commands.sql new file mode 100644 index 000000000..fe7d72b61 --- /dev/null +++ b/packages/db/drizzle/0109_project_release_commands.sql @@ -0,0 +1,18 @@ +-- Deploy-time release phase, per project. +-- +-- Commands that run ONCE per deploy, after the build and before the new version +-- is activated: `php artisan migrate --force`, `rails db:migrate`, `optimize`, +-- `storage:link`. Before this there was exactly one command per app stack (the +-- start command), so a Laravel/Rails/Django project's migrations never ran and +-- had to be applied by hand through the service terminal. +-- +-- A jsonb LIST, not a text column with `&&`-chained steps: a framework's release +-- set is several independent commands (Laravel 13: migrate --force, optimize, +-- storage:link, reload) and each needs its own log marker and its own +-- attributable failure. Matches how `volumes` / `monorepo_shared_paths` store +-- their lists. +-- +-- Nullable with NO default and no backfill: NULL means "no release phase", so +-- every existing project keeps deploying exactly as it does today. A non-zero +-- exit fails the deploy, so opting in has to be explicit. +ALTER TABLE "project" ADD COLUMN IF NOT EXISTS "release_commands" jsonb; diff --git a/packages/db/drizzle/meta/_journal.json b/packages/db/drizzle/meta/_journal.json index 9e0b1fd7e..3ff717014 100644 --- a/packages/db/drizzle/meta/_journal.json +++ b/packages/db/drizzle/meta/_journal.json @@ -764,6 +764,13 @@ "when": 1788301307325, "tag": "0108_credential", "breakpoints": true + }, + { + "idx": 109, + "version": "7", + "when": 1788387707325, + "tag": "0109_project_release_commands", + "breakpoints": true } ] } diff --git a/packages/db/src/schema/project.ts b/packages/db/src/schema/project.ts index 47ca88a1c..f7c860871 100644 --- a/packages/db/src/schema/project.ts +++ b/packages/db/src/schema/project.ts @@ -180,6 +180,21 @@ export const project = pgTable( composePath: text("compose_path"), /** Start command for production runtime */ startCommand: text("start_command"), + /** + * Commands run ONCE per deploy, between a successful build and the cutover + * to the new version — `php artisan migrate --force`, `rails db:migrate`, + * cache warms. A non-zero exit fails the deploy, so the previous version + * keeps serving. + * + * A LIST rather than one `&&`-chained string (the shape `startCommand` and + * `workspacePrepareCommand` take): each entry is logged and attributed + * separately, and a framework's release set is several independent steps. + * NULL = no release phase, which is the default for every project and is + * byte-identical to the behaviour before this column existed. Seeded from + * `openship.json`'s `releaseCommands`; snapshotted onto each deployment so a + * redeploy of an old release replays the commands as they were. + */ + releaseCommands: jsonb("release_commands").$type(), /** Docker image for build environment (e.g. node:22, oven/bun:latest) */ buildImage: text("build_image"), /** Production mode: host, static, standalone */