diff --git a/CHANGELOG.md b/CHANGELOG.md index dc39a1566..9783d85ff 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,15 @@ # Changelog +## [Unreleased] + +### Added + +- Local data persistence for `base44 dev`: file-backed database in a gitignored `.base44/` state dir, surviving restarts and entity-file edits; `--fresh` starts clean +- `base44 dev status`, `base44 dev seed [--replace]`, and `base44 dev reset` commands (`dev` is now a command group with a default action) +- Seed fixtures (`base44/seed/*.jsonc`, including `users.jsonc`) and a programmatic `base44/seed.ts` hook, auto-applied on first boot and reset +- `base44 data pull` and `base44 data dump` to write seed fixtures from remote or local data +- Local dev-server admin endpoints (`/_base44/dev/status|seed|reset|export`) guarded by a per-instance token + ## [0.0.51] - 2026-04-28 ### Added diff --git a/docs/AGENTS.md b/docs/AGENTS.md index 10f85ab03..35a4e2d5e 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -80,6 +80,7 @@ Read these when working on the relevant area: - **[Making API calls](api-patterns.md)** - HTTP clients, Zod snake_case-to-camelCase transforms, `ApiError.fromHttpError()` - **[Working with resources](resources.md)** - `Resource` interface, adding new resources, site module, unified deploy - **[Plugins](plugins.md)** - Plugin config, namespaces, entity extension rules, function namespacing, pull/deploy behavior +- **[Local data & seeding](local-data.md)** - `.base44` state dir, dev.json, seed fixtures + `seed.ts`, admin endpoints, `dev seed/reset/status`, `data pull/dump` - **[Error handling](error-handling.md)** - Error hierarchy, throwing patterns, error codes, `CLIExitError`, `process.exit` ban - **[Writing tests](testing.md)** - Testkit, Given/When/Then pattern, API mocks, fixtures, test overrides - **[Telemetry & error reporting](telemetry.md)** - PostHog `ErrorReporter`, what's captured, disabling diff --git a/docs/binary-distribution.md b/docs/binary-distribution.md index fa7527f17..b2e506668 100644 --- a/docs/binary-distribution.md +++ b/docs/binary-distribution.md @@ -46,6 +46,8 @@ export function getTemplatesDir(): string { No `assetsDir` parameter is passed through CLIContext or function signatures. Adding new asset types only requires putting them under **dist/assets/** and wiring the build; **build-binaries.ts** collects the whole `dist/assets/` folder with no per-item list. +The Deno wrappers ship this way: `infra/build.ts` copies `deno-runtime/seed.ts` (seed-script runner) into `dist/assets/deno-runtime/` alongside `exec.ts` and `main.ts`. + ## Homebrew Formula `infra/homebrew/base44.rb` is a template for the Homebrew tap formula. It downloads the `.tar.gz` archive for the user's platform from GitHub Releases. Homebrew auto-extracts the tarball, so the install block simply does `bin.install "base44"`. diff --git a/docs/commands.md b/docs/commands.md index 1ca0ef6db..818163ea9 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -1,6 +1,6 @@ # Adding & Modifying CLI Commands -**Keywords:** command, factory pattern, Base44Command, CLIContext, Logger, runTask, spinner, theming, chalk, program.ts, register, banner, intro, outro +**Keywords:** command, factory pattern, Base44Command, CLIContext, Logger, runTask, spinner, theming, chalk, program.ts, register, banner, intro, outro, subcommand, command group, default action, Option.choices, force, confirmDestructiveAction Commands live in `src/cli/commands//`. They use a **factory pattern** — each file exports a function that returns a `Base44Command`. @@ -80,6 +80,64 @@ import { getMyCommand } from "@/cli/commands//.js"; program.addCommand(getMyCommand()); ``` +## Command Groups + +### Group with a default action + +A `Base44Command` can both do work and host subcommands — `base44 dev` starts the server, `base44 dev status` is a subcommand (see `src/cli/commands/dev/index.ts`): + +```typescript +export function getDevCommand(): Command { + return new Base44Command("dev") + .option("--fresh", "Delete local data before starting") + .hook("preAction", (thisCommand, actionCommand) => { + // preAction also fires for subcommands; validate only the default action + if (thisCommand === actionCommand) { + validateDevOptions(thisCommand); + } + }) + .action(devAction) + .addCommand(getDevStatusCommand()) + .addCommand(getDevSeedCommand()) + .addCommand(getDevResetCommand()); +} +``` + +### Plain group (no action) + +A pure namespace is a bare Commander `Command` wrapping `Base44Command` subcommands — see `src/cli/commands/data/index.ts` (`base44 data pull|dump`). Register the group in `src/cli/program.ts` like any command. + +## Enum Options (`Option.choices`) + +For options with a fixed value set, use `.addOption()` with Commander's `Option` class — invalid values fail at parse time and help output lists the choices (see `src/cli/commands/data/pull.ts`): + +```typescript +import { Option } from "commander"; + +.addOption( + new Option("--data-env ", "Remote data environment to read from") + .choices(["prod", "dev"]) + .default("prod"), +) +``` + +## Destructive Operations (`--force` + Confirm) + +Destructive commands confirm in a TTY and require `--force` non-interactively. Use `confirmDestructiveAction` from `src/cli/commands/dev/seed-shared.ts`: `--force` skips, TTY prompts, non-interactive without `--force` throws `InvalidInputError`, and a declined prompt exits 0 via `CLIExitError`. Example from `src/cli/commands/dev/seed.ts`: + +```typescript +if (mode === "replace") { + await confirmDestructiveAction( + isNonInteractive, + options.force === true, + "Replace mode deletes existing records in seeded collections. Continue?", + "--force is required to use --replace in non-interactive mode", + ); +} +``` + +For the local-data domain itself (seed lifecycle, dev-server state, `data pull/dump`), see [Local data & seeding](local-data.md). + ## CLIContext (Automatic Injection) `CLIContext` is automatically injected as the **first argument** to all action functions by `Base44Command`. Destructure what you need: diff --git a/docs/local-data.md b/docs/local-data.md new file mode 100644 index 000000000..5eaecf785 --- /dev/null +++ b/docs/local-data.md @@ -0,0 +1,103 @@ +# Local Data & Seeding + +**Keywords:** seed, seeding, fixtures, persistence, NeDB, .base44, state dir, dev.json, meta.json, applySeeds, readSeedFiles, SeedSummary, seed hash, admin endpoints, x-base44-dev-admin, seed.ts, runSeedScript, data pull, data dump, dev seed, dev reset, dev status, --fresh, ephemeral dev server, service role + +`base44 dev` persists entity data in file-backed NeDB collections under a gitignored, project-relative `.base44/` dir, and seeds it from `base44/seed/*.jsonc` fixtures plus an optional programmatic `base44/seed.ts`. Design rationale: [proposals/local-data-and-seeding.md](proposals/local-data-and-seeding.md). + +## Module Map + +| Module | Owns | +|---|---| +| `src/core/local-state/` | `.base44/` path helpers; Zod schemas + read/write for `meta.json` and `dev.json` (pid-based stale detection); local dev JWTs (`createServiceToken`) | +| `src/core/resources/seed/` | Fixture file formats (Zod), `readSeedFiles()`, seed hash, `SeedSummary`/`DevResetResult` shapes, `normalizeSeedName()` | +| `src/core/seed-script/` | `runSeedScript()` — spawns Deno on the seed wrapper (test seams: `spawnImpl`, `wrapperPath`; `seedScript` test override) | +| `deno-runtime/seed.ts` | Deno wrapper: builds the script's `ctx` from env vars and calls its default export | +| `src/cli/dev/dev-server/db/seed.ts` | `applySeeds()` — applies users + entity fixtures against the `Database` | +| `src/cli/dev/dev-server/routes/admin-router.ts` | `/_base44/dev/*` admin endpoints behind the per-instance token | +| `src/cli/dev/seed-script-step.ts` | Runs `seed.ts` after fixtures, records the outcome on the summary (never throws) | +| `src/cli/commands/dev/` | `dev` group: default server action + `status`/`seed`/`reset`; `seed-shared.ts` holds the live/offline duality helpers | +| `src/cli/commands/data/` | `data pull` (remote → fixtures) and `data dump` (local → fixtures) | + +The layering rule holds: `core/` owns formats, paths, hashing, and the script runner; anything touching the `Database` class stays in `cli/` (dev-server). + +## State Dir Layout + +``` +.base44/ # gitignored, safe to delete +├── dev.json # instance descriptor while a dev server runs (incl. adminToken) +└── data/ + ├── meta.json # { formatVersion: 1, appId, seed: { hash, appliedAt } | null } + ├── task.db # one NeDB file per collection + └── $user.db # private auth collection (passwords) +``` + +Path helpers: `getStateDir` / `getDataDir` / `getDevJsonPath` / `getMetaJsonPath` in `src/core/local-state/paths.ts`. `readDevInstance()` returns `null` (and deletes the file) when the descriptor is invalid or its pid is dead. + +## Lifecycle + +**Startup** (`createDevServer` in `src/cli/dev/dev-server/main.ts`): + +1. `--fresh` deletes the data dir. A data dir owned by a different `appId` refuses to start (hint: `--fresh`). +2. Auto-seed happens **only when the data dir is new** (no valid `meta.json`): fixtures apply in `replace` mode. Otherwise a changed seed hash just logs a "run `base44 dev seed`" hint. +3. `meta.json` is written; `dev.json` is written after listen and deleted on graceful shutdown. +4. The `seed.ts` step runs **after listen** (it talks to the server over HTTP); startup seed failures warn but never crash the server. +5. Entity-file changes reload schemas only (`db.reloadSchemas`) — data is preserved. + +**Seed modes** (`applySeeds`): users are always upserted by email through the same building blocks as local registration (the CLI login user survives every mode). Entity records: + +- `upsert` (default for `dev seed`): upsert-by-id; id-less records are skipped; never deletes. +- `replace` (first boot, `--fresh`, `dev reset`, `dev seed --replace`): truncate each seeded collection, then insert everything. + +Fixture filenames resolve to entities by normalized comparison (case/`-`/`_`-insensitive, so `team-member.jsonc` → `TeamMember`); `users.jsonc` is reserved. Records are validated and stamped (`id`, `created_by`, dates) exactly like the entity POST route, as service role (bypasses RLS/FLS). + +**Live vs offline — one implementation.** `dev seed`, `dev reset`, and `data dump` read `dev.json` (`src/cli/commands/dev/seed-shared.ts`): + +- Live instance → call its admin endpoints. +- No instance, no `seed.ts` → open the NeDB files directly (`openOfflineDatabase`). +- No instance, `seed.ts` present → `withTempDevInstance()` boots an **ephemeral** internal dev server (random port, `ephemeral: true` — no `dev.json`, no startup auto-seed, `serveCommand` stripped), drives the same admin endpoints, then shuts down. + +## Admin Endpoints + +Mounted at `/_base44/dev` (`admin-router.ts`); every route requires the `x-base44-dev-admin` header matching the token in `dev.json`: + +| Endpoint | Returns | +|---|---| +| `GET /status` | `{ appId, port, startedAt, seed, collections: {name: count} }` | +| `POST /seed` body `{ mode }` | `SeedSummary` | +| `POST /reset` | `DevResetResult` | +| `GET /export?entities=a,b` | `{ collections: { Name: records[] } }` (`data dump` live path) | + +## seed.ts Runner Contract + +`runSeedScript()` (`src/core/seed-script/run-script.ts`) copies `deno-runtime/seed.ts` to a temp file (Deno blocks `npm:` specifiers under `node_modules`) and spawns `deno run --allow-all`. The wrapper builds: + +- `ctx.base44` — SDK client bound to the local dev server with a service-subject JWT in the plain token field (the server resolves `server@server.com` to the service principal; bypasses RLS/FLS) +- `ctx.remote({ dataEnv? })` — SDK client factory for the linked remote app as the CLI user; `dataEnv: "dev"` adds the `X-Data-Env: dev` header. Throws the recorded reason when remote credentials were unavailable. +- `ctx.log(msg)` — stderr logger + +| Env var | Value | +|---|---| +| `SCRIPT_PATH` | `file://` URL of the project's `base44/seed.ts` | +| `BASE44_APP_ID` | App id | +| `BASE44_LOCAL_URL` | Local dev server base URL | +| `BASE44_LOCAL_SERVICE_TOKEN` | Local service-role JWT | +| `BASE44_ACCESS_TOKEN` | Remote app-user token (may be empty) | +| `BASE44_APP_BASE_URL` | Remote app's published URL (may be empty) | +| `BASE44_REMOTE_ERROR` | Why remote credentials are missing, if they are | + +The child's stdout **and** stderr are piped to the CLI's stderr, so script output can never corrupt `--json` stdout. Deno is required only for this step: without it, fixtures still apply, the summary reports `script: { ran: false }` plus a warning, and `dev seed`/`dev reset` exit non-zero. + +## Extending + +- **New fixture field** (like `created_by`): add it to `SeedRecordSchema` in `src/core/resources/seed/schema.ts` (structure) and handle it in `applyRecordsFixture` in `src/cli/dev/dev-server/db/seed.ts` (behavior). Per-record entity validation stays at apply time. +- **New admin endpoint**: add the route in `admin-router.ts` **inside** the token middleware, a typed callback on `AdminRouterDeps`, its implementation in `dev-server/main.ts`, and a Zod-parsed client call in `seed-shared.ts` (`callAdminEndpoint`) so live, offline, and ephemeral callers all get it. +- **New `ctx` capability for seed.ts**: extend the wrapper (`deno-runtime/seed.ts`), pass host-side values through env vars in `run-script.ts`, and register new SDK clients in the wrapper's `clients` array so cleanup lets the process exit. + +## Rules + +1. **Seed summary/reset shapes are frozen Zod contracts** (`SeedSummarySchema`, `DevResetResultSchema` in `src/core/resources/seed/schema.ts`) — returned by the applier and the admin endpoints, and printed as `--json` stdout. Change them deliberately and everywhere at once. +2. **Never write to stdout from the runner or the wrapper** — script output goes to stderr (`ctx.log`, piped child stdio); stdout is reserved for the CLI's `--json` document. +3. **Admin routes stay behind the token middleware** — new endpoints go inside `createAdminRouter` after the `x-base44-dev-admin` check; never accept the token from anywhere but that header. +4. **core/ must not import cli/** — the `Database`-touching applier lives in `src/cli/dev/dev-server/db/seed.ts`; parsing, validation, and hashing stay in `src/core/resources/seed/`. +5. **Never silently re-seed existing data** — auto-apply happens only when the data dir is new; everything else is an explicit command. +6. **Users are special** — seeded via `users.jsonc` through the registration building blocks, upserted by email in every mode, never deleted (the CLI login user always survives), and never exported by `data dump`. diff --git a/docs/proposals/local-data-and-seeding.md b/docs/proposals/local-data-and-seeding.md new file mode 100644 index 000000000..3b960a09a --- /dev/null +++ b/docs/proposals/local-data-and-seeding.md @@ -0,0 +1,360 @@ +# Proposal: Local Data Persistence & Seeding for `base44 dev` + +**Keywords:** seed, seeding, persistence, local data, dev server, fixtures, reset, data pull, NeDB, state dir, worktree isolation + +**Status:** Phases 1–3 implemented (see status update below) + +## Status update (2026-07-13) + +Phases 1–3 — persistence + lifecycle, seeding, and the remote bridge — are +implemented on branch `claude/base44-cli-seed-feature-30br0e`, **including the +programmatic `base44/seed.ts` hook**, which was pulled forward from the "Later" +phase at user request: remote access is programmable-first (`ctx.remote({ dataEnv })` +inside `seed.ts`), with `data pull`/`data dump` as the zero-code sugar. Sections +2, 6, and 7 and Open question 1 below are revised to match what shipped; the +research sections are unchanged. Contributor-facing architecture docs: +[`local-data.md`](../local-data.md). + +## Problem + +The `base44 dev` entity database is purely in-memory (`new Datastore()` in +`src/cli/dev/dev-server/db/database.ts`). Consequences: + +- **All data is lost on every restart.** Users report re-creating test data by hand or + writing their own scripts that pull records from the remote app and re-insert them on + every run. +- **All data is lost on any entity-file edit.** The watcher calls `db.dropAll()` when + anything in `entitiesDir` changes (`dev-server/main.ts`), so iterating on a schema + wipes your working data mid-session. +- **No seed mechanism exists.** The only auto-created record is the CLI-logged-in user + (as `admin`). Apps gated on `auth.me()` + roles + RLS (the common "private SaaS" + shape) render empty locally, and there is no supported way to establish test users + with roles or baseline records. +- **Not agent-friendly.** AI agents driving `base44 dev` need deterministic, + re-runnable environments: seed once, verify UI, restart freely, run several isolated + envs in parallel across git worktrees. + +Signals: an enterprise user (role-gated, RLS-heavy app) explicitly asked for the +intended seeding/persistence pattern; another user built a personal +pull-from-remote-then-seed script and asked for it to be built in; the team direction +for `dev` is docker-style envs (detached, listable, inspectable), which requires +data isolation and durable state per env. + +## Prior art (research summary) + +| Platform | Persistence default | Seed format | Seed trigger | Reset | Remote bridge | +|---|---|---|---|---|---| +| Supabase | Persistent (Docker volumes); `stop --no-backup` wipes | SQL files, `[db.seed].sql_paths` globs | First start + `db reset` only | `db reset` = recreate + migrate + seed | `db dump --data-only -f seed.sql` | +| Firebase emulators | Ephemeral; snapshot opt-in | Exported snapshot dirs | `--import` at start, `--export-on-exit` | Start without `--import` | `auth:export` from prod | +| Wrangler (D1/KV) | **Persistent by default** in `.wrangler/state` (project-relative) | Plain SQL | Manual `d1 execute --local --file` | Delete state dir (always safe) | Same command with `--remote` | +| Convex | Persistent per-deployment | **Idempotent TS mutation** (`convex/init.ts`) | `convex dev --run init`, re-run anytime | Import `--replace` | First-class `export`/`import` (zip/JSONL) | +| Prisma | BYO DB | TS script in package.json | Auto after `migrate reset` (v7: explicit only) | `migrate reset` = drop + migrate + seed | — | +| PocketBase | Persistent (SQLite `pb_data/`) | JS migrations that insert records | Auto-apply on serve | Delete `pb_data/` | Copy the dir | +| Amplify Gen 2 | Cloud sandbox | TS `seed.ts` that **can create auth users** | `ampx sandbox seed` | Redeploy sandbox | — | + +Strongest lessons: + +1. **Persist by default; wipe explicitly.** Supabase's most-complained-about early + behavior was data loss on restart — they inverted it. Firebase's + ephemeral-by-default is widely worked around. Wrangler's project-relative, + safe-to-delete state dir is the cleanest model and gives git-worktree isolation + for free. +2. **Seeds run on first boot and on reset — never silently against existing data.** + (Supabase, Docker `initdb.d`.) A standalone "re-seed now" command is Supabase's + top unmet request (supabase/cli#1711); Convex has it and it's their doctrine: + idempotent seed, safe to run anytime. +3. **Seed auth users through the real auth path, not raw inserts.** Supabase's worst + recurring pain is seeding `auth.users` by SQL (breaks on every GoTrue change). + Amplify ships an auth-API-compatible seed SDK instead. +4. **One canonical reset command** (`prisma migrate reset`, `migrate:fresh --seed`) + beats a documented delete-then-restart dance — especially for agents. +5. **Remote→local snapshot is the most-requested workflow everywhere** ("develop + against realistic data"). Convex's ID-preserving `export`/`import` with explicit + `--replace`/`--append` is best-in-class. +6. **Agent contract:** `--json` everywhere, never prompt when non-interactive, + destructive ops need `--force` in non-TTY, machine-discoverable instance state + (Convex writes the deployment URL to `.env.local`; docker has `ps`/`inspect`). + +In-house prior art: internal `b44 worktree` has `seedFiles` with "copy once at +creation, user-owned afterwards" semantics; the platform already has +`DataEnvironment` (`prod`/`dev`) separation, an `is_sample` record flag, and admin +`seed_entity_records` with ID remapping — the CLI feature should stay conceptually +aligned with those. + +## Design + +### 1. Persistence: file-backed store in a project-local state dir + +Local dev state moves to a **gitignored, project-relative state dir**, sibling of the +committed `base44/` dir (mirrors `.wrangler/state`): + +``` +.base44/ # gitignored, safe to delete at any time +├── dev.json # running-instance descriptor (see §5) +└── data/ + ├── meta.json # { appId, seedHash, seededAt, formatVersion } + ├── task.db # one NeDB file per collection (append-only journal) + ├── user.db + └── $user.db # private auth collection (passwords/OTP) +``` + +- `Database` switches from `new Datastore()` to + `new Datastore({ filename, autoload: true })` — `@seald-io/nedb` supports file + persistence natively, so **no new dependency** (keeps the zero-dependency + distribution rule) and no native modules (works in npm mode and compiled binaries). +- Data now survives restarts by default. **Ephemeral is the opt-in:** + `base44 dev --fresh` starts from a clean state (wipe + re-seed). +- **Entity-file edits no longer wipe data.** The watcher reloads schemas but keeps + collections (NeDB is schemaless; validation applies on write). Removed entities' + files are left on disk until `reset`. This fixes today's silent data-loss footgun. +- Keyed per app: if `.base44/data/meta.json` records a different `appId` than the + linked app, warn and offer `--fresh` (protects against relinking a folder). +- `base44 create` templates and `base44 link` add `.base44/` to `.gitignore`. +- Compaction runs on startup (`persistence.compactDatafile`); local scale makes + journal growth a non-issue in practice. + +Because state is project-relative, **every git worktree automatically gets isolated +data** — no flags, no config. This is the data-isolation half of the docker-style +`dev` envs direction; the `dev.json` descriptor (§5) is the discovery half. + +### 2. Seed source: declarative fixtures in `base44/seed/` + programmatic `base44/seed.ts` + +Seeds are **both** declarative fixtures and an optional programmatic script, run in +that order (both shipped). Fixtures live in a new committed directory (configurable +as `seedDir`, default `"seed"`, alongside `entitiesDir`/`functionsDir` in +`base44/config.jsonc`): + +``` +base44/seed/ +├── users.jsonc # test app users, created through the real local auth path +├── Task.jsonc # records for entity "Task" (filename = entity name, like entities/) +└── Project.jsonc +``` + +`users.jsonc` — solves the "role-gated app is unusable locally" problem first-class: + +```jsonc +[ + { "email": "admin@example.com", "role": "admin", "password": "admin1234", "full_name": "Ada Admin" }, + { "email": "user@example.com", "role": "user", "password": "user1234" } + // extra keys = custom User entity fields, validated against the merged User schema +] +``` + +Users are created through the same code path as local registration (password hashed +into `$user`, verified, role respected — seeding is privileged, so `role: "admin"` +is allowed). The CLI-logged-in user keeps being auto-created as admin, unchanged. + +`.jsonc` — an array of records validated against the entity schema: + +```jsonc +[ + { + "id": "seed-task-1", // optional; stable id => upsert on re-seed + "title": "Ship the seed feature", + "status": "in_progress", + "created_by": "user@example.com" // optional; attributes the record to a seeded user (RLS testing) + } +] +``` + +Semantics: + +- Applied with the **service role** (bypasses RLS — it must, or you couldn't seed + other users' rows), after schema load, users before entities. +- Validation reuses the dev server's `Validator` (`prepareRecord` + `validate`); + errors report file + record index. A fixture for an unknown entity is a warning, + not a failure (glob/no-match warning, per Supabase). +- **Idempotency contract:** records with an explicit `id` are upserted; records + without one are inserted only by a run that starts from empty (first boot, reset, + `--replace`). The scaffolded template ships with explicit ids so agents copy the + idempotent pattern. +- Fixtures are plain data — reviewable, diffable, and writable by agents without + running anything. Zod schemas for both file formats live in `core/`. + +Why declarative-first rather than only a TS seed script: fixtures need no extra +runtime (a `seed.ts` needs Deno, which is otherwise only required when the app has +functions), they match the existing JSONC resource convention +(`base44/entities/.jsonc`), and they cover the observed asks (test users + +baseline records + frozen remote snapshots). **Shipped change:** the programmatic +hook did not stay on the roadmap — at user request it shipped alongside fixtures, +because remote access should be **programmable-first rather than pull-only**. +`base44/seed.ts` runs after fixtures in the existing Deno runtime (exec-wrapper +pattern) with a service-role SDK client bound to the local server (`ctx.base44`) +and a remote client factory (`ctx.remote({ dataEnv })`) for the +filter/transform/generate cases fixtures can't express. Deno is required only for +this step: without it, fixtures still apply and the script step is reported as +failed. The two compose (fixtures, then script), as research predicted. + +### 3. Seed lifecycle + +``` +base44 dev # data dir empty (first run / after reset / --fresh)? + ├─ yes → apply seeds, record seedHash in meta.json + └─ no → leave data alone; if seed files changed since last apply, + log a hint: "seed files changed — run `base44 dev seed` to apply" + +base44 dev seed # apply seeds NOW (server running or not) + ├─ default: upsert-by-id (idempotent, non-destructive) + └─ --replace: truncate seeded collections first (non-TTY requires --force) + +base44 dev reset # THE canonical clean-slate command + └─ wipe data dir → reload schemas → apply seeds + (non-TTY requires --force; prints what was wiped/re-created) + +base44 dev --fresh # reset semantics fused into startup +``` + +Never silently re-seed existing data (lesson #2). Exactly one reset command +(lesson #4). + +### 4. Working against a running server + +`dev seed` / `dev reset` must work while `base44 dev` is running (agents will call +them mid-session). Two paths, one applier: + +- The seed/reset logic lives in `core/` and operates on the `Database` API + (`prepareRecord`/`validate`/insert, emitting realtime `create` events so open UIs + update live). +- The dev server exposes **local-only admin endpoints** (`POST /_base44/dev/seed`, + `POST /_base44/dev/reset`, `GET /_base44/dev/status`), bound to `127.0.0.1` like + everything else and authenticated with a per-instance token written into + `dev.json` (never accepted from the network). +- The CLI command reads `.base44/dev.json`: if a live instance is found (pid + port + check), it calls the endpoint; otherwise it opens the datastores directly. Same + observable result either way. + +This also removes the file-lock problem of two processes opening the same NeDB +files. + +### 5. Instance descriptor: `.base44/dev.json` + +Written on startup, removed on shutdown; stale entries detected by pid: + +```json +{ + "appId": "abc123", + "url": "http://localhost:4400", + "port": 4400, + "pid": 51234, + "dataDir": ".base44/data", + "adminToken": "…", + "startedAt": "2026-07-13T10:00:00Z", + "seed": { "hash": "sha256:…", "appliedAt": "2026-07-13T10:00:01Z" } +} +``` + +This is the machine-discoverable state agents need ("what URL is my env on, where +are the logs") and is the natural substrate for the planned docker-style +`dev ps` / `dev inspect` / `dev logs` commands — those read the same descriptor, +this proposal just introduces it. When the default port is taken, `dev` should +auto-pick a free one (parallel worktrees) and record it here. + +### 6. Remote bridge: programmable-first, `data pull` / `data dump` as sugar + +The single most-requested workflow ("develop against my real app's data") and +exactly what one user hand-rolled. As shipped, the **primary** bridge is +programmatic: `base44/seed.ts` gets `ctx.remote({ dataEnv })` — an SDK client +authenticated as the CLI user against the linked remote app (`dataEnv: "dev"` +targets the dev data environment via the `X-Data-Env` header) — so +filter/transform/subset pulls are ordinary code writing through `ctx.base44`. The +zero-code commands cover the common case: + +``` +base44 data pull [--entity ...] [--data-env prod|dev] [--query ] [--limit ] + └─ fetch records from the linked remote app → write seed fixtures + (read-only against remote; never writes to the remote DB) + +base44 data dump [--entity ...] + └─ local dev data → seed fixtures + ("I hand-crafted good data in the local UI — freeze it as the committed seed") +``` + +- `pull` pages the existing runtime entities list API via `getAppClient()` + (limit/skip, page size 500), honoring the platform's `DataEnvironment` selector + (`--data-env`, `X-Data-Env: dev` header) and an optional `--query` JSON filter. +- The API returns bare record arrays with no total count, so the reported `total` + always equals `pulled`; when pagination stops at `--limit` with a full last page, + the CLI notes "limit reached, more may exist". +- Ids are preserved on pull/dump, which makes the resulting fixtures idempotent by + construction (stable-id upsert). `dump` strips NeDB's internal `_id`. +- `dump` reads a running instance via the admin export endpoint, or the NeDB files + directly when none is running — and **never exports users in v1** (warn + skip): + user fixtures are `users.jsonc`-shaped (roles, passwords), not + entity-fixture-shaped. +- Default `--limit 1000` per entity — seed fixtures are for representative data, + not full replication. A raw NDJSON snapshot mode (import/export à la Convex) can + come later if needed. +- Sensitive data: pull is explicit and per-entity; an anonymization/subsetting story + is deliberately out of scope for v1 (documented; see Open questions). + +`dump` doubles as the answer to "Studio-made data dies on reset" (Supabase's +recurring surprise): hand-made local data becomes a committed artifact in one +command. + +### 7. Command surface & conventions + +New commands follow the existing factory pattern (`Base44Command`, `runTask`, +non-interactive guards) and the global `--json` contract. Final surface as shipped: + +| Command | JSON stdout (shape as shipped) | +|---|---| +| `base44 dev` | unchanged default action + `--fresh` (wipe data dir before load) | +| `base44 dev status` | `dev.json` minus `adminToken`/`pid`, plus `running: bool` (`{ "running": false }` when no instance) | +| `base44 dev seed [--replace] [--force]` | `{ "applied": true, "mode": "upsert", "users": 2, "records": { "Task": { "created": 10, "updated": 2, "skipped": 0 } }, "script": { "ran": true } \| null, "warnings": [] }` | +| `base44 dev reset [--force]` | `{ "reset": true, "seeded": true, "dataDir": "…", "seed": \| null }` | +| `base44 data pull [--entity ] [--data-env prod\|dev] [--query ] [--limit ] [--out ] [--force]` | `{ "entities": { "Task": { "pulled": 120, "total": 120 } }, "wrote": ["…/seed/task.jsonc"] }` | +| `base44 data dump [--entity ] [--out ] [--force]` | same shape as pull; user records are never dumped (warn + skip) | + +Destructive ops (`reset`, `seed --replace`, overwriting existing fixture files with +`pull`/`dump`) prompt in a TTY and require `--force` otherwise. `dev` is now a +command group with a default action, preserving plain `base44 dev` (and leaving +room for `run -d` / `ps` / `logs` / `inspect` later). + +Rollout phases (1–3 shipped; `base44/seed.ts` moved up from phase 4 — see status +update): + +1. **Persistence + lifecycle** — `.base44/` state dir, file-backed NeDB, `--fresh`, + keep-data-on-schema-change, `dev.json`, `dev status`, `.gitignore` templating. +2. **Seeding** — `base44/seed/` fixtures (users + entities), auto-seed on empty, + `dev seed`, `dev reset`, admin endpoints, scaffolded example seed in + `base44 create` templates + docs/skill updates. +3. **Remote bridge** — `data pull` / `data dump`. +4. **Later** — programmatic `base44/seed.ts` (Deno + service-role SDK client), + deterministic fake-data generation (seeded pRNG, drizzle-seed-style), + anonymization on pull, snapshot import/export, docker-style env lifecycle + commands built on `dev.json`. + +Phases 1+2 are the MVP: they fix data loss and make role-gated/RLS apps usable +locally with zero remote dependency. + +### How this maps to the reported problems + +| Report | Answer | +|---|---| +| "Local entity DB is in-memory and clears on restart; no fixtures/persist/import path" | Persistent by default (§1); `base44/seed/` fixtures (§2); `data pull` (§6) | +| "Can't get a usable local session for a role-gated, RLS-driven SPA" | `users.jsonc` seeded through the real auth path with roles; `created_by` attribution for RLS-shaped fixtures (§2) | +| "I wrote a script that pulls from remote DB and seeds in-memory on every run" | `base44 data pull` once → committed fixtures → auto-seed (§3, §6); pull stays read-only against remote | +| Docker-style multi-agent envs (detached, ps/logs/inspect, isolation) | Project-relative state = per-worktree data isolation; `dev.json` descriptor + `dev status` as the substrate (§1, §5) | +| asServiceRole broken locally (cli#491) | Seeding runs as a real local service token; fixing the function-proxy header injection falls out of the same work and should ride along | + +## Open questions + +1. **Command naming — resolved (shipped):** `base44 dev seed/reset/status` under + the `dev` group, `data pull/dump` top-level, as proposed — seeds are a local-dev + concept, `data` touches the remote app. Still open for review: issue #285 + sketches `base44 entities record list`; if an `entities record` namespace lands, + `data` vs `entities record` naming must be reconciled. +2. **Seed users vs production parity:** locally, seeded users can hold any role; + there is no equivalent remote operation. Acceptable asymmetry, or should + `users.jsonc` be explicitly documented as local-only? (Proposed: local-only, + documented.) +3. **Pull auth semantics:** pull as the platform user via the app-scoped client — + should it require an explicit `--include-user-data` style acknowledgment when + entities contain user PII? (v1: no, but log the entity list and counts loudly.) +4. **`dev --fresh` vs `dev reset` overlap:** keep both (start-time flag + standalone + command) or only the command? (Proposed: both; agents restarting an env want the + one-shot flag.) +5. **Large datasets:** fixtures are JSONC in git; at what size do we push users + toward a future snapshot format? (Proposed guardrail: warn above ~1 MB per + fixture file.) diff --git a/docs/testing.md b/docs/testing.md index 3a81485a1..4555b4eca 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -71,6 +71,7 @@ tests/ ├── function-discovery-entry-at-root/ # Error: entry at root ├── duplicate-function-names/ # Error: duplicate function names ├── with-zero-config-functions/ # Full project: zero-config + path-named functions (CLI integration) + ├── with-seed/ # Project with seed fixtures (users.jsonc + entity fixtures) for dev seed/reset and data dump tests ├── with-site/ # Project with site config ├── full-project/ # All resources combined ├── no-app-config/ # Unlinked project (no .app.jsonc) @@ -139,6 +140,11 @@ await t.givenProject(fixture("with-entities")); // Mock the npm version check (null = no upgrade available, string = upgrade available) t.givenLatestVersion(null); // Default: no upgrade notification t.givenLatestVersion("2.0.0"); // Simulate upgrade available + +// Fake the base44/seed.ts Deno run (no Deno needed): the runner skips +// spawning and reports this exit code on the seed summary +t.givenSeedScriptResult(0); // Script "succeeds" +t.givenSeedScriptResult(1); // Script "fails" — summary gets script: { ran: false } ``` ### When (Execute) @@ -317,6 +323,7 @@ For behaviors that can't be mocked via the API server (like filesystem-based con **Current overrides:** - `appConfig` -- Mock app configuration (id, projectRoot). Set automatically by `givenProject()` - `latestVersion` -- Mock version check response (string for newer version, null for no update). Defaults to `null` +- `seedScript` -- Fake the `base44/seed.ts` Deno run: `runSeedScript` skips spawning and returns `{ exitCode }`. Set via `t.givenSeedScriptResult(exitCode)`. (`runSeedScript` also has `spawnImpl`/`wrapperPath` injection seams for core unit tests.) ### Adding a New Override diff --git a/packages/cli/deno-runtime/seed.ts b/packages/cli/deno-runtime/seed.ts new file mode 100644 index 000000000..fe97d9b9e --- /dev/null +++ b/packages/cli/deno-runtime/seed.ts @@ -0,0 +1,106 @@ +/** + * Deno Seed Wrapper + * + * Executed by Deno to run a project's `base44/seed.ts` programmatic seed + * hook. Builds a `ctx` object and calls the script's default export with it: + * + * - `ctx.base44` — SDK client bound to the LOCAL dev server as service role + * (bypasses RLS/FLS; the local server resolves the service-subject JWT in + * the Authorization header to its service principal). + * - `ctx.remote({ dataEnv? })` — factory for SDK clients authenticated as the + * CLI user against the linked REMOTE app. `dataEnv: "dev"` targets the dev + * data environment via the `X-Data-Env` header. + * - `ctx.log(msg)` — stderr logger (stdout is reserved for the CLI). + * + * Environment variables: + * - SCRIPT_PATH: file:// URL of the user's seed script + * - BASE44_APP_ID: App identifier + * - BASE44_LOCAL_URL: Base URL of the running local dev server + * - BASE44_LOCAL_SERVICE_TOKEN: Local service-role JWT + * - BASE44_ACCESS_TOKEN: Remote app-user token (may be empty) + * - BASE44_APP_BASE_URL: Remote app's published URL (may be empty) + * - BASE44_REMOTE_ERROR: Reason remote credentials are unavailable, if any + */ + +export {}; + +const scriptPath = Deno.env.get("SCRIPT_PATH"); +const appId = Deno.env.get("BASE44_APP_ID"); +const localUrl = Deno.env.get("BASE44_LOCAL_URL"); +const localServiceToken = Deno.env.get("BASE44_LOCAL_SERVICE_TOKEN"); +const remoteAccessToken = Deno.env.get("BASE44_ACCESS_TOKEN"); +const remoteAppBaseUrl = Deno.env.get("BASE44_APP_BASE_URL"); +const remoteError = Deno.env.get("BASE44_REMOTE_ERROR"); + +if (!scriptPath) { + console.error("SCRIPT_PATH environment variable is required"); + Deno.exit(1); +} + +if (!appId || !localUrl || !localServiceToken) { + console.error( + "BASE44_APP_ID, BASE44_LOCAL_URL, and BASE44_LOCAL_SERVICE_TOKEN are required", + ); + Deno.exit(1); +} + +import { createClient } from "npm:@base44/sdk"; + +const base44 = createClient({ + appId, + serverUrl: localUrl, + token: localServiceToken, +}); + +// Track every client created so we can clean them all up (clears analytics +// heartbeat intervals, disconnects sockets) and let the process exit. +const clients: { cleanup: () => void }[] = [base44]; + +interface RemoteOptions { + dataEnv?: "prod" | "dev"; +} + +function remote(options?: RemoteOptions) { + const dataEnv = options?.dataEnv ?? "prod"; + if (dataEnv !== "prod" && dataEnv !== "dev") { + throw new Error(`Invalid dataEnv "${dataEnv}": expected "prod" or "dev"`); + } + if (!remoteAccessToken || !remoteAppBaseUrl) { + throw new Error( + `Remote app credentials are unavailable${remoteError ? `: ${remoteError}` : ""}`, + ); + } + const client = createClient({ + appId, + serverUrl: remoteAppBaseUrl, + token: remoteAccessToken, + ...(dataEnv === "dev" ? { headers: { "X-Data-Env": "dev" } } : {}), + }); + clients.push(client); + return client; +} + +const ctx = { + base44, + remote, + log: (message: unknown) => console.error(message), +}; + +try { + const module = await import(scriptPath); + const seed = module?.default; + if (typeof seed !== "function") { + console.error( + "Seed script must have a default export function: export default async function seed(ctx) { ... }", + ); + Deno.exit(1); + } + await seed(ctx); +} catch (error) { + console.error("Seed script failed:", error); + Deno.exit(1); +} finally { + for (const client of clients) { + client.cleanup(); + } +} diff --git a/packages/cli/infra/build.ts b/packages/cli/infra/build.ts index 26dbaf07b..d2837ed26 100644 --- a/packages/cli/infra/build.ts +++ b/packages/cli/infra/build.ts @@ -30,6 +30,7 @@ const copyDenoRuntime = () => { mkdirSync(outDir, { recursive: true }); copyFileSync("./deno-runtime/main.ts", `${outDir}/main.ts`); copyFileSync("./deno-runtime/exec.ts", `${outDir}/exec.ts`); + copyFileSync("./deno-runtime/seed.ts", `${outDir}/seed.ts`); return outDir; }; diff --git a/packages/cli/src/cli/commands/data/dump.ts b/packages/cli/src/cli/commands/data/dump.ts new file mode 100644 index 000000000..918fd5571 --- /dev/null +++ b/packages/cli/src/cli/commands/data/dump.ts @@ -0,0 +1,129 @@ +import type { Command } from "commander"; +import { + buildDataJsonOutput, + type DataResultEntry, + resolveOutDir, + resolveRequestedEntities, + writeFixtureFiles, +} from "@/cli/commands/data/shared.js"; +import { + exportViaInstance, + openOfflineDatabase, + requireDevProject, +} from "@/cli/commands/dev/seed-shared.js"; +import { exportCollections } from "@/cli/dev/dev-server/db/export.js"; +import type { CLIContext, RunCommandResult } from "@/cli/types.js"; +import { Base44Command } from "@/cli/utils/index.js"; +import { readDevInstance } from "@/core/local-state/index.js"; +import { readProjectConfig } from "@/core/project/config.js"; +import { normalizeSeedName } from "@/core/resources/seed/index.js"; + +interface DataDumpOptions { + entity?: string[]; + out?: string; + force?: boolean; +} + +const USER_SKIPPED_WARNING = + "User records are not dumped — user fixtures are users.jsonc-shaped; skipping User"; + +async function dataDumpAction( + ctx: CLIContext, + options: DataDumpOptions, +): Promise { + const { log, app, jsonMode, isNonInteractive } = ctx; + const project = requireDevProject(app, "data dump"); + const projectData = await readProjectConfig(project.projectRoot); + + // User dumps are out of scope (v1): user fixtures carry passwords/roles in + // users.jsonc shape, not entity-fixture shape. + const warnings: string[] = []; + const explicit = (options.entity?.length ?? 0) > 0; + let requested: string[] | undefined; + if (explicit) { + const kept = (options.entity ?? []).filter( + (name) => normalizeSeedName(name) !== "user", + ); + if (kept.length < (options.entity?.length ?? 0)) { + warnings.push(USER_SKIPPED_WARNING); + } + // Validates names and canonicalizes them to entity display names. + requested = + kept.length > 0 + ? resolveRequestedEntities( + projectData.entities.filter( + (entity) => normalizeSeedName(entity.name) !== "user", + ), + kept, + ).map((entity) => entity.name) + : []; + } + + const instance = await readDevInstance(project.projectRoot); + let collections: Record[]> = {}; + if (!explicit || (requested?.length ?? 0) > 0) { + if (instance) { + collections = (await exportViaInstance(instance, requested)).collections; + } else { + const { db } = await openOfflineDatabase(project, projectData); + collections = await exportCollections(db, requested); + } + } + + // Skip empty collections unless the entity was explicitly requested. + const results: (DataResultEntry & { records: unknown[] })[] = Object.entries( + collections, + ) + .filter(([, records]) => explicit || records.length > 0) + .map(([entityName, records]) => ({ + entityName, + records, + pulled: records.length, + total: records.length, + })); + + const outDir = resolveOutDir(projectData.project, options.out); + const wrote = await writeFixtureFiles({ + outDir, + entries: results, + force: options.force === true, + isNonInteractive, + }); + + for (const warning of warnings) { + log.warn(warning); + } + + const outroMessage = `Wrote ${wrote.length} fixture file(s) to ${outDir}`; + + if (jsonMode) { + return { outroMessage, stdout: buildDataJsonOutput(results, wrote) }; + } + + if (results.length > 0) { + log.info( + results + .map( + (result) => + `${result.entityName}: pulled ${result.pulled} of ${result.total}`, + ) + .join("\n"), + ); + } + return { outroMessage }; +} + +export function getDataDumpCommand(): Command { + return new Base44Command("dump") + .description("Dump local dev data into seed fixtures") + .option( + "--entity ", + "Only dump these entities (default: all non-empty collections)", + ) + .option( + "--out ", + "Output directory (default: the project's seed directory)", + ) + .option("--force", "Overwrite existing fixture files without confirming") + .action(dataDumpAction); +} diff --git a/packages/cli/src/cli/commands/data/index.ts b/packages/cli/src/cli/commands/data/index.ts new file mode 100644 index 000000000..503fe7bdd --- /dev/null +++ b/packages/cli/src/cli/commands/data/index.ts @@ -0,0 +1,10 @@ +import { Command } from "commander"; +import { getDataDumpCommand } from "./dump.js"; +import { getDataPullCommand } from "./pull.js"; + +export function getDataCommand(): Command { + return new Command("data") + .description("Move data between the remote app, local dev, and fixtures") + .addCommand(getDataPullCommand()) + .addCommand(getDataDumpCommand()); +} diff --git a/packages/cli/src/cli/commands/data/pull.ts b/packages/cli/src/cli/commands/data/pull.ts new file mode 100644 index 000000000..89d82d1e5 --- /dev/null +++ b/packages/cli/src/cli/commands/data/pull.ts @@ -0,0 +1,149 @@ +import type { Command } from "commander"; +import { Option } from "commander"; +import { + buildDataJsonOutput, + type DataResultEntry, + resolveOutDir, + resolveRequestedEntities, + writeFixtureFiles, +} from "@/cli/commands/data/shared.js"; +import { requireDevProject } from "@/cli/commands/dev/seed-shared.js"; +import type { CLIContext, RunCommandResult } from "@/cli/types.js"; +import { Base44Command } from "@/cli/utils/index.js"; +import { InvalidInputError } from "@/core/errors.js"; +import { readProjectConfig } from "@/core/project/config.js"; +import { + type DataEnv, + fetchEntityRecords, +} from "@/core/resources/entity/index.js"; + +const DEFAULT_LIMIT = 1000; + +interface DataPullOptions { + entity?: string[]; + dataEnv: DataEnv; + query?: string; + limit?: string; + out?: string; + force?: boolean; +} + +function parseQueryOption( + query: string | undefined, +): Record | undefined { + if (query === undefined) { + return undefined; + } + let parsed: unknown; + try { + parsed = JSON.parse(query); + } catch { + throw new InvalidInputError("--query must be valid JSON"); + } + if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) { + throw new InvalidInputError( + '--query must be a JSON object, e.g. \'{"status": "open"}\'', + ); + } + return parsed as Record; +} + +function parseLimitOption(limit: string | undefined): number { + if (limit === undefined) { + return DEFAULT_LIMIT; + } + const parsed = Number.parseInt(limit, 10); + if (Number.isNaN(parsed) || parsed <= 0) { + throw new InvalidInputError("--limit must be a positive integer"); + } + return parsed; +} + +async function dataPullAction( + ctx: CLIContext, + options: DataPullOptions, +): Promise { + const { log, app, jsonMode, isNonInteractive } = ctx; + const project = requireDevProject(app, "data pull"); + + const query = parseQueryOption(options.query); + const limit = parseLimitOption(options.limit); + + const projectData = await readProjectConfig(project.projectRoot); + const entities = resolveRequestedEntities( + projectData.entities, + options.entity, + ); + + const results: (DataResultEntry & { + records: unknown[]; + limitReached: boolean; + })[] = []; + for (const entity of entities) { + const { records, limitReached } = await fetchEntityRecords(entity.name, { + dataEnv: options.dataEnv, + query, + limit, + }); + results.push({ + entityName: entity.name, + records, + limitReached, + pulled: records.length, + total: records.length, + }); + } + + const outDir = resolveOutDir(projectData.project, options.out); + const wrote = await writeFixtureFiles({ + outDir, + entries: results, + force: options.force === true, + isNonInteractive, + }); + + const outroMessage = `Wrote ${wrote.length} fixture file(s) to ${outDir}`; + + if (jsonMode) { + return { outroMessage, stdout: buildDataJsonOutput(results, wrote) }; + } + + log.info( + results + .map( + (result) => + `${result.entityName}: pulled ${result.pulled} of ${result.total}${ + result.limitReached ? " (limit reached, more may exist)" : "" + }`, + ) + .join("\n"), + ); + return { outroMessage }; +} + +export function getDataPullCommand(): Command { + return new Base44Command("pull") + .description( + "Pull entity records from the linked remote app into seed fixtures", + ) + .option( + "--entity ", + "Only pull these entities (default: all project entities)", + ) + .addOption( + new Option("--data-env ", "Remote data environment to read from") + .choices(["prod", "dev"]) + .default("prod"), + ) + .option("--query ", "Filter records with a JSON query") + .option( + "--limit ", + `Maximum records to pull per entity (default: ${DEFAULT_LIMIT})`, + ) + .option( + "--out ", + "Output directory (default: the project's seed directory)", + ) + .option("--force", "Overwrite existing fixture files without confirming") + .action(dataPullAction); +} diff --git a/packages/cli/src/cli/commands/data/shared.ts b/packages/cli/src/cli/commands/data/shared.ts new file mode 100644 index 000000000..91e259f98 --- /dev/null +++ b/packages/cli/src/cli/commands/data/shared.ts @@ -0,0 +1,116 @@ +import { dirname, join, resolve } from "node:path"; +import { confirmDestructiveAction } from "@/cli/commands/dev/seed-shared.js"; +import { InvalidInputError } from "@/core/errors.js"; +import type { ProjectData } from "@/core/project/types.js"; +import type { Entity } from "@/core/resources/entity/index.js"; +import { normalizeSeedName } from "@/core/resources/seed/index.js"; +import { pathExists, writeFile } from "@/core/utils/fs.js"; + +/** `TeamMember` → `team-member` (seed fixture file naming). */ +export function kebabCaseEntityName(name: string): string { + return name + .replace(/([a-z0-9])([A-Z])/g, "$1-$2") + .replace(/[\s_]+/g, "-") + .toLowerCase(); +} + +/** + * Resolve `--entity` names against the project's entities (normalized + * comparison, so `--entity task` matches `Task`). No names = all entities. + */ +export function resolveRequestedEntities( + available: Entity[], + requested: string[] | undefined, +): Entity[] { + if (!requested?.length) { + return available; + } + return requested.map((name) => { + const match = available.find( + (entity) => normalizeSeedName(entity.name) === normalizeSeedName(name), + ); + if (!match) { + throw new InvalidInputError( + `Unknown entity "${name}". Known entities: ${ + available.map((entity) => entity.name).join(", ") || "none" + }`, + ); + } + return match; + }); +} + +/** Default fixture output dir: `/`, or `--out`. */ +export function resolveOutDir( + project: ProjectData["project"], + out: string | undefined, +): string { + return out + ? resolve(out) + : join(dirname(project.configPath), project.seedDir); +} + +export interface FixtureEntry { + entityName: string; + records: unknown[]; +} + +/** + * Write one `.jsonc` per entry (pretty JSON array). + * Existing files require `--force` (or one TTY confirm covering all). + */ +export async function writeFixtureFiles(options: { + outDir: string; + entries: FixtureEntry[]; + force: boolean; + isNonInteractive: boolean; +}): Promise { + const targets = options.entries.map((entry) => ({ + ...entry, + path: join( + options.outDir, + `${kebabCaseEntityName(entry.entityName)}.jsonc`, + ), + })); + + const existing: string[] = []; + for (const target of targets) { + if (await pathExists(target.path)) { + existing.push(target.path); + } + } + if (existing.length > 0) { + await confirmDestructiveAction( + options.isNonInteractive, + options.force, + `Overwrite ${existing.length} existing fixture file(s) in ${options.outDir}?`, + "--force is required to overwrite existing fixture files in non-interactive mode", + ); + } + + for (const target of targets) { + await writeFile( + target.path, + `${JSON.stringify(target.records, null, 2)}\n`, + ); + } + return targets.map((target) => target.path); +} + +export interface DataResultEntry { + entityName: string; + pulled: number; + total: number; +} + +/** `--json` stdout shape shared by `data pull` and `data dump`. */ +export function buildDataJsonOutput( + entries: DataResultEntry[], + wrote: string[], +): string { + const entities: Record = {}; + for (const entry of entries) { + entities[entry.entityName] = { pulled: entry.pulled, total: entry.total }; + } + return `${JSON.stringify({ entities, wrote }, null, 2)}\n`; +} diff --git a/packages/cli/src/cli/commands/dev.ts b/packages/cli/src/cli/commands/dev/index.ts similarity index 73% rename from packages/cli/src/cli/commands/dev.ts rename to packages/cli/src/cli/commands/dev/index.ts index cfcb7d8e3..4bc93509c 100644 --- a/packages/cli/src/cli/commands/dev.ts +++ b/packages/cli/src/cli/commands/dev/index.ts @@ -1,4 +1,7 @@ import type { Command } from "commander"; +import { getDevResetCommand } from "@/cli/commands/dev/reset.js"; +import { getDevSeedCommand } from "@/cli/commands/dev/seed.js"; +import { getDevStatusCommand } from "@/cli/commands/dev/status.js"; import { createDevServer } from "@/cli/dev/dev-server/main.js"; import type { CLIContext, RunCommandResult } from "@/cli/types.js"; import { type AppIdOptions, Base44Command, theme } from "@/cli/utils/index.js"; @@ -10,6 +13,7 @@ import { readProjectConfig } from "@/core/project/config.js"; interface DevOptions { port?: string; + fresh?: boolean; } function localServerUrl(port: number): string { @@ -44,6 +48,7 @@ async function devAction( log, port, appId, + state: { projectRoot: app.projectRoot, fresh: options.fresh === true }, denoWrapperPath: getDenoWrapperPath(), loadResources: async () => { const { functions, entities, project } = await readProjectConfig(); @@ -63,6 +68,16 @@ export function getDevCommand(): Command { return new Base44Command("dev") .description("Start the development server") .option("-p, --port ", "Port for the development server") - .hook("preAction", validateDevOptions) - .action(devAction); + .option("--fresh", "Delete local data before starting") + .hook("preAction", (thisCommand, actionCommand) => { + // The hook also fires for subcommands (e.g. `dev status`); the + // --app-id restriction only applies to the default (server) action. + if (thisCommand === actionCommand) { + validateDevOptions(thisCommand); + } + }) + .action(devAction) + .addCommand(getDevStatusCommand()) + .addCommand(getDevSeedCommand()) + .addCommand(getDevResetCommand()); } diff --git a/packages/cli/src/cli/commands/dev/reset.ts b/packages/cli/src/cli/commands/dev/reset.ts new file mode 100644 index 000000000..49482c999 --- /dev/null +++ b/packages/cli/src/cli/commands/dev/reset.ts @@ -0,0 +1,63 @@ +import type { Command } from "commander"; +import type { CLIContext, RunCommandResult } from "@/cli/types.js"; +import { Base44Command } from "@/cli/utils/index.js"; +import { readDevInstance } from "@/core/local-state/index.js"; +import { + confirmDestructiveAction, + logSeedSummary, + requireDevProject, + resetOffline, + resetViaInstance, +} from "./seed-shared.js"; + +interface DevResetOptions { + force?: boolean; +} + +async function devResetAction( + ctx: CLIContext, + options: DevResetOptions, +): Promise { + const { log, app, jsonMode, isNonInteractive } = ctx; + const project = requireDevProject(app, "dev reset"); + + await confirmDestructiveAction( + isNonInteractive, + options.force === true, + "This deletes ALL local dev data and re-applies seeds. Continue?", + "--force is required to reset in non-interactive mode", + ); + + const instance = await readDevInstance(project.projectRoot); + const result = instance + ? await resetViaInstance(instance) + : await resetOffline(project, log); + + // Reset + fixtures succeeded but seed.ts failed: report, exit non-zero. + if (result.seed?.script?.ran === false) { + process.exitCode = 1; + } + + const outroMessage = result.seeded + ? "Local data reset and seeds applied" + : "Local data reset"; + + if (jsonMode) { + return { + outroMessage, + stdout: `${JSON.stringify(result, null, 2)}\n`, + }; + } + + if (result.seed) { + logSeedSummary(log, result.seed); + } + return { outroMessage }; +} + +export function getDevResetCommand(): Command { + return new Base44Command("reset") + .description("Wipe the local dev database and re-apply seeds") + .option("--force", "Skip the confirmation prompt") + .action(devResetAction); +} diff --git a/packages/cli/src/cli/commands/dev/seed-shared.ts b/packages/cli/src/cli/commands/dev/seed-shared.ts new file mode 100644 index 000000000..c4c84c85e --- /dev/null +++ b/packages/cli/src/cli/commands/dev/seed-shared.ts @@ -0,0 +1,343 @@ +import type { Logger } from "@base44-cli/logger"; +import { cancel, confirm, isCancel } from "@clack/prompts"; +import getPort from "get-port"; +import { z } from "zod"; +import type { DevLogger } from "@/cli/dev/createDevLogger.js"; +import { Database } from "@/cli/dev/dev-server/db/database.js"; +import { applySeeds } from "@/cli/dev/dev-server/db/seed.js"; +import { createDevServer } from "@/cli/dev/dev-server/main.js"; +import { + DEV_ADMIN_BASE_PATH, + DEV_ADMIN_HEADER, +} from "@/cli/dev/dev-server/routes/admin-router.js"; +import { formatSeedCounts } from "@/cli/dev/seed-summary.js"; +import { CLIExitError } from "@/cli/errors.js"; +import { getDenoWrapperPath } from "@/core/assets.js"; +import { + ApiError, + ConfigInvalidError, + InvalidInputError, + SchemaValidationError, +} from "@/core/errors.js"; +import { + type DevInstance, + getDataDir, + getMetaJsonPath, + readDataDirMeta, + writeDataDirMeta, +} from "@/core/local-state/index.js"; +import type { AppContext } from "@/core/project/app-config.js"; +import { readProjectConfig } from "@/core/project/config.js"; +import type { ProjectData } from "@/core/project/types.js"; +import { + type DevResetResult, + DevResetResultSchema, + emptySeedSummary, + readSeedFiles, + type SeedMode, + type SeedSummary, + SeedSummarySchema, +} from "@/core/resources/seed/index.js"; + +export interface DevProjectContext { + id: string; + projectRoot: string; +} + +export function requireDevProject( + app: AppContext | undefined, + commandName: string, +): DevProjectContext { + if (!app?.projectRoot) { + throw new ConfigInvalidError( + `base44 ${commandName} requires a linked local project. Run it from a project with base44/.app.jsonc.`, + ); + } + return { id: app.id, projectRoot: app.projectRoot }; +} + +/** + * Gate a destructive dev-data operation: `--force` skips, TTY prompts, + * non-interactive without `--force` fails. + */ +export async function confirmDestructiveAction( + isNonInteractive: boolean, + force: boolean, + message: string, + forceHint: string, +): Promise { + if (force) { + return; + } + if (isNonInteractive) { + throw new InvalidInputError(forceHint); + } + const confirmed = await confirm({ message }); + if (isCancel(confirmed) || !confirmed) { + cancel("Operation cancelled."); + throw new CLIExitError(0); + } +} + +interface AdminRequest { + method: "GET" | "POST"; + path: string; + body?: unknown; +} + +/** Call an admin endpoint on the running dev server, Zod-parse the response. */ +async function callAdminEndpoint( + instance: DevInstance, + request: AdminRequest, + schema: Schema, +): Promise> { + const url = `${instance.url}${DEV_ADMIN_BASE_PATH}${request.path}`; + let response: Response; + try { + response = await fetch(url, { + method: request.method, + headers: { + "content-type": "application/json", + [DEV_ADMIN_HEADER]: instance.adminToken, + }, + body: + request.body === undefined ? undefined : JSON.stringify(request.body), + }); + } catch (error) { + throw new ApiError(`Failed to reach the dev server at ${instance.url}`, { + requestUrl: url, + cause: error instanceof Error ? error : undefined, + }); + } + + const responseBody: unknown = await response.json().catch(() => undefined); + if (!response.ok) { + const message = + (responseBody as { error?: string } | undefined)?.error ?? + `Dev server responded with status ${response.status}`; + throw new ApiError(message, { + statusCode: response.status, + requestUrl: url, + responseBody, + }); + } + + const result = schema.safeParse(responseBody); + if (!result.success) { + throw new SchemaValidationError( + "Invalid response from the dev server", + result.error, + ); + } + return result.data; +} + +export async function seedViaInstance( + instance: DevInstance, + mode: SeedMode, +): Promise { + return await callAdminEndpoint( + instance, + { method: "POST", path: "/seed", body: { mode } }, + SeedSummarySchema, + ); +} + +export async function resetViaInstance( + instance: DevInstance, +): Promise { + return await callAdminEndpoint( + instance, + { method: "POST", path: "/reset", body: {} }, + DevResetResultSchema, + ); +} + +const DevExportSchema = z.object({ + collections: z.record(z.string(), z.array(z.record(z.string(), z.unknown()))), +}); + +export type DevExport = z.infer; + +/** Fetch local collections from a running dev server (`data dump` live path). */ +export async function exportViaInstance( + instance: DevInstance, + entityNames?: string[], +): Promise { + const query = entityNames?.length + ? `?entities=${encodeURIComponent(entityNames.join(","))}` + : ""; + return await callAdminEndpoint( + instance, + { method: "GET", path: `/export${query}` }, + DevExportSchema, + ); +} + +export interface OfflineDatabase { + db: Database; + dataDir: string; +} + +/** + * Open the project's local datastore directly (no dev server running), + * guarding against data that belongs to a different app — same rule as + * `base44 dev` startup. + */ +export async function openOfflineDatabase( + app: DevProjectContext, + projectData: ProjectData, +): Promise { + const dataDir = getDataDir(app.projectRoot); + + const meta = await readDataDirMeta(dataDir); + if (meta.status === "ok" && meta.meta.appId !== app.id) { + throw new ConfigInvalidError( + `Local dev data in ${dataDir} belongs to app "${meta.meta.appId}", but this project is linked to app "${app.id}".`, + getMetaJsonPath(dataDir), + { + hints: [ + { + message: + "Run 'base44 dev --fresh' to delete the local data and start clean", + command: "base44 dev --fresh", + }, + ], + }, + ); + } + + const db = new Database({ dataDir }); + await db.load(projectData.entities); + return { db, dataDir }; +} + +/** DevLogger that writes to stderr, keeping stdout pure under `--json`. */ +function stderrDevLogger(): DevLogger { + const write = (args: unknown[]) => + process.stderr.write(`${args.map(String).join(" ")}\n`); + return { + log: (...args) => write(args), + warn: (...args) => write(args), + error: (msg, err) => write(err === undefined ? [msg] : [msg, err]), + }; +} + +/** + * Boot a temporary internal dev-server instance on a random port so the + * `seed.ts` script step has a server to talk to when none is running. + * Ephemeral: no dev.json descriptor, no startup auto-seed (the callback + * drives seeding through the admin endpoints), no frontend serve command. + */ +export async function withTempDevInstance( + app: DevProjectContext, + log: Logger, + fn: (instance: DevInstance) => Promise, +): Promise { + const { url, port, adminToken, shutdown } = await createDevServer({ + log, + port: await getPort(), + appId: app.id, + state: { projectRoot: app.projectRoot, ephemeral: true }, + denoWrapperPath: getDenoWrapperPath(), + logger: stderrDevLogger(), + loadResources: async () => { + const { functions, entities, project } = await readProjectConfig( + app.projectRoot, + ); + // Never launch the project's frontend from a temporary instance. + const site = project.site + ? { ...project.site, serveCommand: undefined } + : project.site; + return { functions, entities, project: { ...project, site } }; + }, + }); + + try { + return await fn({ + appId: app.id, + url, + port, + pid: process.pid, + dataDir: getDataDir(app.projectRoot), + adminToken, + startedAt: new Date().toISOString(), + seed: null, + }); + } finally { + await shutdown(); + } +} + +export async function seedOffline( + app: DevProjectContext, + mode: SeedMode, + log: Logger, +): Promise { + const projectData = await readProjectConfig(app.projectRoot); + const seedData = await readSeedFiles(projectData.project); + if (!seedData) { + return emptySeedSummary(mode); + } + + if (seedData.scriptPath) { + // seed.ts talks to the dev server over HTTP: boot a temporary instance + // and drive fixtures + script through the same path as a live server. + return await withTempDevInstance(app, log, (instance) => + seedViaInstance(instance, mode), + ); + } + + const { db, dataDir } = await openOfflineDatabase(app, projectData); + const summary = await applySeeds(db, seedData, { mode }); + await writeDataDirMeta(dataDir, { + formatVersion: 1, + appId: app.id, + seed: { hash: seedData.hash, appliedAt: new Date().toISOString() }, + }); + return summary; +} + +export async function resetOffline( + app: DevProjectContext, + log: Logger, +): Promise { + const projectData = await readProjectConfig(app.projectRoot); + const seedData = await readSeedFiles(projectData.project); + + if (seedData?.scriptPath) { + return await withTempDevInstance(app, log, (instance) => + resetViaInstance(instance), + ); + } + + const { db, dataDir } = await openOfflineDatabase(app, projectData); + await db.resetData(); + + const summary = seedData + ? await applySeeds(db, seedData, { mode: "replace" }) + : null; + + await writeDataDirMeta(dataDir, { + formatVersion: 1, + appId: app.id, + seed: seedData + ? { hash: seedData.hash, appliedAt: new Date().toISOString() } + : null, + }); + + return { + reset: true, + seeded: summary?.applied ?? false, + dataDir, + seed: summary, + }; +} + +/** Print per-fixture counts and warnings (human, non-json output). */ +export function logSeedSummary(log: Logger, summary: SeedSummary): void { + log.info(formatSeedCounts(summary).join("\n")); + for (const warning of summary.warnings) { + log.warn(warning); + } +} diff --git a/packages/cli/src/cli/commands/dev/seed.ts b/packages/cli/src/cli/commands/dev/seed.ts new file mode 100644 index 000000000..1d38d69ad --- /dev/null +++ b/packages/cli/src/cli/commands/dev/seed.ts @@ -0,0 +1,70 @@ +import type { Command } from "commander"; +import type { CLIContext, RunCommandResult } from "@/cli/types.js"; +import { Base44Command } from "@/cli/utils/index.js"; +import { readDevInstance } from "@/core/local-state/index.js"; +import type { SeedMode } from "@/core/resources/seed/index.js"; +import { + confirmDestructiveAction, + logSeedSummary, + requireDevProject, + seedOffline, + seedViaInstance, +} from "./seed-shared.js"; + +interface DevSeedOptions { + replace?: boolean; + force?: boolean; +} + +async function devSeedAction( + ctx: CLIContext, + options: DevSeedOptions, +): Promise { + const { log, app, jsonMode, isNonInteractive } = ctx; + const project = requireDevProject(app, "dev seed"); + + const mode: SeedMode = options.replace ? "replace" : "upsert"; + if (mode === "replace") { + await confirmDestructiveAction( + isNonInteractive, + options.force === true, + "Replace mode deletes existing records in seeded collections. Continue?", + "--force is required to use --replace in non-interactive mode", + ); + } + + const instance = await readDevInstance(project.projectRoot); + const summary = instance + ? await seedViaInstance(instance, mode) + : await seedOffline(project, mode, log); + + // Fixtures applied but seed.ts failed: report everything, exit non-zero. + if (summary.script?.ran === false) { + process.exitCode = 1; + } + + const outroMessage = summary.applied + ? `Seeds applied (${summary.mode} mode)` + : "No seed files found — nothing applied"; + + if (jsonMode) { + return { + outroMessage, + stdout: `${JSON.stringify(summary, null, 2)}\n`, + }; + } + + logSeedSummary(log, summary); + return { outroMessage }; +} + +export function getDevSeedCommand(): Command { + return new Base44Command("seed") + .description("Apply seed fixtures to the local dev database") + .option( + "--replace", + "Delete existing records in seeded collections before inserting", + ) + .option("--force", "Skip the confirmation prompt for --replace") + .action(devSeedAction); +} diff --git a/packages/cli/src/cli/commands/dev/status.ts b/packages/cli/src/cli/commands/dev/status.ts new file mode 100644 index 000000000..98f57754a --- /dev/null +++ b/packages/cli/src/cli/commands/dev/status.ts @@ -0,0 +1,56 @@ +import type { Command } from "commander"; +import type { CLIContext, RunCommandResult } from "@/cli/types.js"; +import { Base44Command, theme } from "@/cli/utils/index.js"; +import { ConfigInvalidError } from "@/core/errors.js"; +import { readDevInstance } from "@/core/local-state/index.js"; + +async function devStatusAction(ctx: CLIContext): Promise { + const { log, app, jsonMode } = ctx; + if (!app?.projectRoot) { + throw new ConfigInvalidError( + "base44 dev status requires a linked local project. Run it from a project with base44/.app.jsonc.", + ); + } + + const instance = await readDevInstance(app.projectRoot); + // dev.json minus adminToken (and pid) — the machine-readable status shape. + const status = instance + ? { + running: true, + appId: instance.appId, + url: instance.url, + port: instance.port, + startedAt: instance.startedAt, + dataDir: instance.dataDir, + seed: instance.seed, + } + : { running: false }; + + const outroMessage = instance + ? `Dev server is running at ${theme.colors.links(instance.url)}` + : "No dev server is running for this project."; + + if (jsonMode) { + return { outroMessage, stdout: `${JSON.stringify(status, null, 2)}\n` }; + } + + if (instance) { + log.info( + [ + `App ID: ${instance.appId}`, + `URL: ${theme.colors.links(instance.url)}`, + `Port: ${instance.port}`, + `Started at: ${instance.startedAt}`, + `Data dir: ${instance.dataDir}`, + ].join("\n"), + ); + } + + return { outroMessage }; +} + +export function getDevStatusCommand(): Command { + return new Base44Command("status", { requireAuth: false }) + .description("Show the status of the local development server") + .action(devStatusAction); +} diff --git a/packages/cli/src/cli/dev/dev-server/auth/tokens.ts b/packages/cli/src/cli/dev/dev-server/auth/tokens.ts index b1f259e46..6defcb44d 100644 --- a/packages/cli/src/cli/dev/dev-server/auth/tokens.ts +++ b/packages/cli/src/cli/dev/dev-server/auth/tokens.ts @@ -1,29 +1,8 @@ -import jwt from "jsonwebtoken"; - -const LOCAL_DEV_SECRET = "LOCAL_DEV_SECRET"; - -/** - * Sentinel identity used for service-role (`asServiceRole`) requests in dev. - * In production Base44 injects a privileged service token; locally we mint a - * JWT for this subject and grant it full access (see `checkRLS`). - */ -export const SERVICE_ROLE_EMAIL = "server@server.com"; - -export const createJwtToken = (email: string) => { - return jwt.sign({ sub: email }, LOCAL_DEV_SECRET, { - expiresIn: "360d", - }); -}; - -/** - * Mints the service-role JWT injected as `Base44-Service-Authorization` so - * `asServiceRole` works locally regardless of how the caller is authenticated. - */ -const createServiceToken = () => createJwtToken(SERVICE_ROLE_EMAIL); - -export const createServiceAuthorizationHeader = () => - `Bearer ${createServiceToken()}`; - -/** True when a JWT subject identifies the service-role principal. */ -export const isServiceSubject = (subject: string): boolean => - subject === SERVICE_ROLE_EMAIL; +// The JWT helpers moved to core so the seed-script runner (core layer) can +// mint local service tokens; this re-export keeps dev-server imports stable. +export { + createJwtToken, + createServiceAuthorizationHeader, + isServiceSubject, + SERVICE_ROLE_EMAIL, +} from "@/core/local-state/tokens.js"; diff --git a/packages/cli/src/cli/dev/dev-server/db/create-record.ts b/packages/cli/src/cli/dev/dev-server/db/create-record.ts new file mode 100644 index 000000000..46e9db0e7 --- /dev/null +++ b/packages/cli/src/cli/dev/dev-server/db/create-record.ts @@ -0,0 +1,50 @@ +import { nanoid } from "nanoid"; +import type { Entity } from "@/core/resources/entity/schema.js"; +import type { Database } from "./database.js"; +import { applyFLS } from "./rls.js"; +import type { EntityRecord } from "./validator.js"; + +export interface RecordOwner { + email: string; + id: string; +} + +export interface PrepareRecordForCreateOptions { + /** Principal used for field-level security (service role bypasses FLS). */ + actor: Record | undefined; + /** Identity stamped into created_by/created_by_id; omit for none. */ + owner: RecordOwner | undefined; + now: string; + /** Stable id override (seed fixtures); defaults to a fresh nanoid. */ + id?: string; +} + +/** + * Assemble a record for insertion exactly like the entity POST route: filter + * to schema fields, apply defaults, validate (throws EntityValidationError), + * then stamp id, owner, and timestamps. RLS is the caller's concern. + */ +export function prepareRecordForCreate( + db: Database, + entityName: string, + schema: Entity, + body: EntityRecord, + { actor, owner, now, id }: PrepareRecordForCreateOptions, +): EntityRecord { + const { _id, ...recordBody } = body; + const filteredBody = applyFLS( + db.prepareRecord(entityName, recordBody), + schema, + actor, + "write", + ); + db.validate(entityName, filteredBody); + + return { + ...filteredBody, + id: id ?? nanoid(), + ...(owner ? { created_by: owner.email, created_by_id: owner.id } : {}), + created_date: now, + updated_date: now, + }; +} diff --git a/packages/cli/src/cli/dev/dev-server/db/database.ts b/packages/cli/src/cli/dev/dev-server/db/database.ts index b6128b38e..376c41757 100644 --- a/packages/cli/src/cli/dev/dev-server/db/database.ts +++ b/packages/cli/src/cli/dev/dev-server/db/database.ts @@ -1,8 +1,9 @@ +import { join } from "node:path"; import Datastore from "@seald-io/nedb"; import { nanoid } from "nanoid"; import { readAuth } from "@/core/index.js"; import type { Entity } from "@/core/resources/entity/schema.js"; -import { getNowISOTimestamp } from "../utils.js"; +import { buildUserDocument } from "./users.js"; import { type EntityRecord, Validator } from "./validator.js"; // Developer can't create collection with names that are not alphanumeric. @@ -12,53 +13,142 @@ export const USER_COLLECTION = "user"; export const PRIVATE_USER_COLLECTION = PRIVATE_COLLECTION_PREFIX + USER_COLLECTION; +interface DatabaseOptions { + /** + * Directory for file-backed NeDB collections (one `.db` per + * collection). Omit for the in-memory database used by tests. + */ + dataDir?: string; +} + export class Database { private collections: Map = new Map(); private schemas: Map = new Map(); private validator: Validator = new Validator(); + private readonly dataDir?: string; + + constructor(options: DatabaseOptions = {}) { + this.dataDir = options.dataDir; + } async load(entities: Entity[]) { - await this.loadUserCollection(entities); + this.applySchemas(entities); + this.ensureCollections(entities); + await this.bootstrapCliUser(); + await this.compactAll(); + } - for (const entity of entities) { - const entityName = this.normalizeName(entity.name); - if (entityName === USER_COLLECTION) { - continue; + /** + * Replace entity schemas without touching stored data. Collections for new + * entities are created; collections for removed entities are dropped from + * the map but their on-disk files are left untouched. + */ + reloadSchemas(entities: Entity[]) { + this.applySchemas(entities); + this.ensureCollections(entities); + for (const name of this.collections.keys()) { + if ( + !this.schemas.has(name) && + !name.startsWith(PRIVATE_COLLECTION_PREFIX) + ) { + this.collections.delete(name); } - - this.collections.set(entityName, new Datastore()); - this.schemas.set(entityName, entity); } } - private async loadUserCollection(entities: Entity[]) { + private applySchemas(entities: Entity[]) { const userEntity = entities.find( (e) => this.normalizeName(e.name) === USER_COLLECTION, ); + // Build before clearing so an invalid User schema leaves the current + // schemas intact (the watcher logs the error and keeps serving). + const userSchema = this.buildUserSchema(userEntity); - this.schemas.set(USER_COLLECTION, this.buildUserSchema(userEntity)); + this.schemas.clear(); + this.schemas.set(USER_COLLECTION, userSchema); - const collection = new Datastore(); - this.collections.set(USER_COLLECTION, collection); + for (const entity of entities) { + const entityName = this.normalizeName(entity.name); + if (entityName === USER_COLLECTION) { + continue; + } + this.schemas.set(entityName, entity); + } + } + private ensureCollections(entities: Entity[]) { + this.ensureCollection(USER_COLLECTION); // Private user collection will store data that is not accessible for the client. // Data like password. - this.collections.set(PRIVATE_USER_COLLECTION, new Datastore()); + this.ensureCollection(PRIVATE_USER_COLLECTION); + + for (const entity of entities) { + this.ensureCollection(this.normalizeName(entity.name)); + } + } + + private ensureCollection(normalizedName: string) { + if (this.collections.has(normalizedName)) { + return; + } + const datastore = this.dataDir + ? new Datastore({ + filename: join(this.dataDir, `${normalizedName}.db`), + autoload: true, + }) + : new Datastore(); + this.collections.set(normalizedName, datastore); + } + + /** + * Insert the logged-in CLI user as the local admin unless a user with that + * email already exists (idempotent across dev-server restarts). + */ + private async bootstrapCliUser() { + const collection = this.collections.get(USER_COLLECTION); + if (!collection) { + return; + } const userInfo = await readAuth(); - const now = getNowISOTimestamp(); - await collection.insertAsync({ - id: nanoid(), - email: userInfo.email, - full_name: userInfo.name, - is_service: false, - is_verified: true, - disabled: null, - role: "admin", - collaborator_role: "editor", - created_date: now, - updated_date: now, - }); + const existing = await collection.findOneAsync({ email: userInfo.email }); + if (existing) { + return; + } + + await collection.insertAsync( + buildUserDocument({ + id: nanoid(), + email: userInfo.email, + fullName: userInfo.name, + role: "admin", + }), + ); + } + + /** + * Delete every document from every collection (datastore files are kept), + * then re-insert the bootstrap CLI login user. + */ + async resetData() { + await Promise.all( + Array.from(this.collections.values(), (collection) => + collection.removeAsync({}, { multi: true }), + ), + ); + await this.bootstrapCliUser(); + } + + /** Compact file-backed datafiles after load; no-op for in-memory mode. */ + private async compactAll() { + if (!this.dataDir) { + return; + } + await Promise.all( + Array.from(this.collections.values(), (collection) => + collection.compactDatafileAsync(), + ), + ); } private buildUserSchema(customUserEntity: Entity | undefined): Entity { diff --git a/packages/cli/src/cli/dev/dev-server/db/export.ts b/packages/cli/src/cli/dev/dev-server/db/export.ts new file mode 100644 index 000000000..31ba421f3 --- /dev/null +++ b/packages/cli/src/cli/dev/dev-server/db/export.ts @@ -0,0 +1,47 @@ +import { InvalidInputError } from "@/core/errors.js"; +import { normalizeSeedName } from "@/core/resources/seed/index.js"; +import { stripInternalFields } from "../utils.js"; +import { type Database, USER_COLLECTION } from "./database.js"; +import type { EntityRecord } from "./validator.js"; + +export type CollectionsExport = Record; + +/** + * Read local collections for `data dump`, keyed by entity display name and + * stripped of NeDB's `_id`. By default the user collection is excluded (user + * fixtures are users.jsonc-shaped, not entity fixtures); naming an entity + * explicitly includes exactly the requested collections. + */ +export async function exportCollections( + db: Database, + entityNames?: string[], +): Promise { + const available = new Map< + string, + { displayName: string; collection: string } + >(); + for (const collection of db.getCollectionNames()) { + const displayName = db.getSchema(collection)?.name ?? collection; + available.set(normalizeSeedName(displayName), { displayName, collection }); + } + + const selected = entityNames?.length + ? entityNames.map((entityName) => { + const match = available.get(normalizeSeedName(entityName)); + if (!match) { + throw new InvalidInputError(`Unknown entity "${entityName}"`); + } + return match; + }) + : [...available.values()].filter( + ({ collection }) => collection !== USER_COLLECTION, + ); + + const result: CollectionsExport = {}; + for (const { displayName, collection } of selected) { + const docs = + (await db.getCollection(collection)?.findAsync({})) ?? []; + result[displayName] = stripInternalFields(docs); + } + return result; +} diff --git a/packages/cli/src/cli/dev/dev-server/db/seed.ts b/packages/cli/src/cli/dev/dev-server/db/seed.ts new file mode 100644 index 000000000..ddbe9c75e --- /dev/null +++ b/packages/cli/src/cli/dev/dev-server/db/seed.ts @@ -0,0 +1,312 @@ +import type Datastore from "@seald-io/nedb"; +import { nanoid } from "nanoid"; +import { SERVICE_USER } from "@/cli/dev/dev-server/routes/entities/current-user.js"; +import { InvalidInputError } from "@/core/errors.js"; +import type { Entity } from "@/core/resources/entity/schema.js"; +import { + normalizeSeedName, + type SeedData, + type SeedEntityCounts, + type SeedMode, + type SeedRecordsFixture, + type SeedSummary, + type SeedUsersFixture, + USERS_FIXTURE_BASENAME, +} from "@/core/resources/seed/index.js"; +import type { EntityEvent } from "../realtime.js"; +import { stripInternalFields } from "../utils.js"; +import { prepareRecordForCreate, type RecordOwner } from "./create-record.js"; +import { + type Database, + PRIVATE_USER_COLLECTION, + USER_COLLECTION, +} from "./database.js"; +import { + buildUserDocument, + fullNameFromEmail, + upsertUserCredentials, +} from "./users.js"; +import { type EntityRecord, EntityValidationError } from "./validator.js"; + +export interface ApplySeedsOptions { + mode: SeedMode; + /** Realtime broadcast hook (dev-server path); best effort, omit offline. */ + emit?: (entityName: string, event: EntityEvent) => void; +} + +interface SeededUserDocument extends Record { + id: string; + email: string; +} + +function invalidSeedRecord( + relPath: string, + index: number, + message: string, +): InvalidInputError { + return new InvalidInputError( + `Invalid seed record in ${relPath} at index ${index}: ${message}`, + ); +} + +function requireCollection(db: Database, name: string): Datastore { + const collection = db.getCollection(name); + if (!collection) { + throw new InvalidInputError(`Collection "${name}" not found`); + } + return collection; +} + +function emitEvent( + emit: ApplySeedsOptions["emit"], + entityName: string, + type: EntityEvent["type"], + record: EntityRecord, +): void { + const data = stripInternalFields(record); + emit?.(entityName, { + type, + data, + id: data.id as string, + timestamp: new Date().toISOString(), + }); +} + +/** + * Upsert seed users by email through the same building blocks as local + * registration: public fields into the user collection (verified), password + * into the private user collection. Existing users keep their id and + * created_date, so issued tokens stay valid. Never deletes users — the CLI + * login user and locally registered users survive every mode. + */ +async function applyUsersFixture( + db: Database, + fixture: SeedUsersFixture, +): Promise { + const userCollection = requireCollection(db, USER_COLLECTION); + const privateUserCollection = requireCollection(db, PRIVATE_USER_COLLECTION); + + for (const [index, seedUser] of fixture.users.entries()) { + const { email, role, password, full_name, ...customFields } = seedUser; + + const custom = db.prepareRecord(USER_COLLECTION, customFields, true); + try { + db.validate(USER_COLLECTION, custom, true); + } catch (error) { + if (error instanceof EntityValidationError) { + throw invalidSeedRecord(fixture.relPath, index, error.message); + } + throw error; + } + + const existing = await userCollection.findOneAsync({ + email, + }); + const document = { + ...buildUserDocument({ + id: existing?.id ?? nanoid(), + email, + fullName: full_name ?? fullNameFromEmail(email), + role, + createdDate: existing?.created_date as string | undefined, + }), + ...custom, + }; + + if (existing) { + await userCollection.updateAsync({ email }, document); + } else { + await userCollection.insertAsync(document); + } + + if (password !== undefined) { + await upsertUserCredentials(privateUserCollection, { + id: document.id as string, + email, + password, + }); + } + } + + return fixture.users.length; +} + +interface ResolvedEntity { + schema: Entity; + collection: Datastore; +} + +/** + * Resolve a fixture file base name to an entity schema by normalized + * comparison (lowercased, `-`/`_` stripped), so `task.jsonc`, `Task.jsonc`, + * and `team-member.jsonc` all resolve. The user collection is reserved for + * the users fixture. + */ +function resolveEntity(db: Database, baseName: string): ResolvedEntity | null { + const target = normalizeSeedName(baseName); + for (const name of db.getCollectionNames()) { + if (name === USER_COLLECTION) { + continue; + } + const schema = db.getSchema(name); + if (schema && normalizeSeedName(schema.name) === target) { + const collection = db.getCollection(name); + if (collection) { + return { schema, collection }; + } + } + } + return null; +} + +async function resolveOwner( + userCollection: Datastore, + createdBy: string | undefined, +): Promise { + if (createdBy === undefined) { + return undefined; + } + const user = await userCollection.findOneAsync({ + email: createdBy, + }); + if (!user) { + throw new Error(`created_by references unknown user "${createdBy}"`); + } + return { email: user.email, id: user.id }; +} + +/** + * Apply one entity fixture. All records are resolved and validated before + * any write, so an invalid file leaves its collection untouched. + * + * - `upsert`: records with an id are updated-or-inserted by id; id-less + * records are skipped. Never deletes. + * - `replace`: the collection is truncated, then every record (including + * id-less ones) is inserted. + */ +async function applyRecordsFixture( + db: Database, + { schema, collection }: ResolvedEntity, + fixture: SeedRecordsFixture, + mode: SeedMode, + emit: ApplySeedsOptions["emit"], +): Promise { + const userCollection = requireCollection(db, USER_COLLECTION); + const now = new Date().toISOString(); + + const prepared: { document: EntityRecord; hasId: boolean }[] = []; + for (const [index, seedRecord] of fixture.records.entries()) { + const { id, created_by, ...body } = seedRecord; + try { + const owner = await resolveOwner(userCollection, created_by); + const document = prepareRecordForCreate(db, schema.name, schema, body, { + actor: SERVICE_USER, + owner, + now, + id, + }); + prepared.push({ document, hasId: id !== undefined }); + } catch (error) { + if (error instanceof Error) { + throw invalidSeedRecord(fixture.relPath, index, error.message); + } + throw error; + } + } + + const counts: SeedEntityCounts = { created: 0, updated: 0, skipped: 0 }; + + if (mode === "replace") { + await collection.removeAsync({}, { multi: true }); + for (const { document } of prepared) { + const inserted = await collection.insertAsync(document); + counts.created++; + emitEvent(emit, schema.name, "create", inserted); + } + return counts; + } + + for (const { document, hasId } of prepared) { + if (!hasId) { + counts.skipped++; + continue; + } + const existing = await collection.findOneAsync({ id: document.id }); + if (existing) { + const { created_date: _created_date, ...updateFields } = document; + const { affectedDocuments } = await collection.updateAsync( + { id: document.id }, + { $set: { ...updateFields, updated_date: now } }, + { returnUpdatedDocs: true }, + ); + counts.updated++; + if (affectedDocuments) { + emitEvent(emit, schema.name, "update", affectedDocuments); + } + } else { + const inserted = await collection.insertAsync(document); + counts.created++; + emitEvent(emit, schema.name, "create", inserted); + } + } + + return counts; +} + +/** + * Apply seed fixtures to the local database: users first, then entity + * fixtures in filename order. Seeding runs as service role (bypasses RLS and + * FLS); records are stamped with id/created_by/created_date/updated_date + * exactly like the entity POST route. The `script` step (base44/seed.ts) is + * not part of fixture application and is reported as null here. + */ +export async function applySeeds( + db: Database, + seedData: SeedData, + { mode, emit }: ApplySeedsOptions, +): Promise { + const warnings: string[] = []; + + const reservedEntity = db + .getCollectionNames() + .map((name) => db.getSchema(name)) + .find( + (schema) => + schema && normalizeSeedName(schema.name) === USERS_FIXTURE_BASENAME, + ); + if (seedData.users && reservedEntity) { + warnings.push( + `Entity "${reservedEntity.name}" collides with the reserved users.jsonc fixture name; ${seedData.users.relPath} is applied as the users fixture`, + ); + } + + const users = seedData.users + ? await applyUsersFixture(db, seedData.users) + : 0; + + const records: Record = {}; + for (const fixture of seedData.fixtures) { + if (normalizeSeedName(fixture.baseName) === USER_COLLECTION) { + warnings.push( + `Seed fixture "${fixture.relPath}" targets the built-in User entity — use users.jsonc to seed users; file skipped`, + ); + continue; + } + const resolved = resolveEntity(db, fixture.baseName); + if (!resolved) { + warnings.push( + `Seed fixture "${fixture.relPath}" does not match any entity; file skipped`, + ); + continue; + } + records[resolved.schema.name] = await applyRecordsFixture( + db, + resolved, + fixture, + mode, + emit, + ); + } + + return { applied: true, mode, users, records, script: null, warnings }; +} diff --git a/packages/cli/src/cli/dev/dev-server/db/users.ts b/packages/cli/src/cli/dev/dev-server/db/users.ts new file mode 100644 index 000000000..101d2af70 --- /dev/null +++ b/packages/cli/src/cli/dev/dev-server/db/users.ts @@ -0,0 +1,58 @@ +import type Datastore from "@seald-io/nedb"; +import { getNowISOTimestamp } from "../utils.js"; + +export interface BuildUserDocumentOptions { + id: string; + email: string; + fullName: string; + role: "admin" | "user"; + /** Preserve the original creation timestamp when updating an existing user. */ + createdDate?: string; +} + +/** Derive the default full name from an email, like OTP registration does. */ +export function fullNameFromEmail(email: string): string { + const match = /^([^@]+)/.exec(email); + return match ? match[1] : email; +} + +/** + * Canonical local user document. Single shape shared by the CLI bootstrap + * user, OTP registration, and seeding so all local users look alike. + */ +export function buildUserDocument({ + id, + email, + fullName, + role, + createdDate, +}: BuildUserDocumentOptions): Record { + const now = getNowISOTimestamp(); + return { + id, + email, + full_name: fullName, + is_service: false, + is_verified: true, + disabled: null, + role, + collaborator_role: "editor", + created_date: createdDate ?? now, + updated_date: now, + }; +} + +/** + * Store login credentials in the private user collection the same way + * `/register` does (minus the OTP step), so `/login` accepts the password. + */ +export async function upsertUserCredentials( + privateUserCollection: Datastore, + { id, email, password }: { id: string; email: string; password: string }, +): Promise { + await privateUserCollection.updateAsync( + { email }, + { $set: { id, email, password, createdAt: Date.now() } }, + { upsert: true }, + ); +} diff --git a/packages/cli/src/cli/dev/dev-server/main.ts b/packages/cli/src/cli/dev/dev-server/main.ts index fdd57254f..8ad2e7007 100644 --- a/packages/cli/src/cli/dev/dev-server/main.ts +++ b/packages/cli/src/cli/dev/dev-server/main.ts @@ -1,3 +1,5 @@ +import { randomBytes } from "node:crypto"; +import { rm } from "node:fs/promises"; import type { Server } from "node:http"; import { dirname, join } from "node:path"; import type { Logger } from "@base44-cli/logger"; @@ -6,17 +8,45 @@ import express from "express"; import getPort from "get-port"; import { createProxyMiddleware } from "http-proxy-middleware"; import { dir } from "tmp-promise"; -import { createDevLogger } from "@/cli/dev/createDevLogger.js"; +import { createDevLogger, type DevLogger } from "@/cli/dev/createDevLogger.js"; import { FunctionManager } from "@/cli/dev/dev-server/function-manager.js"; import { createFunctionRouter } from "@/cli/dev/dev-server/routes/functions.js"; +import { runSeedScriptStep } from "@/cli/dev/seed-script-step.js"; +import { formatSeedCounts } from "@/cli/dev/seed-summary.js"; import { theme } from "@/cli/utils/index.js"; +import { ConfigInvalidError } from "@/core/errors.js"; +import { + deleteDevInstance, + getDataDir, + getMetaJsonPath, + readDataDirMeta, + type SeedState, + writeDataDirMeta, + writeDevInstance, +} from "@/core/local-state/index.js"; import type { ProjectData } from "@/core/project/types.js"; +import { + type DevResetResult, + emptySeedSummary, + readSeedFiles, + type SeedData, + type SeedMode, + type SeedSummary, +} from "@/core/resources/seed/index.js"; import { Database } from "./db/database.js"; +import { exportCollections } from "./db/export.js"; +import { applySeeds } from "./db/seed.js"; import { type BroadcastEntityEvent, broadcastEntityEvent, createRealtimeServer, + type EntityEvent, } from "./realtime.js"; +import { + createAdminRouter, + DEV_ADMIN_BASE_PATH, + type DevServerStatus, +} from "./routes/admin-router.js"; import { createAuthRouter } from "./routes/auth-router.js"; import { createEntityRoutes } from "./routes/entities/entities-router.js"; import { @@ -35,6 +65,23 @@ interface DevServerOptions { port?: number; denoWrapperPath: string; appId?: string; + /** + * Enable file-backed persistence and the dev.json instance descriptor under + * `/.base44`. Requires `appId`. Omit for in-memory mode. + */ + state?: { + projectRoot: string; + /** Delete the local data dir before loading (start clean). */ + fresh?: boolean; + /** + * Temporary internal instance (offline `dev seed`/`dev reset` script + * step): skip the dev.json descriptor and the startup auto-seed — the + * caller drives seeding through the admin endpoints. + */ + ephemeral?: boolean; + }; + /** Log sink override (temporary instances log to stderr). */ + logger?: DevLogger; loadResources: () => Promise<{ functions: ProjectData["functions"]; entities: ProjectData["entities"]; @@ -43,10 +90,136 @@ interface DevServerOptions { }>; } +interface PersistenceContext { + projectRoot: string; + appId: string; + dataDir: string; + /** Seed state carried over from an existing meta.json (null until seeded). */ + seed: SeedState; + /** + * True when the data dir has no (valid) meta.json — first boot, after + * `--fresh`, or corrupt meta. Triggers the auto-seed. + */ + isNew: boolean; +} + +/** + * Prepare the on-disk data dir: honor `--fresh`, guard against reusing data + * that belongs to another app, and carry over the recorded seed state. + */ +async function preparePersistence( + options: DevServerOptions, + devLogger: ReturnType, +): Promise { + if (!options.state || !options.appId) { + return undefined; + } + + const { projectRoot, fresh } = options.state; + const appId = options.appId; + const dataDir = getDataDir(projectRoot); + + if (fresh) { + await rm(dataDir, { recursive: true, force: true }); + devLogger.log("--fresh: local data cleared"); + } + + const meta = await readDataDirMeta(dataDir); + if (meta.status === "corrupt") { + devLogger.warn( + `Ignoring corrupt ${getMetaJsonPath(dataDir)}; treating local data as new`, + ); + } + if (meta.status === "ok" && meta.meta.appId !== appId) { + throw new ConfigInvalidError( + `Local dev data in ${dataDir} belongs to app "${meta.meta.appId}", but this project is linked to app "${appId}".`, + getMetaJsonPath(dataDir), + { + hints: [ + { + message: + "Run 'base44 dev --fresh' to delete the local data and start clean", + command: "base44 dev --fresh", + }, + ], + }, + ); + } + + return { + projectRoot, + appId, + dataDir, + seed: meta.status === "ok" ? meta.meta.seed : null, + isNew: meta.status !== "ok", + }; +} + +interface StartupSeedOutcome { + seedState: SeedState; + /** + * Fixture summary + pending `seed.ts` path. The script step needs the + * listening server, so the caller runs it (and logs the single summary) + * after listen. + */ + pending: { summary: SeedSummary; scriptPath: string | null } | null; +} + +/** + * Auto-seed on startup: apply fixtures (replace mode) when the data dir is + * new, or hint when existing data was seeded from different seed files. + * Failures are logged and never crash the dev server — it keeps serving with + * whatever data applied. Returns the seed state to record in meta/dev.json. + */ +async function runStartupSeed( + db: Database, + project: ProjectData["project"], + persistence: PersistenceContext, + devLogger: DevLogger, +): Promise { + let seedData: SeedData | null = null; + try { + seedData = await readSeedFiles(project); + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + devLogger.error(`Failed to read seed files: ${message}`); + return { seedState: persistence.seed, pending: null }; + } + if (!seedData) { + return { seedState: persistence.seed, pending: null }; + } + + if (!persistence.isNew) { + if (persistence.seed?.hash !== seedData.hash) { + devLogger.log("Seed files changed — run `base44 dev seed` to apply"); + } + return { seedState: persistence.seed, pending: null }; + } + + try { + const summary = await applySeeds(db, seedData, { mode: "replace" }); + return { + seedState: { hash: seedData.hash, appliedAt: new Date().toISOString() }, + pending: { summary, scriptPath: seedData.scriptPath }, + }; + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + devLogger.error( + `Seeding failed: ${message}. Continuing with partially seeded data — fix the seed files and run 'base44 dev seed'.`, + ); + return { seedState: persistence.seed, pending: null }; + } +} + interface DevServerResult { port: number; + url: string; server: Server; isServingFrontend: boolean; + /** Admin-endpoint token for this instance (also in dev.json when written). */ + adminToken: string; + /** Gracefully stop the server (same path as SIGINT/SIGTERM). */ + shutdown: () => Promise; } export async function createDevServer( @@ -90,7 +263,8 @@ export async function createDevServer( next(); }); - const devLogger = createDevLogger("backend", theme.styles.info); + const devLogger = + options.logger ?? createDevLogger("backend", theme.styles.info); const functionManager = new FunctionManager( functions, @@ -106,8 +280,26 @@ export async function createDevServer( ); } - const db = new Database(); + const persistence = await preparePersistence(options, devLogger); + + const db = new Database({ dataDir: persistence?.dataDir }); await db.load(entities); + + const ephemeral = options.state?.ephemeral === true; + let seedState: SeedState = persistence?.seed ?? null; + let startupSeed: StartupSeedOutcome["pending"] = null; + if (persistence) { + if (!ephemeral) { + const outcome = await runStartupSeed(db, project, persistence, devLogger); + seedState = outcome.seedState; + startupSeed = outcome.pending; + } + await writeDataDirMeta(persistence.dataDir, { + formatVersion: 1, + appId: persistence.appId, + seed: seedState, + }); + } if (db.getCollectionNames().length > 0) { devLogger.log(`Loaded entities: ${db.getCollectionNames().join(", ")}`); } @@ -123,6 +315,108 @@ export async function createDevServer( const authRouter = createAuthRouter(db, devLogger); app.use("/api/apps/:appId/auth", authRouter); + const startedAt = new Date().toISOString(); + const adminToken = randomBytes(32).toString("hex"); + let writeInstanceFile: (() => Promise) | undefined; + + if (persistence) { + const state = persistence; + if (!ephemeral) { + writeInstanceFile = () => + writeDevInstance(state.projectRoot, { + appId: state.appId, + url: baseUrl, + port, + pid: process.pid, + dataDir: state.dataDir, + adminToken, + startedAt, + seed: seedState, + }); + } + + const updateSeedState = async (next: SeedState) => { + seedState = next; + await writeDataDirMeta(state.dataDir, { + formatVersion: 1, + appId: state.appId, + seed: next, + }); + await writeInstanceFile?.(); + }; + + const seedEmit = (entityName: string, event: EntityEvent) => + emitEntityEvent(state.appId, entityName, event); + + const runSeed = async (mode: SeedMode): Promise => { + const seedData = await readSeedFiles(project); + if (!seedData) { + return emptySeedSummary(mode); + } + const summary = await applySeeds(db, seedData, { mode, emit: seedEmit }); + await updateSeedState({ + hash: seedData.hash, + appliedAt: new Date().toISOString(), + }); + await runSeedScriptStep(summary, seedData.scriptPath, { + appId: state.appId, + baseUrl, + }); + return summary; + }; + + const runReset = async (): Promise => { + await db.resetData(); + const seedData = await readSeedFiles(project); + const summary = seedData + ? await applySeeds(db, seedData, { mode: "replace", emit: seedEmit }) + : null; + await updateSeedState( + seedData + ? { hash: seedData.hash, appliedAt: new Date().toISOString() } + : null, + ); + if (summary && seedData) { + await runSeedScriptStep(summary, seedData.scriptPath, { + appId: state.appId, + baseUrl, + }); + } + return { + reset: true, + seeded: summary?.applied ?? false, + dataDir: state.dataDir, + seed: summary, + }; + }; + + const getStatus = async (): Promise => { + const collections: Record = {}; + for (const name of db.getCollectionNames()) { + collections[name] = (await db.getCollection(name)?.countAsync({})) ?? 0; + } + return { + appId: state.appId, + port, + startedAt, + seed: seedState, + collections, + }; + }; + + app.use( + DEV_ADMIN_BASE_PATH, + createAdminRouter({ + adminToken, + logger: devLogger, + getStatus, + runSeed, + runReset, + getExport: (entityNames) => exportCollections(db, entityNames), + }), + ); + } + const { path: mediaFilesDir } = await dir(); app.use("/media/private/:fileUri", (req, res, next) => { @@ -201,6 +495,24 @@ export async function createDevServer( broadcastEntityEvent(io, appId, entityName, event); }; + await writeInstanceFile?.(); + + // The startup seed's `seed.ts` step needs the listening server; run it + // now, then log the single seed summary (fixtures + script outcome). + // Script failures only warn — the server keeps serving. + if (persistence && startupSeed) { + await runSeedScriptStep(startupSeed.summary, startupSeed.scriptPath, { + appId: persistence.appId, + baseUrl, + }); + devLogger.log( + `Seeds applied: ${formatSeedCounts(startupSeed.summary).join("; ")}`, + ); + for (const warning of startupSeed.summary.warnings) { + devLogger.warn(warning); + } + } + const base44ConfigWatcher = new WatchBase44( { functions: join(dirname(project.configPath), project.functionsDir), @@ -225,12 +537,8 @@ export async function createDevServer( } if (name === "entities") { - const previousEntityCount = db.getCollectionNames().length; - db.dropAll(); - if (previousEntityCount > 0) { - devLogger.log("Entities directory changed, clearing data..."); - } - await db.load(entities); + db.reloadSchemas(entities); + devLogger.log("Entities changed, schemas reloaded (data preserved)"); if (db.getCollectionNames().length > 0) { devLogger.log( `Loaded entities: ${db.getCollectionNames().join(", ")}`, @@ -284,6 +592,11 @@ export async function createDevServer( }; const runShutdown = async () => { + process.off("SIGINT", shutdown); + process.off("SIGTERM", shutdown); + if (persistence && !ephemeral) { + await deleteDevInstance(persistence.projectRoot); + } base44ConfigWatcher.close(); await io.close(); await functionManager.stopAll(); @@ -309,5 +622,12 @@ export async function createDevServer( serveRunner.start(); } - return { port, server, isServingFrontend: serveRunner !== undefined }; + return { + port, + url: baseUrl, + server, + isServingFrontend: serveRunner !== undefined, + adminToken, + shutdown, + }; } diff --git a/packages/cli/src/cli/dev/dev-server/routes/admin-router.ts b/packages/cli/src/cli/dev/dev-server/routes/admin-router.ts new file mode 100644 index 000000000..77f6a04ff --- /dev/null +++ b/packages/cli/src/cli/dev/dev-server/routes/admin-router.ts @@ -0,0 +1,130 @@ +import type { Request, Response, Router } from "express"; +import { Router as createRouter, json } from "express"; +import { z } from "zod"; +import type { DevLogger } from "@/cli/dev/createDevLogger.js"; +import { isUserError as isCLIUserError } from "@/core/errors.js"; +import { + type DevResetResult, + type SeedMode, + SeedModeSchema, + type SeedState, + type SeedSummary, +} from "@/core/index.js"; +import { EntityValidationError } from "../db/validator.js"; + +/** Header carrying the per-instance admin token from dev.json. */ +export const DEV_ADMIN_HEADER = "x-base44-dev-admin"; + +/** Local admin API base path (mounted on the dev-server Express app). */ +export const DEV_ADMIN_BASE_PATH = "/_base44/dev"; + +export interface DevServerStatus { + appId: string; + port: number; + startedAt: string; + seed: SeedState; + collections: Record; +} + +const SeedBodySchema = z.object({ + mode: SeedModeSchema.default("upsert"), +}); + +export interface AdminRouterDeps { + adminToken: string; + logger: DevLogger; + getStatus: () => Promise; + runSeed: (mode: SeedMode) => Promise; + runReset: () => Promise; + getExport: ( + entityNames?: string[], + ) => Promise[]>>; +} + +/** Seed/validation problems are the caller's to fix — report them as 400. */ +function isUserError(error: unknown): boolean { + return error instanceof EntityValidationError || isCLIUserError(error); +} + +function handleError( + error: unknown, + res: Response, + logger: DevLogger, + operation: string, +): void { + const message = error instanceof Error ? error.message : String(error); + if (isUserError(error)) { + res.status(400).json({ error: message }); + return; + } + logger.error(`Error in ${operation}:`, error); + res.status(500).json({ error: message }); +} + +/** + * Local admin endpoints (`/_base44/dev/*`). Every route requires the + * per-instance admin token from dev.json in the `x-base44-dev-admin` header. + */ +export function createAdminRouter({ + adminToken, + logger, + getStatus, + runSeed, + runReset, + getExport, +}: AdminRouterDeps): Router { + const router = createRouter(); + + router.use((req: Request, res: Response, next) => { + if (req.headers[DEV_ADMIN_HEADER] !== adminToken) { + res.status(401).json({ error: "Unauthorized" }); + return; + } + next(); + }); + + router.get("/status", async (_req, res) => { + try { + res.json(await getStatus()); + } catch (error) { + handleError(error, res, logger, "GET /_base44/dev/status"); + } + }); + + router.post("/seed", json(), async (req, res) => { + const body = SeedBodySchema.safeParse(req.body ?? {}); + if (!body.success) { + res.status(400).json({ error: 'mode must be "upsert" or "replace"' }); + return; + } + try { + res.json(await runSeed(body.data.mode)); + } catch (error) { + handleError(error, res, logger, "POST /_base44/dev/seed"); + } + }); + + router.post("/reset", async (_req, res) => { + try { + res.json(await runReset()); + } catch (error) { + handleError(error, res, logger, "POST /_base44/dev/reset"); + } + }); + + router.get("/export", async (req, res) => { + const entitiesParam = + typeof req.query.entities === "string" ? req.query.entities : undefined; + const entityNames = entitiesParam + ?.split(",") + .map((name) => name.trim()) + .filter((name) => name.length > 0); + try { + res.json({ collections: await getExport(entityNames) }); + } catch (error) { + handleError(error, res, logger, "GET /_base44/dev/export"); + } + }); + + return router; +} diff --git a/packages/cli/src/cli/dev/dev-server/routes/auth-router.ts b/packages/cli/src/cli/dev/dev-server/routes/auth-router.ts index d71151b18..a7862b363 100644 --- a/packages/cli/src/cli/dev/dev-server/routes/auth-router.ts +++ b/packages/cli/src/cli/dev/dev-server/routes/auth-router.ts @@ -11,7 +11,7 @@ import { PRIVATE_USER_COLLECTION, USER_COLLECTION, } from "../db/database.js"; -import { getNowISOTimestamp } from "../utils.js"; +import { buildUserDocument, fullNameFromEmail } from "../db/users.js"; const TEN_MINUTES = 10 * 60 * 1000; @@ -183,21 +183,14 @@ export function createAuthRouter(db: Database, logger: DevLogger): Router { ); const collection = db.getCollection(USER_COLLECTION); - const now = getNowISOTimestamp(); - const nameFromEmailMatch = /^([^@]+)/.exec(email); - const fullName = nameFromEmailMatch ? nameFromEmailMatch[1] : email; - await collection?.insertAsync({ - id: privateUserData.id, - email: email, - full_name: fullName, - is_service: false, - is_verified: true, - disabled: null, - role: "user", - collaborator_role: "editor", - created_date: now, - updated_date: now, - }); + await collection?.insertAsync( + buildUserDocument({ + id: privateUserData.id, + email, + fullName: fullNameFromEmail(email), + role: "user", + }), + ); res.json({ id: privateUserData.id, access_token: createJwtToken(email), diff --git a/packages/cli/src/cli/dev/dev-server/routes/entities/current-user.ts b/packages/cli/src/cli/dev/dev-server/routes/entities/current-user.ts index 3d5909cc3..335e71585 100644 --- a/packages/cli/src/cli/dev/dev-server/routes/entities/current-user.ts +++ b/packages/cli/src/cli/dev/dev-server/routes/entities/current-user.ts @@ -17,7 +17,8 @@ export type UserDocument = Document<{ role: "admin" | "user"; }>; -const SERVICE_USER: UserDocument = { +/** Synthetic principal for service-role requests; bypasses RLS and FLS. */ +export const SERVICE_USER: UserDocument = { _id: "service-role", id: "service-role", email: SERVICE_ROLE_EMAIL, diff --git a/packages/cli/src/cli/dev/dev-server/routes/entities/entities-router.ts b/packages/cli/src/cli/dev/dev-server/routes/entities/entities-router.ts index 473235d5b..69b1e8cf8 100644 --- a/packages/cli/src/cli/dev/dev-server/routes/entities/entities-router.ts +++ b/packages/cli/src/cli/dev/dev-server/routes/entities/entities-router.ts @@ -1,8 +1,8 @@ import type Datastore from "@seald-io/nedb"; import type { Request, Response, Router } from "express"; import { Router as createRouter, json } from "express"; -import { nanoid } from "nanoid"; import type { DevLogger } from "@/cli/dev/createDevLogger.js"; +import { prepareRecordForCreate } from "@/cli/dev/dev-server/db/create-record.js"; import type { Database } from "@/cli/dev/dev-server/db/database.js"; import { applyFLS, checkRLS } from "@/cli/dev/dev-server/db/rls.js"; import { @@ -93,17 +93,14 @@ export async function createEntityRoutes( now: string, ): EntityRecord | undefined { const { _id, ...recordBody } = body; - const ownerFields = { - created_by: currentUser?.email, - created_by_id: currentUser?.id, - }; if ( !checkRLS( schema.rls?.create, { ...recordBody, - ...ownerFields, + created_by: currentUser?.email, + created_by_id: currentUser?.id, }, currentUser, ) @@ -111,21 +108,13 @@ export async function createEntityRoutes( return undefined; } - const filteredBody = applyFLS( - db.prepareRecord(entityName, recordBody), - schema, - currentUser, - "write", - ); - db.validate(entityName, filteredBody); - - return { - ...filteredBody, - id: nanoid(), - ...ownerFields, - created_date: now, - updated_date: now, - }; + return prepareRecordForCreate(db, entityName, schema, recordBody, { + actor: currentUser, + owner: currentUser + ? { email: currentUser.email, id: currentUser.id } + : undefined, + now, + }); } const userRouter = createUserRouter(db, logger); diff --git a/packages/cli/src/cli/dev/seed-script-step.ts b/packages/cli/src/cli/dev/seed-script-step.ts new file mode 100644 index 000000000..546559ca8 --- /dev/null +++ b/packages/cli/src/cli/dev/seed-script-step.ts @@ -0,0 +1,44 @@ +import type { SeedSummary } from "@/core/resources/seed/index.js"; +import { runSeedScript } from "@/core/seed-script/index.js"; + +export interface SeedScriptStepOptions { + appId: string; + /** Base URL of the listening dev server the script runs against. */ + baseUrl: string; +} + +/** + * Run the project's `base44/seed.ts` after fixture application and record + * the outcome on the summary. Never throws: the summary keeps the fixture + * results, `script.ran` reports the outcome, and a warning carries the + * failure reason — callers decide whether a failure is fatal (`dev seed` + * exits non-zero) or not (startup keeps serving). + */ +export async function runSeedScriptStep( + summary: SeedSummary, + scriptPath: string | null, + { appId, baseUrl }: SeedScriptStepOptions, +): Promise { + if (!scriptPath) { + return; + } + try { + const { exitCode } = await runSeedScript({ + appId, + scriptPath, + localUrl: baseUrl, + }); + if (exitCode === 0) { + summary.script = { ran: true }; + return; + } + summary.script = { ran: false }; + summary.warnings.push( + `Seed script ${scriptPath} exited with code ${exitCode}`, + ); + } catch (error) { + summary.script = { ran: false }; + const message = error instanceof Error ? error.message : String(error); + summary.warnings.push(`Seed script failed: ${message}`); + } +} diff --git a/packages/cli/src/cli/dev/seed-summary.ts b/packages/cli/src/cli/dev/seed-summary.ts new file mode 100644 index 000000000..3c05fc30c --- /dev/null +++ b/packages/cli/src/cli/dev/seed-summary.ts @@ -0,0 +1,21 @@ +import type { SeedSummary } from "@/core/resources/seed/index.js"; + +/** + * Human-readable per-fixture count lines for a seed summary. Shared by the + * dev-server startup log and the `dev seed`/`dev reset` command output. + */ +export function formatSeedCounts(summary: SeedSummary): string[] { + const lines: string[] = []; + if (summary.users > 0) { + lines.push(`Users: ${summary.users} seeded`); + } + for (const [entityName, counts] of Object.entries(summary.records)) { + lines.push( + `${entityName}: ${counts.created} created, ${counts.updated} updated, ${counts.skipped} skipped`, + ); + } + if (lines.length === 0) { + lines.push("Nothing to seed"); + } + return lines; +} diff --git a/packages/cli/src/cli/program.ts b/packages/cli/src/cli/program.ts index cfa1a45bb..0deea7a5e 100644 --- a/packages/cli/src/cli/program.ts +++ b/packages/cli/src/cli/program.ts @@ -6,6 +6,7 @@ import { getLogoutCommand } from "@/cli/commands/auth/logout.js"; import { getWhoamiCommand } from "@/cli/commands/auth/whoami.js"; import { getConnectorsCommand } from "@/cli/commands/connectors/index.js"; import { getDashboardCommand } from "@/cli/commands/dashboard/index.js"; +import { getDataCommand } from "@/cli/commands/data/index.js"; import { getEntitiesPushCommand } from "@/cli/commands/entities/push.js"; import { getFunctionsCommand } from "@/cli/commands/functions/index.js"; import { getCreateCommand } from "@/cli/commands/project/create.js"; @@ -20,7 +21,7 @@ import { getTypesCommand } from "@/cli/commands/types/index.js"; import { Base44Command } from "@/cli/utils/index.js"; import { BASE44_APP_ID_ENV_VAR } from "@/core/consts.js"; import packageJson from "../../package.json"; -import { getDevCommand } from "./commands/dev.js"; +import { getDevCommand } from "./commands/dev/index.js"; import { getExecCommand } from "./commands/exec.js"; import { getEjectCommand } from "./commands/project/eject.js"; import type { CLIContext } from "./types.js"; @@ -103,6 +104,9 @@ export function createProgram(context: CLIContext): Command { // Register development commands program.addCommand(getDevCommand()); + // Register local/remote data commands + program.addCommand(getDataCommand()); + // Register logs command program.addCommand(getLogsCommand()); diff --git a/packages/cli/src/core/assets.ts b/packages/cli/src/core/assets.ts index 237b09906..1e3e58451 100644 --- a/packages/cli/src/core/assets.ts +++ b/packages/cli/src/core/assets.ts @@ -21,6 +21,10 @@ export function getExecWrapperPath(): string { return join(ASSETS_DIR, "deno-runtime", "exec.ts"); } +export function getSeedWrapperPath(): string { + return join(ASSETS_DIR, "deno-runtime", "seed.ts"); +} + /** * For the npm distribution: copy bundled assets to the standard location * on first run. Binary entry handles its own extraction separately. diff --git a/packages/cli/src/core/index.ts b/packages/cli/src/core/index.ts index 723c3e77c..530798b45 100644 --- a/packages/cli/src/core/index.ts +++ b/packages/cli/src/core/index.ts @@ -3,6 +3,7 @@ export * from "./clients/index.js"; export * from "./config.js"; export * from "./consts.js"; export * from "./errors.js"; +export * from "./local-state/index.js"; export * from "./project/index.js"; export * from "./resources/index.js"; export * from "./site/index.js"; diff --git a/packages/cli/src/core/local-state/dev-instance.ts b/packages/cli/src/core/local-state/dev-instance.ts new file mode 100644 index 000000000..588d0fcc2 --- /dev/null +++ b/packages/cli/src/core/local-state/dev-instance.ts @@ -0,0 +1,59 @@ +import { + deleteFile, + pathExists, + readJsonFile, + writeJsonFile, +} from "@/core/utils/fs.js"; +import { getDevJsonPath } from "./paths.js"; +import { type DevInstance, DevInstanceSchema } from "./schema.js"; + +/** True when a process with the given pid is alive (EPERM counts as alive). */ +export function isPidAlive(pid: number): boolean { + try { + process.kill(pid, 0); + return true; + } catch (error) { + return (error as NodeJS.ErrnoException).code !== "ESRCH"; + } +} + +/** Write the instance descriptor for a running dev server. */ +export async function writeDevInstance( + projectRoot: string, + instance: DevInstance, +): Promise { + await writeJsonFile(getDevJsonPath(projectRoot), instance); +} + +/** Remove the instance descriptor (graceful dev-server shutdown). */ +export async function deleteDevInstance(projectRoot: string): Promise { + await deleteFile(getDevJsonPath(projectRoot)); +} + +/** + * Read the dev-server instance descriptor. Returns null when the file is + * missing, invalid, or stale (owning process no longer alive); invalid and + * stale files are deleted. + */ +export async function readDevInstance( + projectRoot: string, +): Promise { + const devJsonPath = getDevJsonPath(projectRoot); + if (!(await pathExists(devJsonPath))) { + return null; + } + + let instance: DevInstance | null = null; + try { + const result = DevInstanceSchema.safeParse(await readJsonFile(devJsonPath)); + instance = result.success ? result.data : null; + } catch { + instance = null; + } + + if (!instance || !isPidAlive(instance.pid)) { + await deleteFile(devJsonPath); + return null; + } + return instance; +} diff --git a/packages/cli/src/core/local-state/index.ts b/packages/cli/src/core/local-state/index.ts new file mode 100644 index 000000000..3feb9617e --- /dev/null +++ b/packages/cli/src/core/local-state/index.ts @@ -0,0 +1,33 @@ +export { + deleteDevInstance, + isPidAlive, + readDevInstance, + writeDevInstance, +} from "./dev-instance.js"; +export { + type MetaReadResult, + readDataDirMeta, + writeDataDirMeta, +} from "./meta.js"; +export { + getDataDir, + getDevJsonPath, + getMetaJsonPath, + getStateDir, + STATE_DIR_NAME, +} from "./paths.js"; +export { + type DataDirMeta, + DataDirMetaSchema, + type DevInstance, + DevInstanceSchema, + type SeedState, + SeedStateSchema, +} from "./schema.js"; +export { + createJwtToken, + createServiceAuthorizationHeader, + createServiceToken, + isServiceSubject, + SERVICE_ROLE_EMAIL, +} from "./tokens.js"; diff --git a/packages/cli/src/core/local-state/meta.ts b/packages/cli/src/core/local-state/meta.ts new file mode 100644 index 000000000..adb0761d4 --- /dev/null +++ b/packages/cli/src/core/local-state/meta.ts @@ -0,0 +1,40 @@ +import { pathExists, readJsonFile, writeJsonFile } from "@/core/utils/fs.js"; +import { getMetaJsonPath } from "./paths.js"; +import { type DataDirMeta, DataDirMetaSchema } from "./schema.js"; + +export type MetaReadResult = + | { status: "ok"; meta: DataDirMeta } + | { status: "missing" } + | { status: "corrupt" }; + +/** + * Read `/meta.json`. Corrupt (unreadable or schema-invalid) meta is + * reported rather than thrown — callers warn and treat the data dir as new. + */ +export async function readDataDirMeta( + dataDir: string, +): Promise { + const metaPath = getMetaJsonPath(dataDir); + if (!(await pathExists(metaPath))) { + return { status: "missing" }; + } + + let parsed: unknown; + try { + parsed = await readJsonFile(metaPath); + } catch { + return { status: "corrupt" }; + } + + const result = DataDirMetaSchema.safeParse(parsed); + return result.success + ? { status: "ok", meta: result.data } + : { status: "corrupt" }; +} + +export async function writeDataDirMeta( + dataDir: string, + meta: DataDirMeta, +): Promise { + await writeJsonFile(getMetaJsonPath(dataDir), meta); +} diff --git a/packages/cli/src/core/local-state/paths.ts b/packages/cli/src/core/local-state/paths.ts new file mode 100644 index 000000000..dfbd4454d --- /dev/null +++ b/packages/cli/src/core/local-state/paths.ts @@ -0,0 +1,28 @@ +import { join } from "node:path"; + +/** Name of the gitignored per-project local state directory. */ +export const STATE_DIR_NAME = ".base44"; + +const DATA_DIR_NAME = "data"; +const DEV_JSON_FILE = "dev.json"; +const META_JSON_FILE = "meta.json"; + +/** `/.base44` — root of all local dev state for a project. */ +export function getStateDir(projectRoot: string): string { + return join(projectRoot, STATE_DIR_NAME); +} + +/** `/.base44/data` — file-backed NeDB collections + meta.json. */ +export function getDataDir(projectRoot: string): string { + return join(getStateDir(projectRoot), DATA_DIR_NAME); +} + +/** `/.base44/dev.json` — running dev-server instance descriptor. */ +export function getDevJsonPath(projectRoot: string): string { + return join(getStateDir(projectRoot), DEV_JSON_FILE); +} + +/** `/meta.json` — data dir metadata (owning app id, seed state). */ +export function getMetaJsonPath(dataDir: string): string { + return join(dataDir, META_JSON_FILE); +} diff --git a/packages/cli/src/core/local-state/schema.ts b/packages/cli/src/core/local-state/schema.ts new file mode 100644 index 000000000..21796c686 --- /dev/null +++ b/packages/cli/src/core/local-state/schema.ts @@ -0,0 +1,34 @@ +import { z } from "zod"; + +/** Seed application state; `null` until seeds have been applied. */ +export const SeedStateSchema = z + .object({ + hash: z.string(), + appliedAt: z.string(), + }) + .nullable(); + +export type SeedState = z.infer; + +/** Contents of `/meta.json`. */ +export const DataDirMetaSchema = z.object({ + formatVersion: z.literal(1), + appId: z.string(), + seed: SeedStateSchema.default(null), +}); + +export type DataDirMeta = z.infer; + +/** Contents of `/.base44/dev.json` (instance descriptor). */ +export const DevInstanceSchema = z.object({ + appId: z.string(), + url: z.string(), + port: z.number(), + pid: z.number(), + dataDir: z.string(), + adminToken: z.string(), + startedAt: z.string(), + seed: SeedStateSchema.default(null), +}); + +export type DevInstance = z.infer; diff --git a/packages/cli/src/core/local-state/tokens.ts b/packages/cli/src/core/local-state/tokens.ts new file mode 100644 index 000000000..041993bd6 --- /dev/null +++ b/packages/cli/src/core/local-state/tokens.ts @@ -0,0 +1,30 @@ +import jwt from "jsonwebtoken"; + +const LOCAL_DEV_SECRET = "LOCAL_DEV_SECRET"; + +/** + * Sentinel identity used for service-role (`asServiceRole`) requests in dev. + * In production Base44 injects a privileged service token; locally we mint a + * JWT for this subject and grant it full access (see `checkRLS`). + */ +export const SERVICE_ROLE_EMAIL = "server@server.com"; + +export const createJwtToken = (email: string) => { + return jwt.sign({ sub: email }, LOCAL_DEV_SECRET, { + expiresIn: "360d", + }); +}; + +/** + * Mints the JWT the local dev server resolves to the service-role principal + * (bypasses RLS and FLS). Sent as `Authorization: Bearer ` by the + * seed-script SDK client and as `Base44-Service-Authorization` for functions. + */ +export const createServiceToken = () => createJwtToken(SERVICE_ROLE_EMAIL); + +export const createServiceAuthorizationHeader = () => + `Bearer ${createServiceToken()}`; + +/** True when a JWT subject identifies the service-role principal. */ +export const isServiceSubject = (subject: string): boolean => + subject === SERVICE_ROLE_EMAIL; diff --git a/packages/cli/src/core/project/schema.ts b/packages/cli/src/core/project/schema.ts index 22938a990..f9bfcc476 100644 --- a/packages/cli/src/core/project/schema.ts +++ b/packages/cli/src/core/project/schema.ts @@ -48,6 +48,7 @@ export const ProjectConfigSchema = z.object({ agentsDir: z.string().optional().default("agents"), connectorsDir: z.string().optional().default("connectors"), authDir: z.string().optional().default("auth"), + seedDir: z.string().optional().default("seed"), plugin: PluginMetadataSchema.optional(), plugins: z.array(PluginReferenceSchema).optional().default([]), }); @@ -92,6 +93,8 @@ export const TestOverridesSchema = z.object({ }) .optional(), latestVersion: z.string().nullable().optional(), + /** Fakes the `base44/seed.ts` Deno run: skip spawning, return this exit code. */ + seedScript: z.object({ exitCode: z.number() }).optional(), }); export type TestOverrides = z.infer; diff --git a/packages/cli/src/core/resources/entity/index.ts b/packages/cli/src/core/resources/entity/index.ts index 90b197a7b..b630b6918 100644 --- a/packages/cli/src/core/resources/entity/index.ts +++ b/packages/cli/src/core/resources/entity/index.ts @@ -1,5 +1,6 @@ export * from "./api.js"; export * from "./config.js"; export * from "./deploy.js"; +export * from "./records-api.js"; export * from "./resource.js"; export * from "./schema.js"; diff --git a/packages/cli/src/core/resources/entity/records-api.ts b/packages/cli/src/core/resources/entity/records-api.ts new file mode 100644 index 000000000..30cebd6f4 --- /dev/null +++ b/packages/cli/src/core/resources/entity/records-api.ts @@ -0,0 +1,89 @@ +import type { KyResponse } from "ky"; +import { z } from "zod"; +import { getAppClient } from "@/core/clients/index.js"; +import { ApiError, SchemaValidationError } from "@/core/errors.js"; + +const EntityRecordsResponseSchema = z.array(z.record(z.string(), z.unknown())); + +export type RemoteEntityRecord = Record; + +export type DataEnv = "prod" | "dev"; + +/** Runtime entities API page size for `data pull`. */ +const PAGE_SIZE = 500; + +export interface FetchEntityRecordsOptions { + /** Data environment to read from; `dev` sends the `X-Data-Env: dev` header. */ + dataEnv?: DataEnv; + /** Filter forwarded as the `q` query param (JSON). */ + query?: Record; + /** Maximum number of records to fetch. */ + limit: number; +} + +export interface FetchEntityRecordsResult { + records: RemoteEntityRecord[]; + /** + * True when pagination stopped at `limit` with a full last page — more + * records may exist on the server. + */ + limitReached: boolean; +} + +/** + * Page through `GET apps/:appId/entities/:entityName` (limit/skip params, + * page size 500) until `limit` records are collected or the server runs out. + * Records are returned exactly as the API serves them (ids, created_by, + * created_date preserved). + */ +export async function fetchEntityRecords( + entityName: string, + options: FetchEntityRecordsOptions, +): Promise { + const client = + options.dataEnv === "dev" + ? getAppClient().extend({ headers: { "X-Data-Env": "dev" } }) + : getAppClient(); + + const records: RemoteEntityRecord[] = []; + while (records.length < options.limit) { + const pageLimit = Math.min(PAGE_SIZE, options.limit - records.length); + const searchParams: Record = { + limit: pageLimit, + skip: records.length, + }; + if (options.query) { + searchParams.q = JSON.stringify(options.query); + } + + let response: KyResponse; + try { + response = await client.get( + `entities/${encodeURIComponent(entityName)}`, + { + searchParams, + }, + ); + } catch (error) { + throw await ApiError.fromHttpError( + error, + `pulling records for entity "${entityName}"`, + ); + } + + const result = EntityRecordsResponseSchema.safeParse(await response.json()); + if (!result.success) { + throw new SchemaValidationError( + `Invalid records response for entity "${entityName}"`, + result.error, + ); + } + + records.push(...result.data); + if (result.data.length < pageLimit) { + return { records, limitReached: false }; + } + } + + return { records, limitReached: true }; +} diff --git a/packages/cli/src/core/resources/index.ts b/packages/cli/src/core/resources/index.ts index a8b80eaff..7551aafeb 100644 --- a/packages/cli/src/core/resources/index.ts +++ b/packages/cli/src/core/resources/index.ts @@ -4,3 +4,4 @@ export * from "./connector/index.js"; export * from "./entity/index.js"; export * from "./function/index.js"; export * from "./secret/index.js"; +export * from "./seed/index.js"; diff --git a/packages/cli/src/core/resources/seed/config.ts b/packages/cli/src/core/resources/seed/config.ts new file mode 100644 index 000000000..6ad64b85a --- /dev/null +++ b/packages/cli/src/core/resources/seed/config.ts @@ -0,0 +1,153 @@ +import { createHash } from "node:crypto"; +import { basename, dirname, join, relative, sep } from "node:path"; +import { globby } from "globby"; +import { CONFIG_FILE_EXTENSION_GLOB } from "@/core/consts.js"; +import { SchemaValidationError } from "@/core/errors.js"; +import { pathExists, readFile, readJsonFile } from "@/core/utils/fs.js"; +import { + type SeedRecord, + SeedRecordsFileSchema, + type SeedUser, + SeedUsersFileSchema, +} from "./schema.js"; + +/** + * File base name (case-insensitive) reserved for the users fixture. Never + * resolved as an entity fixture. + */ +export const USERS_FIXTURE_BASENAME = "users"; + +/** Fixed script hook path, relative to the project config dir. */ +const SEED_SCRIPT_FILENAME = "seed.ts"; + +export interface SeedUsersFixture { + path: string; + /** Path relative to the config dir, forward slashes (messages + hashing). */ + relPath: string; + users: SeedUser[]; +} + +export interface SeedRecordsFixture { + path: string; + relPath: string; + /** File base name without extension; resolved to an entity at apply time. */ + baseName: string; + records: SeedRecord[]; +} + +export interface SeedData { + users: SeedUsersFixture | null; + /** Entity fixtures, sorted alphabetically by filename (application order). */ + fixtures: SeedRecordsFixture[]; + /** Absolute path of `seed.ts` when present; run after fixtures, hashed here. */ + scriptPath: string | null; + /** `sha256:` over all seed files; drives the "seed changed" hint. */ + hash: string; +} + +function toPosix(path: string): string { + return path.split(sep).join("/"); +} + +function fileBaseName(path: string): string { + return basename(path).replace(/\.[^.]+$/, ""); +} + +/** + * Hash seed file contents so any change (add/remove/edit) changes the hash, + * independent of filesystem enumeration order. + */ +export function computeSeedHash( + entries: { relPath: string; bytes: Uint8Array }[], +): string { + const hash = createHash("sha256"); + const sorted = [...entries].sort((a, b) => + a.relPath.localeCompare(b.relPath), + ); + for (const entry of sorted) { + hash.update(entry.relPath); + hash.update("\0"); + hash.update(entry.bytes); + hash.update("\0"); + } + return `sha256:${hash.digest("hex")}`; +} + +/** + * Read and validate the project's seed fixtures (`/*.jsonc`). + * Returns null when the project has no seed files at all. Fixture contents + * are validated structurally here; per-record entity validation happens at + * apply time against the live schemas. + */ +export async function readSeedFiles(project: { + configPath: string; + seedDir: string; +}): Promise { + const configDir = dirname(project.configPath); + const seedDir = join(configDir, project.seedDir); + const scriptPath = join(configDir, SEED_SCRIPT_FILENAME); + const hasScript = await pathExists(scriptPath); + + const files = (await pathExists(seedDir)) + ? await globby(`*.${CONFIG_FILE_EXTENSION_GLOB}`, { + cwd: seedDir, + absolute: true, + }) + : []; + files.sort((a, b) => basename(a).localeCompare(basename(b))); + + if (files.length === 0 && !hasScript) { + return null; + } + + const hashFiles = hasScript ? [...files, scriptPath] : files; + const hash = computeSeedHash( + await Promise.all( + hashFiles.map(async (path) => ({ + relPath: toPosix(relative(configDir, path)), + bytes: new Uint8Array(await readFile(path)), + })), + ), + ); + + let users: SeedUsersFixture | null = null; + const fixtures: SeedRecordsFixture[] = []; + + for (const path of files) { + const relPath = toPosix(relative(configDir, path)); + const baseName = fileBaseName(path); + const parsed = await readJsonFile(path); + + if (baseName.toLowerCase() === USERS_FIXTURE_BASENAME) { + const result = SeedUsersFileSchema.safeParse(parsed); + if (!result.success) { + throw new SchemaValidationError( + "Invalid seed users file", + result.error, + path, + ); + } + users = { path, relPath, users: result.data }; + } else { + const result = SeedRecordsFileSchema.safeParse(parsed); + if (!result.success) { + throw new SchemaValidationError( + "Invalid seed fixture file", + result.error, + path, + ); + } + fixtures.push({ path, relPath, baseName, records: result.data }); + } + } + + return { users, fixtures, scriptPath: hasScript ? scriptPath : null, hash }; +} + +/** + * Normalize a name for fixture-to-entity resolution: lowercase with `-`/`_` + * stripped, so `team-member.jsonc` resolves the entity named "TeamMember". + */ +export function normalizeSeedName(name: string): string { + return name.toLowerCase().replace(/[-_]/g, ""); +} diff --git a/packages/cli/src/core/resources/seed/index.ts b/packages/cli/src/core/resources/seed/index.ts new file mode 100644 index 000000000..499048fd2 --- /dev/null +++ b/packages/cli/src/core/resources/seed/index.ts @@ -0,0 +1,26 @@ +export { + computeSeedHash, + normalizeSeedName, + readSeedFiles, + type SeedData, + type SeedRecordsFixture, + type SeedUsersFixture, + USERS_FIXTURE_BASENAME, +} from "./config.js"; +export { + type DevResetResult, + DevResetResultSchema, + emptySeedSummary, + type SeedEntityCounts, + SeedEntityCountsSchema, + type SeedMode, + SeedModeSchema, + type SeedRecord, + SeedRecordSchema, + SeedRecordsFileSchema, + type SeedSummary, + SeedSummarySchema, + type SeedUser, + SeedUserSchema, + SeedUsersFileSchema, +} from "./schema.js"; diff --git a/packages/cli/src/core/resources/seed/schema.ts b/packages/cli/src/core/resources/seed/schema.ts new file mode 100644 index 000000000..bde0f3f04 --- /dev/null +++ b/packages/cli/src/core/resources/seed/schema.ts @@ -0,0 +1,85 @@ +import { z } from "zod"; + +export const SeedModeSchema = z.enum(["upsert", "replace"]); + +export type SeedMode = z.infer; + +/** + * One entry of `users.jsonc`. Extra keys are custom User-entity fields and + * flow through to the seeded user document (validated against the User schema + * at apply time). + */ +export const SeedUserSchema = z.looseObject({ + email: z.email(), + role: z.enum(["admin", "user"]).default("user"), + password: z.string().optional(), + full_name: z.string().optional(), +}); + +export type SeedUser = z.infer; + +export const SeedUsersFileSchema = z.array(SeedUserSchema); + +/** + * One entry of an entity fixture file. `id` makes the record upsertable by + * id; `created_by` is a user email resolved at apply time. Everything else is + * entity data validated against the entity schema at apply time. + */ +export const SeedRecordSchema = z.looseObject({ + id: z.string().min(1).optional(), + created_by: z.string().optional(), +}); + +export type SeedRecord = z.infer; + +export const SeedRecordsFileSchema = z.array(SeedRecordSchema); + +export const SeedEntityCountsSchema = z.object({ + created: z.number(), + updated: z.number(), + skipped: z.number(), +}); + +export type SeedEntityCounts = z.infer; + +/** + * Result of one seed application. Returned by the applier, the + * `POST /_base44/dev/seed` admin endpoint, and `base44 dev seed --json`. + * `script` reports the `base44/seed.ts` step: null when the project has no + * script, `{ran: false}` plus a warning when it failed. + */ +export const SeedSummarySchema = z.object({ + applied: z.boolean(), + mode: SeedModeSchema, + users: z.number(), + records: z.record(z.string(), SeedEntityCountsSchema), + script: z.object({ ran: z.boolean() }).nullable(), + warnings: z.array(z.string()), +}); + +export type SeedSummary = z.infer; + +/** Summary for a run that found no seed files (nothing to apply). */ +export function emptySeedSummary(mode: SeedMode): SeedSummary { + return { + applied: false, + mode, + users: 0, + records: {}, + script: null, + warnings: ["No seed files found"], + }; +} + +/** + * Result of a dev-server reset. Returned by the `POST /_base44/dev/reset` + * admin endpoint and `base44 dev reset --json`. + */ +export const DevResetResultSchema = z.object({ + reset: z.literal(true), + seeded: z.boolean(), + dataDir: z.string(), + seed: SeedSummarySchema.nullable(), +}); + +export type DevResetResult = z.infer; diff --git a/packages/cli/src/core/seed-script/index.ts b/packages/cli/src/core/seed-script/index.ts new file mode 100644 index 000000000..3ce13535c --- /dev/null +++ b/packages/cli/src/core/seed-script/index.ts @@ -0,0 +1 @@ +export * from "./run-script.js"; diff --git a/packages/cli/src/core/seed-script/run-script.ts b/packages/cli/src/core/seed-script/run-script.ts new file mode 100644 index 000000000..c50ce7e9c --- /dev/null +++ b/packages/cli/src/core/seed-script/run-script.ts @@ -0,0 +1,112 @@ +import { spawn } from "node:child_process"; +import { copyFileSync } from "node:fs"; +import { pathToFileURL } from "node:url"; +import { file } from "tmp-promise"; +import { getSeedWrapperPath } from "@/core/assets.js"; +import { getTestOverrides } from "@/core/config.js"; +import { createServiceToken } from "@/core/local-state/index.js"; +import { getAppUserToken, getSiteUrl } from "@/core/project/api.js"; +import { verifyDenoInstalled } from "@/core/utils/index.js"; + +export interface RunSeedScriptOptions { + appId: string; + /** Absolute path of the project's `base44/seed.ts`. */ + scriptPath: string; + /** Base URL of the running local dev server the script seeds into. */ + localUrl: string; + /** Test seam: replaces `node:child_process.spawn` (skips the Deno check). */ + spawnImpl?: typeof spawn; + /** Test seam: wrapper file to run instead of the shipped asset. */ + wrapperPath?: string; +} + +export interface RunSeedScriptResult { + exitCode: number; +} + +interface RemoteCredentials { + accessToken: string; + appBaseUrl: string; + error: string; +} + +/** + * Remote credentials power `ctx.remote()` but are optional: scripts that only + * seed locally must work offline or before the app is published. When the + * fetch fails, the wrapper throws the recorded reason only if the script + * actually calls `ctx.remote()`. + */ +async function fetchRemoteCredentials(): Promise { + try { + const [accessToken, appBaseUrl] = await Promise.all([ + getAppUserToken(), + getSiteUrl(), + ]); + return { accessToken, appBaseUrl, error: "" }; + } catch (error) { + return { + accessToken: "", + appBaseUrl: "", + error: error instanceof Error ? error.message : String(error), + }; + } +} + +/** + * Run the project's `base44/seed.ts` in Deno via the seed wrapper. The + * wrapper builds the script's `ctx` (local service-role client, `remote()` + * factory, stderr logger) from the environment below. The child's stdout and + * stderr are both piped to the CLI's stderr so script output can never + * corrupt the `--json` stdout contract. + */ +export async function runSeedScript( + options: RunSeedScriptOptions, +): Promise { + const override = getTestOverrides()?.seedScript; + if (override) { + return { exitCode: override.exitCode }; + } + + if (!options.spawnImpl) { + verifyDenoInstalled("to run seed scripts"); + } + + // Copy the wrapper to a temp location outside node_modules. Same + // constraint as the exec wrapper: Deno 2.x treats files inside + // node_modules as Node modules and blocks npm: specifiers in them. + const tempWrapper = await file({ postfix: ".ts" }); + try { + copyFileSync(options.wrapperPath ?? getSeedWrapperPath(), tempWrapper.path); + const remote = await fetchRemoteCredentials(); + + const exitCode = await new Promise((resolvePromise) => { + const child = (options.spawnImpl ?? spawn)( + "deno", + ["run", "--allow-all", "--node-modules-dir=auto", tempWrapper.path], + { + env: { + ...process.env, + SCRIPT_PATH: pathToFileURL(options.scriptPath).href, + BASE44_APP_ID: options.appId, + BASE44_LOCAL_URL: options.localUrl, + BASE44_LOCAL_SERVICE_TOKEN: createServiceToken(), + BASE44_ACCESS_TOKEN: remote.accessToken, + BASE44_APP_BASE_URL: remote.appBaseUrl, + BASE44_REMOTE_ERROR: remote.error, + }, + stdio: ["ignore", "pipe", "pipe"], + }, + ); + + // `end: false` — process.stderr must not be closed when the child exits. + child.stdout?.pipe(process.stderr, { end: false }); + child.stderr?.pipe(process.stderr, { end: false }); + child.on("error", () => resolvePromise(1)); + child.on("close", (code) => resolvePromise(code ?? 1)); + }); + + return { exitCode }; + } finally { + tempWrapper.cleanup(); + } +} diff --git a/packages/cli/templates/backend-and-client/base44/seed/task.jsonc b/packages/cli/templates/backend-and-client/base44/seed/task.jsonc new file mode 100644 index 000000000..dfc18dc0f --- /dev/null +++ b/packages/cli/templates/backend-and-client/base44/seed/task.jsonc @@ -0,0 +1,29 @@ +// Seed records for the Task entity (file name matches the entity name). +// Applied automatically on the first `base44 dev` boot (empty local data) and +// on demand via `base44 dev seed`. +// +// - "id" is optional but recommended: records with a stable id are upserted +// when you re-run `base44 dev seed`; records without an id are only +// inserted in replace mode (first boot, `dev seed --replace`, `dev reset`) +// and skipped otherwise. +// - "created_by" references a seeded user's email from users.jsonc. +[ + { + "id": "task-welcome", + "title": "Welcome to Base44 — this task was seeded", + "completed": true, + "created_by": "admin@example.com" + }, + { + "id": "task-edit-seeds", + "title": "Edit base44/seed/*.jsonc and run `base44 dev seed`", + "completed": false, + "created_by": "user@example.com" + }, + { + "id": "task-reset", + "title": "Run `base44 dev reset` to start over with fresh seed data", + "completed": false, + "created_by": "user@example.com" + } +] diff --git a/packages/cli/templates/backend-and-client/base44/seed/users.jsonc b/packages/cli/templates/backend-and-client/base44/seed/users.jsonc new file mode 100644 index 000000000..7001f42c9 --- /dev/null +++ b/packages/cli/templates/backend-and-client/base44/seed/users.jsonc @@ -0,0 +1,21 @@ +// Local seed users for `base44 dev`. +// Applied automatically on the first dev-server boot (empty local data) and +// on demand via `base44 dev seed`. Users are upserted by email. +// +// - "role": "admin" or "user" (default "user") +// - "password": enables email/password login against the local dev server +// - "full_name" and any custom User-entity fields are optional +[ + { + "email": "admin@example.com", + "full_name": "Ada Admin", + "role": "admin", + "password": "admin1234" + }, + { + "email": "user@example.com", + "full_name": "Uma User", + "role": "user", + "password": "user1234" + } +] diff --git a/packages/cli/templates/backend-and-client/gitignore.ejs b/packages/cli/templates/backend-and-client/gitignore.ejs index 00d4323a7..95c69e427 100644 --- a/packages/cli/templates/backend-and-client/gitignore.ejs +++ b/packages/cli/templates/backend-and-client/gitignore.ejs @@ -21,3 +21,5 @@ dist # Base44 .app.json* .types/ +# Local dev state (`base44 dev` data and instance descriptor) +.base44/ diff --git a/packages/cli/templates/backend-only/base44/gitignore.ejs b/packages/cli/templates/backend-only/base44/gitignore.ejs index b8e4e2f9f..861555608 100644 --- a/packages/cli/templates/backend-only/base44/gitignore.ejs +++ b/packages/cli/templates/backend-only/base44/gitignore.ejs @@ -9,3 +9,5 @@ outputFileName: .gitignore # Base44 .app.json* .types/ +# Local dev state (`base44 dev` data and instance descriptor) +.base44/ diff --git a/packages/cli/tests/cli/data-dump.spec.ts b/packages/cli/tests/cli/data-dump.spec.ts new file mode 100644 index 000000000..4956a49c7 --- /dev/null +++ b/packages/cli/tests/cli/data-dump.spec.ts @@ -0,0 +1,149 @@ +import { describe, expect, it } from "vitest"; +import { waitForDevServer } from "./testkit/dev-utils.js"; +import { fixture, setupCLITests } from "./testkit/index.js"; + +interface DumpOutput { + entities: Record; + wrote: string[]; +} + +describe("data dump command", () => { + const t = setupCLITests(); + + const seedLocalData = async () => { + const result = await t.run("dev", "seed", "--json"); + t.expectResult(result).toSucceed(); + }; + + it("dumps local data offline into fixtures, skipping users", async () => { + // Given: offline-seeded data (2 tasks with ids, 1 team member) + await t.givenLoggedInWithProject(fixture("with-seed")); + await seedLocalData(); + + // When + const result = await t.run("data", "dump", "--force", "--json"); + + // Then + t.expectResult(result).toSucceed(); + const output = JSON.parse(result.stdout) as DumpOutput; + expect(output.entities).toEqual({ + Task: { pulled: 2, total: 2 }, + TeamMember: { pulled: 1, total: 1 }, + }); + expect(output.wrote).toHaveLength(2); + + const tasks = JSON.parse( + (await t.readProjectFile("base44/seed/task.jsonc")) as string, + ) as Record[]; + expect(tasks).toHaveLength(2); + const taskOne = tasks.find((task) => task.id === "task-1"); + expect(taskOne).toMatchObject({ + title: "First seeded task", + created_by: "admin@seed.dev", + }); + // NeDB's internal _id never leaks into fixtures + expect(Object.keys(taskOne ?? {})).not.toContain("_id"); + }); + + it("dumps from a running dev server via the admin export endpoint", async () => { + // Given: startup auto-seed applied 3 tasks (replace mode) + await t.givenLoggedInWithProject(fixture("with-seed")); + const handle = await t.runLive("dev"); + await waitForDevServer(handle); + + // When + const result = await t.run("data", "dump", "--force", "--json"); + + // Then + t.expectResult(result).toSucceed(); + const output = JSON.parse(result.stdout) as DumpOutput; + expect(output.entities.Task).toEqual({ pulled: 3, total: 3 }); + expect(output.entities.TeamMember).toEqual({ pulled: 1, total: 1 }); + + await handle.stop(); + }); + + it("dumps only the requested entity", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-seed")); + await seedLocalData(); + + // When + const result = await t.run( + "data", + "dump", + "--entity", + "Task", + "--force", + "--json", + ); + + // Then + t.expectResult(result).toSucceed(); + const output = JSON.parse(result.stdout) as DumpOutput; + expect(Object.keys(output.entities)).toEqual(["Task"]); + }); + + it("warns and skips when User is requested", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-seed")); + await seedLocalData(); + + // When + const result = await t.run("data", "dump", "--entity", "User", "--json"); + + // Then + t.expectResult(result).toSucceed(); + t.expectResult(result).toContain("skipping User"); + const output = JSON.parse(result.stdout) as DumpOutput; + expect(output.entities).toEqual({}); + expect(output.wrote).toEqual([]); + expect(await t.fileExists("base44/seed/user.jsonc")).toBe(false); + }); + + it("skips empty collections when dumping everything", async () => { + // Given: no seeding — collections exist but are empty + await t.givenLoggedInWithProject(fixture("with-entities")); + + // When + const result = await t.run("data", "dump", "--json"); + + // Then + t.expectResult(result).toSucceed(); + const output = JSON.parse(result.stdout) as DumpOutput; + expect(output.entities).toEqual({}); + expect(output.wrote).toEqual([]); + }); + + it("rejects an unknown --entity listing known names", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-seed")); + await seedLocalData(); + + // When + const result = await t.run("data", "dump", "--entity", "Ghost"); + + // Then + t.expectResult(result).toFail(); + t.expectResult(result).toContain('Unknown entity "Ghost"'); + t.expectResult(result).toContain("Task"); + }); + + it("writes to --out and requires --force for existing fixtures", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-seed")); + await seedLocalData(); + + // When: default out dir collides with existing fixtures + const withoutForce = await t.run("data", "dump"); + // When: a fresh --out dir does not + const withOut = await t.run("data", "dump", "--out", "exported", "--json"); + + // Then + t.expectResult(withoutForce).toFail(); + t.expectResult(withoutForce).toContain("--force"); + t.expectResult(withOut).toSucceed(); + expect(await t.fileExists("exported/task.jsonc")).toBe(true); + expect(await t.fileExists("exported/team-member.jsonc")).toBe(true); + }); +}); diff --git a/packages/cli/tests/cli/data-pull.spec.ts b/packages/cli/tests/cli/data-pull.spec.ts new file mode 100644 index 000000000..b8e6c8d14 --- /dev/null +++ b/packages/cli/tests/cli/data-pull.spec.ts @@ -0,0 +1,239 @@ +import { describe, expect, it } from "vitest"; +import { fixture, setupCLITests } from "./testkit/index.js"; + +interface CapturedRequest { + entityName: string; + query: Record; + dataEnvHeader: string | undefined; +} + +describe("data pull command", () => { + const t = setupCLITests(); + + /** + * Mock GET /api/apps/:appId/entities/:entityName serving slices of + * `records` per limit/skip (the runtime pagination contract), capturing + * each request for assertions. + */ + const mockEntityRecords = ( + recordsByEntity: Record[]>, + ): CapturedRequest[] => { + const captured: CapturedRequest[] = []; + t.api.mockRoute( + "GET", + `/api/apps/${t.api.appId}/entities/:entityName`, + (req, res) => { + const entityName = req.params.entityName as string; + captured.push({ + entityName, + query: req.query as Record, + dataEnvHeader: req.headers["x-data-env"] as string | undefined, + }); + const records = recordsByEntity[entityName] ?? []; + const skip = Number.parseInt((req.query.skip as string) ?? "0", 10); + const limit = Number.parseInt((req.query.limit as string) ?? "500", 10); + res.json(records.slice(skip, skip + limit)); + }, + ); + return captured; + }; + + const makeRecords = (count: number, prefix: string) => + Array.from({ length: count }, (_, i) => ({ + id: `${prefix}-${i + 1}`, + company: `Company ${i + 1}`, + created_by: "owner@example.com", + created_date: "2026-01-01T00:00:00.000Z", + })); + + it("pulls all project entities and writes fixtures with ids preserved", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-entities")); + mockEntityRecords({ + Customer: makeRecords(2, "cust"), + Product: [{ id: "prod-1", title: "Widget" }], + }); + + // When + const result = await t.run("data", "pull", "--json"); + + // Then + t.expectResult(result).toSucceed(); + const output = JSON.parse(result.stdout) as { + entities: Record; + wrote: string[]; + }; + expect(output.entities).toEqual({ + Customer: { pulled: 2, total: 2 }, + Product: { pulled: 1, total: 1 }, + }); + expect(output.wrote).toHaveLength(2); + + const customerFixture = JSON.parse( + (await t.readProjectFile("base44/seed/customer.jsonc")) as string, + ) as Record[]; + expect(customerFixture).toHaveLength(2); + expect(customerFixture[0]).toMatchObject({ + id: "cust-1", + created_by: "owner@example.com", + created_date: "2026-01-01T00:00:00.000Z", + }); + expect(await t.fileExists("base44/seed/product.jsonc")).toBe(true); + }); + + it("paginates with limit/skip until the server runs out", async () => { + // Given: 800 records = one full page of 500 + a short page of 300 + await t.givenLoggedInWithProject(fixture("with-entities")); + const captured = mockEntityRecords({ Customer: makeRecords(800, "c") }); + + // When + const result = await t.run( + "data", + "pull", + "--entity", + "Customer", + "--json", + ); + + // Then + t.expectResult(result).toSucceed(); + const output = JSON.parse(result.stdout) as { + entities: Record; + }; + expect(output.entities.Customer.pulled).toBe(800); + expect(captured.map((r) => [r.query.limit, r.query.skip])).toEqual([ + ["500", "0"], + ["500", "500"], + ]); + }); + + it("stops at --limit", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-entities")); + const captured = mockEntityRecords({ Customer: makeRecords(800, "c") }); + + // When + const result = await t.run( + "data", + "pull", + "--entity", + "Customer", + "--limit", + "600", + "--json", + ); + + // Then + t.expectResult(result).toSucceed(); + const output = JSON.parse(result.stdout) as { + entities: Record; + }; + expect(output.entities.Customer.pulled).toBe(600); + expect(captured.map((r) => [r.query.limit, r.query.skip])).toEqual([ + ["500", "0"], + ["100", "500"], + ]); + }); + + it("sends X-Data-Env only for --data-env dev", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-entities")); + const captured = mockEntityRecords({ Customer: [], Product: [] }); + + // When + await t.run("data", "pull", "--data-env", "dev", "--json"); + const devRequests = captured.length; + await t.run("data", "pull", "--json"); + + // Then + expect(devRequests).toBeGreaterThan(0); + for (const request of captured.slice(0, devRequests)) { + expect(request.dataEnvHeader).toBe("dev"); + } + for (const request of captured.slice(devRequests)) { + expect(request.dataEnvHeader).toBeUndefined(); + } + }); + + it("forwards --query as the q param", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-entities")); + const captured = mockEntityRecords({ Customer: [] }); + + // When + const result = await t.run( + "data", + "pull", + "--entity", + "Customer", + "--query", + '{"company":"Acme"}', + "--json", + ); + + // Then + t.expectResult(result).toSucceed(); + expect(captured[0].query.q).toBe('{"company":"Acme"}'); + }); + + it("rejects a --query that is not valid JSON", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-entities")); + + // When + const result = await t.run( + "data", + "pull", + "--query", + "{not-json", + "--json", + ); + + // Then + t.expectResult(result).toFail(); + t.expectResult(result).toContain("--query must be valid JSON"); + }); + + it("rejects an unknown --entity listing known names", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-entities")); + + // When + const result = await t.run("data", "pull", "--entity", "Ghost"); + + // Then + t.expectResult(result).toFail(); + t.expectResult(result).toContain('Unknown entity "Ghost"'); + t.expectResult(result).toContain("Customer"); + t.expectResult(result).toContain("Product"); + }); + + it("requires --force to overwrite existing fixtures when non-interactive", async () => { + // Given: with-seed already has task.jsonc / team-member.jsonc fixtures + await t.givenLoggedInWithProject(fixture("with-seed")); + mockEntityRecords({ Task: [], TeamMember: [] }); + + // When + const withoutForce = await t.run("data", "pull"); + const withForce = await t.run("data", "pull", "--force"); + + // Then + t.expectResult(withoutForce).toFail(); + t.expectResult(withoutForce).toContain("--force"); + t.expectResult(withForce).toSucceed(); + }); + + it("writes to --out instead of the seed dir", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-entities")); + mockEntityRecords({ Customer: makeRecords(1, "c"), Product: [] }); + + // When + const result = await t.run("data", "pull", "--out", "exported", "--json"); + + // Then + t.expectResult(result).toSucceed(); + expect(await t.fileExists("exported/customer.jsonc")).toBe(true); + expect(await t.fileExists("base44/seed/customer.jsonc")).toBe(false); + }); +}); diff --git a/packages/cli/tests/cli/dev-persistence.spec.ts b/packages/cli/tests/cli/dev-persistence.spec.ts new file mode 100644 index 000000000..7d45a7646 --- /dev/null +++ b/packages/cli/tests/cli/dev-persistence.spec.ts @@ -0,0 +1,289 @@ +import { spawn } from "node:child_process"; +import { mkdir, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { describe, expect, it } from "vitest"; +import { createServiceAuthorizationHeader } from "@/cli/dev/dev-server/auth/tokens.js"; +import { waitForDevServer } from "./testkit/dev-utils.js"; +import { fixture, setupCLITests } from "./testkit/index.js"; + +const DEV_JSON_PATH = ".base44/dev.json"; +const META_JSON_PATH = ".base44/data/meta.json"; + +/** Spawn a short-lived process and wait for it to exit, returning a dead pid. */ +async function getDeadPid(): Promise { + const child = spawn(process.execPath, ["-e", ""], { stdio: "ignore" }); + const pid = child.pid; + if (!pid) { + throw new Error("Failed to spawn process for dead-pid setup"); + } + await new Promise((resolve) => child.once("exit", resolve)); + return pid; +} + +describe("dev persistence", () => { + const t = setupCLITests(); + + const createProduct = async (devServerUrl: string, title: string) => { + const response = await fetch( + `${devServerUrl}/api/apps/${t.api.appId}/entities/Product`, + { + method: "POST", + headers: { + "Content-Type": "application/json", + "X-App-Id": t.api.appId, + }, + body: JSON.stringify({ title, price: 10 }), + }, + ); + expect(response.status).toBe(201); + return (await response.json()) as Record; + }; + + const listProducts = async (devServerUrl: string) => { + const response = await fetch( + `${devServerUrl}/api/apps/${t.api.appId}/entities/Product`, + { headers: { "X-App-Id": t.api.appId } }, + ); + expect(response.status).toBe(200); + return (await response.json()) as Record[]; + }; + + it("persists entity data across dev server restarts", async () => { + await t.givenLoggedInWithProject(fixture("with-entities")); + + const first = await t.runLive("dev"); + const firstUrl = await waitForDevServer(first); + await createProduct(firstUrl, "Widget"); + await first.stop(); + + expect(await t.fileExists(".base44/data/product.db")).toBe(true); + + const second = await t.runLive("dev"); + const secondUrl = await waitForDevServer(second); + const products = await listProducts(secondUrl); + await second.stop(); + + expect(products).toHaveLength(1); + expect(products[0].title).toBe("Widget"); + }); + + it("does not duplicate the CLI bootstrap user across restarts", async () => { + await t.givenLoggedInWithProject(fixture("with-entities")); + + const first = await t.runLive("dev"); + await waitForDevServer(first); + await first.stop(); + + const second = await t.runLive("dev"); + const secondUrl = await waitForDevServer(second); + const response = await fetch( + `${secondUrl}/api/apps/${t.api.appId}/entities/User`, + { + headers: { + Authorization: createServiceAuthorizationHeader(), + "X-App-Id": t.api.appId, + }, + }, + ); + expect(response.status).toBe(200); + const users = (await response.json()) as Record[]; + await second.stop(); + + expect(users.filter((u) => u.email === "test@example.com")).toHaveLength(1); + }); + + it("preserves data when entity files change and loads the new schema", async () => { + await t.givenLoggedInWithProject(fixture("with-entities")); + + const handle = await t.runLive("dev"); + const devServerUrl = await waitForDevServer(handle); + await createProduct(devServerUrl, "Widget"); + + await writeFile( + join(t.getTempDir(), "project", "base44", "entities", "note.json"), + JSON.stringify({ + name: "Note", + type: "object", + properties: { text: { type: "string" } }, + }), + ); + await handle.waitForOutput(/schemas reloaded \(data preserved\)/, 10000); + + const products = await listProducts(devServerUrl); + expect(products).toHaveLength(1); + expect(products[0].title).toBe("Widget"); + + const noteResponse = await fetch( + `${devServerUrl}/api/apps/${t.api.appId}/entities/Note`, + { headers: { "X-App-Id": t.api.appId } }, + ); + expect(noteResponse.status).toBe(200); + await expect(noteResponse.json()).resolves.toEqual([]); + + await handle.stop(); + }); + + it("--fresh wipes local data before starting", async () => { + await t.givenLoggedInWithProject(fixture("with-entities")); + + const first = await t.runLive("dev"); + const firstUrl = await waitForDevServer(first); + await createProduct(firstUrl, "Widget"); + await first.stop(); + + const second = await t.runLive("dev", "--fresh"); + const secondUrl = await waitForDevServer(second); + const products = await listProducts(secondUrl); + await second.stop(); + + expect(products).toEqual([]); + }); + + it("writes dev.json while running and removes it on shutdown", async () => { + await t.givenLoggedInWithProject(fixture("with-entities")); + + const handle = await t.runLive("dev"); + const devServerUrl = await waitForDevServer(handle); + + const raw = await t.readProjectFile(DEV_JSON_PATH); + expect(raw).not.toBeNull(); + const devJson = JSON.parse(raw as string) as Record; + expect(devJson.appId).toBe(t.api.appId); + expect(devJson.url).toBe(devServerUrl); + expect(devJson.port).toBe(Number(new URL(devServerUrl).port)); + expect(devJson.pid).toEqual(expect.any(Number)); + expect(devJson.adminToken).toMatch(/^[0-9a-f]{64}$/); + expect(devJson.startedAt).toEqual(expect.any(String)); + expect(devJson.dataDir).toContain(".base44"); + expect(devJson.seed).toBeNull(); + + await handle.stop(); + + expect(await t.fileExists(DEV_JSON_PATH)).toBe(false); + }); + + it("writes meta.json bound to the linked app", async () => { + await t.givenLoggedInWithProject(fixture("with-entities")); + + const handle = await t.runLive("dev"); + await waitForDevServer(handle); + await handle.stop(); + + const raw = await t.readProjectFile(META_JSON_PATH); + expect(raw).not.toBeNull(); + expect(JSON.parse(raw as string)).toEqual({ + formatVersion: 1, + appId: t.api.appId, + seed: null, + }); + }); + + it("refuses to start when local data belongs to another app unless --fresh", async () => { + await t.givenLoggedInWithProject(fixture("with-entities")); + + const first = await t.runLive("dev"); + await waitForDevServer(first); + await first.stop(); + + const metaPath = join(t.getTempDir(), "project", META_JSON_PATH); + await writeFile( + metaPath, + JSON.stringify({ formatVersion: 1, appId: "other-app", seed: null }), + ); + + const blocked = await t.runLive("dev"); + const result = await blocked.waitForExit(15000); + expect(result.exitCode).not.toBe(0); + t.expectResult(result).toContain('belongs to app "other-app"'); + t.expectResult(result).toContain("--fresh"); + + const fresh = await t.runLive("dev", "--fresh"); + const freshUrl = await waitForDevServer(fresh); + const products = await listProducts(freshUrl); + await fresh.stop(); + + expect(products).toEqual([]); + const meta = JSON.parse( + (await t.readProjectFile(META_JSON_PATH)) as string, + ) as Record; + expect(meta.appId).toBe(t.api.appId); + }); + + describe("dev status", () => { + it("reports a running server in --json mode without the admin token", async () => { + await t.givenLoggedInWithProject(fixture("with-entities")); + + const handle = await t.runLive("dev"); + const devServerUrl = await waitForDevServer(handle); + + const result = await t.run("dev", "status", "--json"); + t.expectResult(result).toSucceed(); + const status = JSON.parse(result.stdout) as Record; + expect(status.running).toBe(true); + expect(status.appId).toBe(t.api.appId); + expect(status.url).toBe(devServerUrl); + expect(status.port).toBe(Number(new URL(devServerUrl).port)); + expect(status.startedAt).toEqual(expect.any(String)); + expect(status.dataDir).toEqual(expect.any(String)); + expect(status.seed).toBeNull(); + expect(status).not.toHaveProperty("adminToken"); + expect(status).not.toHaveProperty("pid"); + + await handle.stop(); + }); + + it("prints a human summary for a running server", async () => { + await t.givenLoggedInWithProject(fixture("with-entities")); + + const handle = await t.runLive("dev"); + const devServerUrl = await waitForDevServer(handle); + + const result = await t.run("dev", "status"); + t.expectResult(result).toSucceed(); + t.expectResult(result).toContain("Dev server is running at"); + t.expectResult(result).toContain(devServerUrl); + t.expectResult(result).toContain(t.api.appId); + + await handle.stop(); + }); + + it("reports not running when no dev server is up", async () => { + await t.givenLoggedInWithProject(fixture("with-entities")); + + const result = await t.run("dev", "status", "--json"); + t.expectResult(result).toSucceed(); + expect(JSON.parse(result.stdout)).toEqual({ running: false }); + + const human = await t.run("dev", "status"); + t.expectResult(human).toSucceed(); + t.expectResult(human).toContain("No dev server is running"); + }); + + it("treats a dev.json left by a dead process as stale and deletes it", async () => { + await t.givenLoggedInWithProject(fixture("with-entities")); + + const devJsonPath = join(t.getTempDir(), "project", DEV_JSON_PATH); + await mkdir(join(t.getTempDir(), "project", ".base44"), { + recursive: true, + }); + await writeFile( + devJsonPath, + JSON.stringify({ + appId: t.api.appId, + url: "http://localhost:4400", + port: 4400, + pid: await getDeadPid(), + dataDir: join(t.getTempDir(), "project", ".base44", "data"), + adminToken: "a".repeat(64), + startedAt: new Date().toISOString(), + seed: null, + }), + ); + + const result = await t.run("dev", "status", "--json"); + t.expectResult(result).toSucceed(); + expect(JSON.parse(result.stdout)).toEqual({ running: false }); + expect(await t.fileExists(DEV_JSON_PATH)).toBe(false); + }); + }); +}); diff --git a/packages/cli/tests/cli/dev-seed-script.spec.ts b/packages/cli/tests/cli/dev-seed-script.spec.ts new file mode 100644 index 000000000..aff97be3c --- /dev/null +++ b/packages/cli/tests/cli/dev-seed-script.spec.ts @@ -0,0 +1,176 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { describe, expect, it } from "vitest"; +import { waitForDevServer } from "./testkit/dev-utils.js"; +import { fixture, setupCLITests } from "./testkit/index.js"; + +describe("dev seed script (base44/seed.ts)", () => { + const t = setupCLITests(); + + const writeSeedScript = async () => { + await writeFile( + join(t.getTempDir(), "project", "base44", "seed.ts"), + "export default async (ctx) => { ctx.log('seeding'); };", + ); + }; + + describe("dev seed (offline, temporary instance)", () => { + it("applies fixtures and runs the script via a temporary dev server", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-seed")); + await writeSeedScript(); + t.givenSeedScriptResult(0); + + // When + const result = await t.run("dev", "seed", "--json"); + + // Then + t.expectResult(result).toSucceed(); + const summary = JSON.parse(result.stdout) as Record; + expect(summary).toMatchObject({ + applied: true, + mode: "upsert", + users: 2, + records: { + Task: { created: 2, updated: 0, skipped: 1 }, + TeamMember: { created: 1, updated: 0, skipped: 0 }, + }, + script: { ran: true }, + warnings: [], + }); + }); + + it("keeps fixture results, warns, and exits non-zero when the script fails", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-seed")); + await writeSeedScript(); + t.givenSeedScriptResult(3); + + // When + const result = await t.run("dev", "seed", "--json"); + + // Then + t.expectResult(result).toFail(); + const summary = JSON.parse(result.stdout) as { + records: Record; + script: { ran: boolean }; + warnings: string[]; + }; + expect(summary.records.Task).toEqual({ + created: 2, + updated: 0, + skipped: 1, + }); + expect(summary.script).toEqual({ ran: false }); + expect(summary.warnings.join("\n")).toContain("exited with code 3"); + }); + }); + + describe("dev seed (live server)", () => { + it("runs the script step against the running server", async () => { + // Given: server starts without a script, so startup seeding is fixture-only + await t.givenLoggedInWithProject(fixture("with-seed")); + t.givenSeedScriptResult(0); + const handle = await t.runLive("dev"); + await waitForDevServer(handle); + await writeSeedScript(); + + // When + const result = await t.run("dev", "seed", "--json"); + + // Then + t.expectResult(result).toSucceed(); + const summary = JSON.parse(result.stdout) as Record; + expect(summary).toMatchObject({ + applied: true, + mode: "upsert", + script: { ran: true }, + }); + + await handle.stop(); + }); + }); + + describe("startup auto-seed", () => { + it("runs the script after listen and logs a single summary", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-seed")); + await writeSeedScript(); + t.givenSeedScriptResult(0); + + // When + const handle = await t.runLive("dev"); + await waitForDevServer(handle); + await handle.waitForOutput(/Seeds applied/); + + // Then: seed state recorded in dev.json + const devJson = JSON.parse( + (await t.readProjectFile(".base44/dev.json")) as string, + ) as Record; + expect(devJson.seed).toMatchObject({ + hash: expect.stringMatching(/^sha256:/), + }); + + await handle.stop(); + }); + + it("keeps serving when the script fails", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-seed")); + await writeSeedScript(); + t.givenSeedScriptResult(1); + + // When: the server still comes up + const handle = await t.runLive("dev"); + await waitForDevServer(handle); + + // Then: fixtures applied, warning logged to stderr + expect(handle.stderr.join("")).toContain("exited with code 1"); + const result = await handle.stop(); + expect(result.stdout).toContain("Seeds applied"); + }); + }); + + describe("dev reset", () => { + it("re-seeds and runs the script via a temporary instance (offline)", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-seed")); + await writeSeedScript(); + t.givenSeedScriptResult(0); + + // When + const result = await t.run("dev", "reset", "--force", "--json"); + + // Then + t.expectResult(result).toSucceed(); + const reset = JSON.parse(result.stdout) as Record; + expect(reset).toMatchObject({ + reset: true, + seeded: true, + seed: { + mode: "replace", + records: { Task: { created: 3, updated: 0, skipped: 0 } }, + script: { ran: true }, + }, + }); + }); + + it("exits non-zero when the script fails during reset", async () => { + // Given + await t.givenLoggedInWithProject(fixture("with-seed")); + await writeSeedScript(); + t.givenSeedScriptResult(2); + + // When + const result = await t.run("dev", "reset", "--force", "--json"); + + // Then + t.expectResult(result).toFail(); + const reset = JSON.parse(result.stdout) as { + seed: { script: { ran: boolean }; warnings: string[] }; + }; + expect(reset.seed.script).toEqual({ ran: false }); + expect(reset.seed.warnings.join("\n")).toContain("exited with code 2"); + }); + }); +}); diff --git a/packages/cli/tests/cli/dev-seed.spec.ts b/packages/cli/tests/cli/dev-seed.spec.ts new file mode 100644 index 000000000..3f49cbcf5 --- /dev/null +++ b/packages/cli/tests/cli/dev-seed.spec.ts @@ -0,0 +1,490 @@ +import { mkdir, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { describe, expect, it } from "vitest"; +import { createServiceAuthorizationHeader } from "@/cli/dev/dev-server/auth/tokens.js"; +import { waitForDevServer } from "./testkit/dev-utils.js"; +import { fixture, setupCLITests } from "./testkit/index.js"; + +const DEV_JSON_PATH = ".base44/dev.json"; +const META_JSON_PATH = ".base44/data/meta.json"; + +describe("dev seeding", () => { + const t = setupCLITests(); + + const serviceHeaders = () => ({ + Authorization: createServiceAuthorizationHeader(), + "X-App-Id": t.api.appId, + }); + + const listEntity = async (devServerUrl: string, entityName: string) => { + const response = await fetch( + `${devServerUrl}/api/apps/${t.api.appId}/entities/${entityName}`, + { headers: serviceHeaders() }, + ); + expect(response.status).toBe(200); + return (await response.json()) as Record[]; + }; + + const createTask = async (devServerUrl: string, title: string) => { + const response = await fetch( + `${devServerUrl}/api/apps/${t.api.appId}/entities/Task`, + { + method: "POST", + headers: { "Content-Type": "application/json", ...serviceHeaders() }, + body: JSON.stringify({ title }), + }, + ); + expect(response.status).toBe(201); + return (await response.json()) as Record; + }; + + const readDevJson = async () => { + const raw = await t.readProjectFile(DEV_JSON_PATH); + expect(raw).not.toBeNull(); + return JSON.parse(raw as string) as Record; + }; + + const projectFile = (...segments: string[]) => + join(t.getTempDir(), "project", ...segments); + + const writeSeedFile = async (name: string, records: unknown) => { + await mkdir(projectFile("base44", "seed"), { recursive: true }); + await writeFile( + projectFile("base44", "seed", name), + JSON.stringify(records, null, 2), + ); + }; + + describe("auto-seed on dev startup", () => { + it("applies users and entity fixtures on first boot", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const handle = await t.runLive("dev"); + const url = await waitForDevServer(handle); + + const tasks = await listEntity(url, "Task"); + expect(tasks).toHaveLength(3); + + const users = await listEntity(url, "User"); + const emails = users.map((u) => u.email); + expect(emails).toContain("test@example.com"); // CLI login user + expect(emails).toContain("admin@seed.dev"); + expect(emails).toContain("member@seed.dev"); + + // created_by attribution points at the seeded user + const admin = users.find((u) => u.email === "admin@seed.dev"); + const taskOne = tasks.find((task) => task.id === "task-1"); + expect(taskOne?.title).toBe("First seeded task"); + expect(taskOne?.completed).toBe(true); + expect(taskOne?.created_by).toBe("admin@seed.dev"); + expect(taskOne?.created_by_id).toBe(admin?.id); + expect(taskOne?.created_date).toEqual(expect.any(String)); + + // kebab-case fixture file resolves to the TeamMember entity + const teamMembers = await listEntity(url, "TeamMember"); + expect(teamMembers).toHaveLength(1); + expect(teamMembers[0].id).toBe("tm-1"); + + // seed state recorded in dev.json and meta.json + const devJson = await readDevJson(); + expect(devJson.seed).toMatchObject({ + hash: expect.stringMatching(/^sha256:/), + appliedAt: expect.any(String), + }); + const meta = JSON.parse( + (await t.readProjectFile(META_JSON_PATH)) as string, + ) as Record; + expect(meta.seed).toEqual(devJson.seed); + + await handle.stop(); + }); + + it("does not re-apply seeds on a plain restart", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const first = await t.runLive("dev"); + const firstUrl = await waitForDevServer(first); + await createTask(firstUrl, "Manual task"); + await first.stop(); + + const second = await t.runLive("dev"); + const secondUrl = await waitForDevServer(second); + const tasks = await listEntity(secondUrl, "Task"); + await second.stop(); + + // replace-mode re-seed would drop the manual task back to 3 + expect(tasks).toHaveLength(4); + }); + + it("hints when seed files changed since the last apply", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const first = await t.runLive("dev"); + await waitForDevServer(first); + await first.stop(); + + await writeSeedFile("task.jsonc", [ + { id: "task-1", title: "Changed title", created_by: "admin@seed.dev" }, + ]); + + const second = await t.runLive("dev"); + const secondUrl = await waitForDevServer(second); + await second.waitForOutput(/Seed files changed/); + + // hint only — data stays as-is until `dev seed` is run + const tasks = await listEntity(secondUrl, "Task"); + expect(tasks).toHaveLength(3); + await second.stop(); + }); + + it("--fresh wipes local data and re-seeds", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const first = await t.runLive("dev"); + const firstUrl = await waitForDevServer(first); + await createTask(firstUrl, "Manual task"); + await first.stop(); + + const fresh = await t.runLive("dev", "--fresh"); + const freshUrl = await waitForDevServer(fresh); + const tasks = await listEntity(freshUrl, "Task"); + await fresh.stop(); + + expect(tasks).toHaveLength(3); + expect(tasks.map((task) => task.title)).not.toContain("Manual task"); + }); + }); + + describe("dev seed command", () => { + it("applies seeds offline (no dev server) in upsert mode", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const result = await t.run("dev", "seed", "--json"); + + t.expectResult(result).toSucceed(); + const summary = JSON.parse(result.stdout) as Record; + expect(summary).toMatchObject({ + applied: true, + mode: "upsert", + users: 2, + records: { + // id-less record is skipped in upsert mode, even on empty data + Task: { created: 2, updated: 0, skipped: 1 }, + TeamMember: { created: 1, updated: 0, skipped: 0 }, + }, + script: null, + warnings: [], + }); + + // dev startup then sees seeded (non-empty) data and does not re-seed + const handle = await t.runLive("dev"); + const url = await waitForDevServer(handle); + const tasks = await listEntity(url, "Task"); + await handle.stop(); + expect(tasks).toHaveLength(2); + }); + + it("upserts by id and skips id-less records against a live server", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const handle = await t.runLive("dev"); + const url = await waitForDevServer(handle); + + await writeSeedFile("task.jsonc", [ + { + id: "task-1", + title: "First seeded task (edited)", + completed: true, + created_by: "admin@seed.dev", + }, + { id: "task-2", title: "Second seeded task" }, + { title: "Task without a stable id" }, + ]); + + const result = await t.run("dev", "seed", "--json"); + t.expectResult(result).toSucceed(); + const summary = JSON.parse(result.stdout) as Record; + expect(summary).toMatchObject({ + applied: true, + mode: "upsert", + users: 2, + records: { + Task: { created: 0, updated: 2, skipped: 1 }, + TeamMember: { created: 0, updated: 1, skipped: 0 }, + }, + }); + + const tasks = await listEntity(url, "Task"); + expect(tasks).toHaveLength(3); + expect(tasks.find((task) => task.id === "task-1")?.title).toBe( + "First seeded task (edited)", + ); + + await handle.stop(); + }); + + it("--replace truncates seeded collections and keeps the CLI login user", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const handle = await t.runLive("dev"); + const url = await waitForDevServer(handle); + await createTask(url, "Manual task"); + + const result = await t.run( + "dev", + "seed", + "--replace", + "--force", + "--json", + ); + t.expectResult(result).toSucceed(); + const summary = JSON.parse(result.stdout) as Record; + expect(summary).toMatchObject({ + mode: "replace", + records: { Task: { created: 3, updated: 0, skipped: 0 } }, + }); + + const tasks = await listEntity(url, "Task"); + expect(tasks).toHaveLength(3); + expect(tasks.map((task) => task.title)).not.toContain("Manual task"); + + const users = await listEntity(url, "User"); + expect(users.map((u) => u.email)).toContain("test@example.com"); + + await handle.stop(); + }); + + it("requires --force for --replace in non-interactive mode", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const result = await t.run("dev", "seed", "--replace"); + + t.expectResult(result).toFail(); + t.expectResult(result).toContain("--force"); + }); + + it("prints per-entity counts in human mode", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const result = await t.run("dev", "seed"); + + t.expectResult(result).toSucceed(); + t.expectResult(result).toContain("Users: 2 seeded"); + t.expectResult(result).toContain("Task: 2 created, 0 updated, 1 skipped"); + }); + + it("reports validation errors citing file and index", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + await writeSeedFile("task.jsonc", [ + { id: "task-1", title: "Valid" }, + { id: "task-2", title: "Broken", completed: "yes" }, + ]); + + const result = await t.run("dev", "seed", "--json"); + + t.expectResult(result).toFail(); + t.expectResult(result).toContain("seed/task.jsonc"); + t.expectResult(result).toContain("at index 1"); + }); + + it("fails when created_by references an unknown user", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + await writeSeedFile("task.jsonc", [ + { id: "task-1", title: "Orphan", created_by: "ghost@nowhere.dev" }, + ]); + + const result = await t.run("dev", "seed", "--json"); + + t.expectResult(result).toFail(); + t.expectResult(result).toContain("created_by references unknown user"); + t.expectResult(result).toContain("ghost@nowhere.dev"); + t.expectResult(result).toContain("at index 0"); + }); + + it("warns about fixtures that match no entity without failing", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + await writeSeedFile("ghost.jsonc", [{ id: "g-1", spooky: true }]); + + const result = await t.run("dev", "seed", "--json"); + + t.expectResult(result).toSucceed(); + const summary = JSON.parse(result.stdout) as { + records: Record; + warnings: string[]; + }; + expect(summary.warnings.join("\n")).toContain("seed/ghost.jsonc"); + expect(summary.records).not.toHaveProperty("Ghost"); + expect(summary.records).toHaveProperty("Task"); + }); + }); + + describe("seeded users", () => { + it("can log in with the seeded password", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const handle = await t.runLive("dev"); + const url = await waitForDevServer(handle); + + const login = (password: string) => + fetch(`${url}/api/apps/${t.api.appId}/auth/login`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ email: "member@seed.dev", password }), + }); + + const ok = await login("seedmember1"); + expect(ok.status).toBe(200); + const body = (await ok.json()) as Record; + expect(body.access_token).toEqual(expect.any(String)); + + const bad = await login("wrong-password"); + expect(bad.status).toBe(400); + + await handle.stop(); + }); + }); + + describe("admin endpoints", () => { + it("rejects requests without or with a wrong admin token", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const handle = await t.runLive("dev"); + const url = await waitForDevServer(handle); + + const noToken = await fetch(`${url}/_base44/dev/status`); + expect(noToken.status).toBe(401); + + const wrongToken = await fetch(`${url}/_base44/dev/status`, { + headers: { "x-base44-dev-admin": "f".repeat(64) }, + }); + expect(wrongToken.status).toBe(401); + + const seedNoToken = await fetch(`${url}/_base44/dev/seed`, { + method: "POST", + }); + expect(seedNoToken.status).toBe(401); + + const resetNoToken = await fetch(`${url}/_base44/dev/reset`, { + method: "POST", + }); + expect(resetNoToken.status).toBe(401); + + await handle.stop(); + }); + + it("serves status, seed, and reset with the admin token", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const handle = await t.runLive("dev"); + const url = await waitForDevServer(handle); + const { adminToken } = (await readDevJson()) as { adminToken: string }; + const headers = { + "x-base44-dev-admin": adminToken, + "Content-Type": "application/json", + }; + + const statusResponse = await fetch(`${url}/_base44/dev/status`, { + headers, + }); + expect(statusResponse.status).toBe(200); + const status = (await statusResponse.json()) as Record; + expect(status.appId).toBe(t.api.appId); + expect(status.port).toBe(Number(new URL(url).port)); + expect(status.startedAt).toEqual(expect.any(String)); + expect(status.seed).toMatchObject({ + hash: expect.stringMatching(/^sha256:/), + }); + expect(status.collections).toMatchObject({ + task: 3, + teammember: 1, + user: 3, + }); + + const seedResponse = await fetch(`${url}/_base44/dev/seed`, { + method: "POST", + headers, + body: JSON.stringify({ mode: "upsert" }), + }); + expect(seedResponse.status).toBe(200); + const summary = (await seedResponse.json()) as Record; + expect(summary).toMatchObject({ applied: true, mode: "upsert" }); + + const resetResponse = await fetch(`${url}/_base44/dev/reset`, { + method: "POST", + headers, + }); + expect(resetResponse.status).toBe(200); + const reset = (await resetResponse.json()) as Record; + expect(reset).toMatchObject({ reset: true, seeded: true }); + + await handle.stop(); + }); + }); + + describe("dev reset command", () => { + it("resets and re-seeds against a live server", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const handle = await t.runLive("dev"); + const url = await waitForDevServer(handle); + await createTask(url, "Manual task"); + + const result = await t.run("dev", "reset", "--force", "--json"); + t.expectResult(result).toSucceed(); + const reset = JSON.parse(result.stdout) as Record; + expect(reset).toMatchObject({ + reset: true, + seeded: true, + dataDir: expect.stringContaining(".base44"), + seed: { + mode: "replace", + records: { Task: { created: 3, updated: 0, skipped: 0 } }, + }, + }); + + const tasks = await listEntity(url, "Task"); + expect(tasks).toHaveLength(3); + expect(tasks.map((task) => task.title)).not.toContain("Manual task"); + + // bootstrap CLI login user was re-inserted after the wipe + const users = await listEntity(url, "User"); + expect(users.map((u) => u.email)).toContain("test@example.com"); + + await handle.stop(); + }); + + it("resets and re-seeds offline", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const first = await t.runLive("dev"); + const firstUrl = await waitForDevServer(first); + await createTask(firstUrl, "Manual task"); + await first.stop(); + + const result = await t.run("dev", "reset", "--force", "--json"); + t.expectResult(result).toSucceed(); + const reset = JSON.parse(result.stdout) as Record; + expect(reset).toMatchObject({ reset: true, seeded: true }); + + const second = await t.runLive("dev"); + const secondUrl = await waitForDevServer(second); + const tasks = await listEntity(secondUrl, "Task"); + await second.stop(); + + expect(tasks).toHaveLength(3); + expect(tasks.map((task) => task.title)).not.toContain("Manual task"); + }); + + it("requires --force in non-interactive mode", async () => { + await t.givenLoggedInWithProject(fixture("with-seed")); + + const result = await t.run("dev", "reset"); + + t.expectResult(result).toFail(); + t.expectResult(result).toContain("--force"); + }); + }); +}); diff --git a/packages/cli/tests/cli/testkit/CLITestkit.ts b/packages/cli/tests/cli/testkit/CLITestkit.ts index 875a62e4f..491c75cbd 100644 --- a/packages/cli/tests/cli/testkit/CLITestkit.ts +++ b/packages/cli/tests/cli/testkit/CLITestkit.ts @@ -49,6 +49,7 @@ export interface RunLiveHandle { interface TestOverrides { appConfig?: { id: string; projectRoot: string }; latestVersion?: string | null; + seedScript?: { exitCode: number }; } export class CLITestkit { @@ -129,6 +130,14 @@ export class CLITestkit { this.testOverrides.latestVersion = version; } + /** + * Fake the `base44/seed.ts` Deno run: the CLI skips spawning Deno and + * reports the given exit code as the script result. + */ + givenSeedScriptResult(exitCode: number): void { + this.testOverrides.seedScript = { exitCode }; + } + /** Simulate piped stdin for the next run() call */ givenStdin(content: string): void { this.stdinContent = content; diff --git a/packages/cli/tests/cli/testkit/index.ts b/packages/cli/tests/cli/testkit/index.ts index f4309b7ec..a536ceb2a 100644 --- a/packages/cli/tests/cli/testkit/index.ts +++ b/packages/cli/tests/cli/testkit/index.ts @@ -35,6 +35,9 @@ export interface TestContext { givenLatestVersion: (version: string | null | undefined) => void; + /** Fake the base44/seed.ts Deno run with the given exit code */ + givenSeedScriptResult: (exitCode: number) => void; + /** Simulate piped stdin for the next run() call */ givenStdin: (content: string) => void; @@ -126,6 +129,8 @@ export function setupCLITests(): TestContext { await getKit().givenProject(fixturePath); }, givenLatestVersion: (version) => getKit().givenLatestVersion(version), + givenSeedScriptResult: (exitCode) => + getKit().givenSeedScriptResult(exitCode), givenStdin: (content) => getKit().givenStdin(content), givenEnv: (vars) => getKit().givenEnv(vars), diff --git a/packages/cli/tests/core/local-state.spec.ts b/packages/cli/tests/core/local-state.spec.ts new file mode 100644 index 000000000..966429ce1 --- /dev/null +++ b/packages/cli/tests/core/local-state.spec.ts @@ -0,0 +1,170 @@ +import { spawn } from "node:child_process"; +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { + type DevInstance, + deleteDevInstance, + getDataDir, + getDevJsonPath, + getMetaJsonPath, + getStateDir, + isPidAlive, + readDataDirMeta, + readDevInstance, + writeDataDirMeta, + writeDevInstance, +} from "@/core/local-state/index.js"; +import { pathExists } from "@/core/utils/fs.js"; + +/** Spawn a short-lived process and wait for it to exit, returning a dead pid. */ +async function getDeadPid(): Promise { + const child = spawn(process.execPath, ["-e", ""], { stdio: "ignore" }); + const pid = child.pid; + if (!pid) { + throw new Error("Failed to spawn process for dead-pid setup"); + } + await new Promise((resolve) => child.once("exit", resolve)); + return pid; +} + +function buildInstance(overrides: Partial = {}): DevInstance { + return { + appId: "app-123", + url: "http://localhost:4400", + port: 4400, + pid: process.pid, + dataDir: "/tmp/data", + adminToken: "a".repeat(64), + startedAt: "2026-01-01T00:00:00.000Z", + seed: null, + ...overrides, + }; +} + +describe("local-state", () => { + let projectRoot: string; + + beforeEach(async () => { + projectRoot = await mkdtemp(join(tmpdir(), "b44-local-state-")); + }); + + afterEach(async () => { + await rm(projectRoot, { recursive: true, force: true }); + }); + + describe("paths", () => { + it("derives state, data, dev.json and meta.json paths from the project root", () => { + expect(getStateDir(projectRoot)).toBe(join(projectRoot, ".base44")); + expect(getDataDir(projectRoot)).toBe( + join(projectRoot, ".base44", "data"), + ); + expect(getDevJsonPath(projectRoot)).toBe( + join(projectRoot, ".base44", "dev.json"), + ); + expect(getMetaJsonPath(getDataDir(projectRoot))).toBe( + join(projectRoot, ".base44", "data", "meta.json"), + ); + }); + }); + + describe("data dir meta", () => { + it("round-trips meta.json", async () => { + const dataDir = getDataDir(projectRoot); + + await writeDataDirMeta(dataDir, { + formatVersion: 1, + appId: "app-123", + seed: null, + }); + + const result = await readDataDirMeta(dataDir); + expect(result).toEqual({ + status: "ok", + meta: { formatVersion: 1, appId: "app-123", seed: null }, + }); + }); + + it("reports missing meta.json", async () => { + const result = await readDataDirMeta(getDataDir(projectRoot)); + expect(result).toEqual({ status: "missing" }); + }); + + it("reports corrupt meta.json for unparseable JSON", async () => { + const dataDir = getDataDir(projectRoot); + await mkdir(dataDir, { recursive: true }); + await writeFile(getMetaJsonPath(dataDir), "not json {{{"); + + const result = await readDataDirMeta(dataDir); + expect(result).toEqual({ status: "corrupt" }); + }); + + it("reports corrupt meta.json for schema-invalid content", async () => { + const dataDir = getDataDir(projectRoot); + await mkdir(dataDir, { recursive: true }); + await writeFile( + getMetaJsonPath(dataDir), + JSON.stringify({ formatVersion: 999, appId: 42 }), + ); + + const result = await readDataDirMeta(dataDir); + expect(result).toEqual({ status: "corrupt" }); + }); + }); + + describe("dev instance descriptor", () => { + it("round-trips dev.json for a live pid", async () => { + const instance = buildInstance(); + + await writeDevInstance(projectRoot, instance); + + await expect(readDevInstance(projectRoot)).resolves.toEqual(instance); + }); + + it("returns null when dev.json is missing", async () => { + await expect(readDevInstance(projectRoot)).resolves.toBeNull(); + }); + + it("deletes dev.json and returns null when the pid is not alive", async () => { + const instance = buildInstance({ pid: await getDeadPid() }); + await writeDevInstance(projectRoot, instance); + + await expect(readDevInstance(projectRoot)).resolves.toBeNull(); + await expect(pathExists(getDevJsonPath(projectRoot))).resolves.toBe( + false, + ); + }); + + it("deletes dev.json and returns null when the content is invalid", async () => { + await mkdir(getStateDir(projectRoot), { recursive: true }); + await writeFile(getDevJsonPath(projectRoot), '{"port": "nope"}'); + + await expect(readDevInstance(projectRoot)).resolves.toBeNull(); + await expect(pathExists(getDevJsonPath(projectRoot))).resolves.toBe( + false, + ); + }); + + it("deletes dev.json on deleteDevInstance and tolerates a missing file", async () => { + await writeDevInstance(projectRoot, buildInstance()); + const raw = await readFile(getDevJsonPath(projectRoot), "utf-8"); + expect(JSON.parse(raw).adminToken).toBe("a".repeat(64)); + + await deleteDevInstance(projectRoot); + await expect(pathExists(getDevJsonPath(projectRoot))).resolves.toBe( + false, + ); + + // Second delete is a no-op. + await expect(deleteDevInstance(projectRoot)).resolves.toBeUndefined(); + }); + }); + + describe("isPidAlive", () => { + it("is true for the current process and false for an exited one", async () => { + expect(isPidAlive(process.pid)).toBe(true); + expect(isPidAlive(await getDeadPid())).toBe(false); + }); + }); +}); diff --git a/packages/cli/tests/core/seed-script.spec.ts b/packages/cli/tests/core/seed-script.spec.ts new file mode 100644 index 000000000..ed846616e --- /dev/null +++ b/packages/cli/tests/core/seed-script.spec.ts @@ -0,0 +1,122 @@ +import type { spawn } from "node:child_process"; +import { EventEmitter } from "node:events"; +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { PassThrough } from "node:stream"; +import { pathToFileURL } from "node:url"; +import jwt from "jsonwebtoken"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { SERVICE_ROLE_EMAIL } from "@/core/local-state/index.js"; +import { runSeedScript } from "@/core/seed-script/index.js"; + +interface CapturedSpawn { + command: string; + args: string[]; + options: { env: Record; stdio: unknown }; +} + +function fakeSpawn(exitCode: number, captured: CapturedSpawn[]) { + return ((command: string, args: string[], options: unknown) => { + captured.push({ command, args, options } as CapturedSpawn); + const child = new EventEmitter() as EventEmitter & { + stdout: PassThrough; + stderr: PassThrough; + }; + child.stdout = new PassThrough(); + child.stderr = new PassThrough(); + setImmediate(() => child.emit("close", exitCode)); + return child; + }) as unknown as typeof spawn; +} + +describe("runSeedScript", () => { + let tempDir: string; + let scriptPath: string; + let wrapperPath: string; + let captured: CapturedSpawn[]; + + beforeEach(async () => { + tempDir = await mkdtemp(join(tmpdir(), "b44-seed-script-")); + scriptPath = join(tempDir, "seed.ts"); + wrapperPath = join(tempDir, "wrapper.ts"); + await writeFile(scriptPath, "export default async () => {};"); + await writeFile(wrapperPath, "// wrapper stub"); + captured = []; + }); + + afterEach(async () => { + await rm(tempDir, { recursive: true, force: true }); + }); + + const run = (exitCode = 0) => + runSeedScript({ + appId: "app-123", + scriptPath, + localUrl: "http://localhost:4400", + spawnImpl: fakeSpawn(exitCode, captured), + wrapperPath, + }); + + it("spawns deno run on a copy of the wrapper", async () => { + // When + const result = await run(); + + // Then + expect(result.exitCode).toBe(0); + expect(captured).toHaveLength(1); + expect(captured[0].command).toBe("deno"); + const args = captured[0].args; + expect(args.slice(0, 3)).toEqual([ + "run", + "--allow-all", + "--node-modules-dir=auto", + ]); + // Wrapper runs from a temp copy, not the shipped asset path + expect(args[3]).toMatch(/\.ts$/); + expect(args[3]).not.toBe(wrapperPath); + }); + + it("wires the local dev-server env with a service-subject token", async () => { + // When + await run(); + + // Then + const env = captured[0].options.env; + expect(env.BASE44_APP_ID).toBe("app-123"); + expect(env.BASE44_LOCAL_URL).toBe("http://localhost:4400"); + expect(env.SCRIPT_PATH).toBe(pathToFileURL(scriptPath).href); + + const decoded = jwt.decode(env.BASE44_LOCAL_SERVICE_TOKEN as string); + expect(decoded).toMatchObject({ sub: SERVICE_ROLE_EMAIL }); + }); + + it("records the reason when remote credentials are unavailable", async () => { + // Given: no app context in this process, so the remote token fetch fails + + // When + await run(); + + // Then: remote creds empty, reason recorded for ctx.remote() to throw + const env = captured[0].options.env; + expect(env.BASE44_ACCESS_TOKEN).toBe(""); + expect(env.BASE44_APP_BASE_URL).toBe(""); + expect(env.BASE44_REMOTE_ERROR).not.toBe(""); + }); + + it("pipes child output instead of inheriting stdio", async () => { + // When + await run(); + + // Then: stdout must stay clean for --json; both streams are piped + expect(captured[0].options.stdio).toEqual(["ignore", "pipe", "pipe"]); + }); + + it("propagates a non-zero exit code", async () => { + // When + const result = await run(3); + + // Then + expect(result.exitCode).toBe(3); + }); +}); diff --git a/packages/cli/tests/core/seed.spec.ts b/packages/cli/tests/core/seed.spec.ts new file mode 100644 index 000000000..70b5970e3 --- /dev/null +++ b/packages/cli/tests/core/seed.spec.ts @@ -0,0 +1,174 @@ +import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { SchemaValidationError } from "@/core/errors.js"; +import { computeSeedHash, readSeedFiles } from "@/core/resources/seed/index.js"; + +const WITH_SEED_CONFIG = resolve( + __dirname, + "../fixtures/with-seed/base44/config.jsonc", +); + +describe("seed config", () => { + describe("readSeedFiles", () => { + let configDir: string; + let configPath: string; + + beforeEach(async () => { + configDir = await mkdtemp(join(tmpdir(), "b44-seed-")); + configPath = join(configDir, "config.jsonc"); + await writeFile(configPath, JSON.stringify({ name: "Seed Test" })); + }); + + afterEach(async () => { + await rm(configDir, { recursive: true, force: true }); + }); + + const givenSeedFile = async (name: string, content: unknown) => { + await mkdir(join(configDir, "seed"), { recursive: true }); + await writeFile( + join(configDir, "seed", name), + JSON.stringify(content, null, 2), + ); + }; + + it("reads the with-seed fixture project", async () => { + // When + const seedData = await readSeedFiles({ + configPath: WITH_SEED_CONFIG, + seedDir: "seed", + }); + + // Then + expect(seedData).not.toBeNull(); + expect(seedData?.users?.relPath).toBe("seed/users.jsonc"); + expect(seedData?.users?.users).toHaveLength(2); + expect(seedData?.fixtures.map((f) => f.baseName)).toEqual([ + "task", + "team-member", + ]); + expect(seedData?.scriptPath).toBeNull(); + expect(seedData?.hash).toMatch(/^sha256:[0-9a-f]{64}$/); + }); + + it("applies defaults to seed users", async () => { + // When + const seedData = await readSeedFiles({ + configPath: WITH_SEED_CONFIG, + seedDir: "seed", + }); + + // Then — member@seed.dev has no explicit role + const member = seedData?.users?.users.find( + (u) => u.email === "member@seed.dev", + ); + expect(member?.role).toBe("user"); + }); + + it("returns null when the project has no seed files", async () => { + // When + const seedData = await readSeedFiles({ configPath, seedDir: "seed" }); + + // Then + expect(seedData).toBeNull(); + }); + + it("treats Users.jsonc as the users fixture regardless of case", async () => { + // Given + await givenSeedFile("Users.jsonc", [{ email: "case@example.com" }]); + + // When + const seedData = await readSeedFiles({ configPath, seedDir: "seed" }); + + // Then + expect(seedData?.users?.users[0]?.email).toBe("case@example.com"); + expect(seedData?.fixtures).toEqual([]); + }); + + it("throws SchemaValidationError citing the file for a malformed users fixture", async () => { + // Given — missing required email + await givenSeedFile("users.jsonc", [{ role: "admin" }]); + + // When / Then + await expect( + readSeedFiles({ configPath, seedDir: "seed" }), + ).rejects.toThrow(SchemaValidationError); + await expect( + readSeedFiles({ configPath, seedDir: "seed" }), + ).rejects.toThrow(/users\.jsonc/); + }); + + it("throws SchemaValidationError when a fixture is not an array", async () => { + // Given + await givenSeedFile("task.jsonc", { title: "not an array" }); + + // When / Then + await expect( + readSeedFiles({ configPath, seedDir: "seed" }), + ).rejects.toThrow(/task\.jsonc/); + }); + + it("resolves seedDir relative to the config dir", async () => { + // Given + await mkdir(join(configDir, "custom-seed"), { recursive: true }); + await writeFile( + join(configDir, "custom-seed", "users.jsonc"), + JSON.stringify([{ email: "custom@example.com" }]), + ); + + // When + const seedData = await readSeedFiles({ + configPath, + seedDir: "custom-seed", + }); + + // Then + expect(seedData?.users?.relPath).toBe("custom-seed/users.jsonc"); + }); + + it("includes seed.ts in the hash when present", async () => { + // Given + await givenSeedFile("users.jsonc", [{ email: "hash@example.com" }]); + const before = await readSeedFiles({ configPath, seedDir: "seed" }); + await writeFile(join(configDir, "seed.ts"), "export default () => {};"); + + // When + const after = await readSeedFiles({ configPath, seedDir: "seed" }); + + // Then + expect(after?.scriptPath).toBe(join(configDir, "seed.ts")); + expect(after?.hash).not.toBe(before?.hash); + }); + }); + + describe("computeSeedHash", () => { + const entry = (relPath: string, content: string) => ({ + relPath, + bytes: new TextEncoder().encode(content), + }); + + it("is independent of entry order", () => { + // Given + const a = entry("seed/a.jsonc", "[]"); + const b = entry("seed/b.jsonc", "[1]"); + + // Then + expect(computeSeedHash([a, b])).toBe(computeSeedHash([b, a])); + }); + + it("changes when file bytes change", () => { + // Then + expect(computeSeedHash([entry("seed/a.jsonc", "[]")])).not.toBe( + computeSeedHash([entry("seed/a.jsonc", "[2]")]), + ); + }); + + it("changes when a file is renamed", () => { + // Then + expect(computeSeedHash([entry("seed/a.jsonc", "[]")])).not.toBe( + computeSeedHash([entry("seed/b.jsonc", "[]")]), + ); + }); + }); +}); diff --git a/packages/cli/tests/fixtures/with-seed/base44/.app.jsonc b/packages/cli/tests/fixtures/with-seed/base44/.app.jsonc new file mode 100644 index 000000000..d7852426c --- /dev/null +++ b/packages/cli/tests/fixtures/with-seed/base44/.app.jsonc @@ -0,0 +1,4 @@ +// Base44 App Configuration +{ + "id": "test-app-id" +} diff --git a/packages/cli/tests/fixtures/with-seed/base44/config.jsonc b/packages/cli/tests/fixtures/with-seed/base44/config.jsonc new file mode 100644 index 000000000..59d4cb8d5 --- /dev/null +++ b/packages/cli/tests/fixtures/with-seed/base44/config.jsonc @@ -0,0 +1,3 @@ +{ + "name": "Seed Test Project" +} diff --git a/packages/cli/tests/fixtures/with-seed/base44/entities/task.jsonc b/packages/cli/tests/fixtures/with-seed/base44/entities/task.jsonc new file mode 100644 index 000000000..82b825678 --- /dev/null +++ b/packages/cli/tests/fixtures/with-seed/base44/entities/task.jsonc @@ -0,0 +1,16 @@ +{ + "name": "Task", + "type": "object", + "properties": { + "title": { + "type": "string", + "description": "Task title" + }, + "completed": { + "type": "boolean", + "default": false, + "description": "Whether the task is completed" + } + }, + "required": ["title"] +} diff --git a/packages/cli/tests/fixtures/with-seed/base44/entities/team-member.jsonc b/packages/cli/tests/fixtures/with-seed/base44/entities/team-member.jsonc new file mode 100644 index 000000000..3e70e43bd --- /dev/null +++ b/packages/cli/tests/fixtures/with-seed/base44/entities/team-member.jsonc @@ -0,0 +1,11 @@ +{ + "name": "TeamMember", + "type": "object", + "properties": { + "nickname": { + "type": "string", + "description": "Member nickname" + } + }, + "required": ["nickname"] +} diff --git a/packages/cli/tests/fixtures/with-seed/base44/seed/task.jsonc b/packages/cli/tests/fixtures/with-seed/base44/seed/task.jsonc new file mode 100644 index 000000000..3ca522d11 --- /dev/null +++ b/packages/cli/tests/fixtures/with-seed/base44/seed/task.jsonc @@ -0,0 +1,17 @@ +// Task fixture: two stable-id records (upsertable) and one id-less record. +[ + { + "id": "task-1", + "title": "First seeded task", + "completed": true, + "created_by": "admin@seed.dev" + }, + { + "id": "task-2", + "title": "Second seeded task", + "created_by": "member@seed.dev" + }, + { + "title": "Task without a stable id" + } +] diff --git a/packages/cli/tests/fixtures/with-seed/base44/seed/team-member.jsonc b/packages/cli/tests/fixtures/with-seed/base44/seed/team-member.jsonc new file mode 100644 index 000000000..57cf99226 --- /dev/null +++ b/packages/cli/tests/fixtures/with-seed/base44/seed/team-member.jsonc @@ -0,0 +1,7 @@ +// Kebab-case file name resolves to the "TeamMember" entity. +[ + { + "id": "tm-1", + "nickname": "kb" + } +] diff --git a/packages/cli/tests/fixtures/with-seed/base44/seed/users.jsonc b/packages/cli/tests/fixtures/with-seed/base44/seed/users.jsonc new file mode 100644 index 000000000..eeb3acfcc --- /dev/null +++ b/packages/cli/tests/fixtures/with-seed/base44/seed/users.jsonc @@ -0,0 +1,13 @@ +// Seed users: upserted by email; password enables local /login. +[ + { + "email": "admin@seed.dev", + "full_name": "Seed Admin", + "role": "admin", + "password": "seedadmin1" + }, + { + "email": "member@seed.dev", + "password": "seedmember1" + } +]