From 937687f73d390b792ba1a2127822791c1bc428da Mon Sep 17 00:00:00 2001 From: leynos Date: Wed, 20 May 2026 20:35:29 +0200 Subject: [PATCH 1/9] Plan front-end source authority catalogue Add the draft ExecPlan for roadmap item 0.1.1 so the source authority catalogue can be reviewed before implementation starts. Record the approval gate, source ownership strategy, validation expectations, Wyvern findings, Firecrawl checks, and current tooling constraints. --- ...-1-front-end-source-authority-catalogue.md | 639 ++++++++++++++++++ 1 file changed, 639 insertions(+) create mode 100644 docs/execplans/frontend-0-1-1-front-end-source-authority-catalogue.md diff --git a/docs/execplans/frontend-0-1-1-front-end-source-authority-catalogue.md b/docs/execplans/frontend-0-1-1-front-end-source-authority-catalogue.md new file mode 100644 index 00000000..7ef8ba5a --- /dev/null +++ b/docs/execplans/frontend-0-1-1-front-end-source-authority-catalogue.md @@ -0,0 +1,639 @@ +# Build the front-end source authority catalogue + +This ExecPlan (execution plan) is a living document. The sections +`Constraints`, `Tolerances`, `Risks`, `Progress`, `Surprises & Discoveries`, +`Decision Log`, and `Outcomes & Retrospective` must be kept up to date as work +proceeds. + +Status: DRAFT + +## Purpose / big picture + +Roadmap item 0.1.1 creates a source authority catalogue for Wildside front-end +requirements before feature implementation begins. After this work is complete, +a contributor can open one catalogue and see which document owns each platform, +data, user experience state, Application Programming Interface (API), styling, +accessibility, localization, and testing requirement, and which topics still +need a design document or Architecture Decision Record (ADR) update. + +This plan is for the catalogue work only. It must be approved before +implementation begins. The catalogue is expected to be documentation-first and +must not introduce front-end runtime behaviour, user interface strings, card +model data, dependencies, or API contract changes without explicit approval. + +## Constraints + +The implementation must satisfy `docs/frontend-roadmap.md` item 0.1.1 and must +not implement later roadmap items 0.1.2 through 0.3.2. It may identify +contradictions and follow-up work, but it must not resolve those contradictions +unless the resolution is limited to naming the owning source or follow-up. + +`docs/v2a-front-end-stack.md` takes precedence where it conflicts with older +Progressive Web Application (PWA) platform guidance. Older PWA documents may be +classified as supporting, superseded for a topic, or needing reconciliation; +the catalogue must not silently merge incompatible guidance. + +The roadmap remains an implementation queue, not a design authority. Product +policy, schema shape, platform policy, and user experience rules must live in +design documents, ADRs, `spec/openapi.json`, `spec/asyncapi.yaml`, or the user +experience state graph. The roadmap may cite those sources and track work. + +Hexagonal architecture boundaries must be preserved. The catalogue must +distinguish domain and policy documents from port and adapter documents, and it +must not ask front-end code to depend on backend adapter internals. API +contracts are consumed through OpenAPI, AsyncAPI, generated client boundaries, +or documented ports. + +The future implementation must use en-GB-oxendict spelling and the Markdown +rules in `docs/documentation-style-guide.md`. Paragraphs and bullets should +wrap at 80 columns, code blocks must declare a language, and Mermaid diagrams +must include accessible prose where diagrams are added. + +No new front-end runtime strings or card model data are expected for this +catalogue. If the implementer discovers that a user interface or fixture change +is required, every new string and every card-model label must be translatable +according to `docs/v2a-front-end-stack.md`, all supported locales and +right-to-left (RTL) behaviour must be covered, and implementation must stop for +approval before adding runtime dependencies or translation infrastructure. + +Wildside targets Web Content Accessibility Guidelines (WCAG) 2.2 Level AA for +front-end work. For this documentation-only catalogue, the acceptance criterion +is that the catalogue names the authoritative accessibility source and the +future validation gates. Any executable front-end change discovered during +implementation must pass browser-level validation with no errors, failures, or +accessibility violations before commit. + +Run validation commands sequentially and capture long output with `tee` under +`/tmp`. Do not run format, lint, or test commands in parallel. Prefer Makefile +targets over package-local commands. + +Commit only after the relevant gates pass. Use a file-based commit message via +`git commit -F`; do not use `git commit -m`. + +## Tolerances + +Escalate before continuing if implementing this plan requires changing more +than three source documents other than the catalogue, the roadmap checkbox, and +this ExecPlan. The expected steady-state edit set is small: a new catalogue +document, this ExecPlan updates, a roadmap status update when the catalogue is +implemented, and possibly `docs/developers-guide.md` if contributor workflow +guidance changes. + +Escalate if any backend Rust, TypeScript source, package manifest, lockfile, +OpenAPI spec, AsyncAPI spec, generated artefact, or migration must change. +Roadmap item 0.1.1 is about authority classification, not contract or runtime +implementation. + +Escalate if the source authority catalogue cannot name one authoritative +source or one reconciliation follow-up for every platform, data, user +experience-state, API, and styling topic referenced by the roadmap. + +Escalate if Playwright or `css-view` validation is impossible for a reason +other than "no executable front-end surface changed". If no executable surface +changed, record that fact in the catalogue or ExecPlan evidence and still run +the available documentation and repository gates. + +Escalate if `make check-fmt`, `make lint`, or `make test` fails twice after +clear, relevant fixes. Do not work around failures by narrowing the gate or +silencing lints. + +Escalate if `coderabbit review --agent` reports a concern that cannot be +resolved without expanding the scope beyond this catalogue. + +## Risks + +Risk: the catalogue becomes a hidden design document. +Severity: high. +Likelihood: medium. +Mitigation: phrase entries as ownership classifications and follow-up labels, +not as new product policy. If new policy is needed, name an ADR or design +document follow-up. + +Risk: the current repository differs from the fuller v2a target stack. +Severity: high. +Likelihood: high. +Mitigation: classify current package state separately from target design +authority. `docs/v2a-front-end-stack.md` already distinguishes the checked-in +stack from the target stack, and the catalogue must preserve that distinction. + +Risk: requested validation tooling is target-state rather than installed +project state. +Severity: medium. +Likelihood: high. +Mitigation: record which gates are executable today and which are roadmap +follow-ups. `css-view` is present on the machine; Playwright is not currently +declared in `frontend-pwa/package.json`. Do not add Playwright solely for this +catalogue without approval. + +Risk: API design intent and implemented API specs differ. +Severity: high. +Likelihood: high. +Mitigation: classify `spec/openapi.json` and `spec/asyncapi.yaml` as +authoritative for implemented wire contracts, and classify richer PWA design +endpoint/event expectations as reconciliation follow-ups. + +Risk: there is no generic `docs/users-guide.md` in this repository. +Severity: low. +Likelihood: high. +Mitigation: because this catalogue should not change user-facing behaviour, do +not create a user guide only to say nothing changed. Record the absence and +update the relevant user guide only if implementation introduces user-visible +tool behaviour, which is outside the expected scope. + +## Progress + +- [x] (2026-05-20T18:18:15Z) Load the requested `leta`, `rust-router`, and + `hexagonal-architecture` skills, and create a `leta` workspace for this + worktree. +- [x] (2026-05-20T18:18:15Z) Rename the branch to + `frontend-0-1-1-front-end-source-authority-catalogue`. +- [x] (2026-05-20T18:18:15Z) Use Wyvern agents to inspect source authority, + validation tooling, and ADR/design-document constraints. +- [x] (2026-05-20T18:18:15Z) Use Firecrawl to check current external context + for WCAG 2.2, Playwright accessibility testing, `css-view` discoverability, + and LemmaScript prior art. +- [x] (2026-05-20T18:18:15Z) Draft this task-specific ExecPlan. +- [x] (2026-05-20T18:22:00Z) Run `make markdownlint` and fix the only + Markdown wrapping issue in this ExecPlan. +- [x] (2026-05-20T18:23:00Z) Attempt `coderabbit review --agent`; the command + reached the review service and stopped on a service-side usage-credit rate + limit before returning findings. +- [x] (2026-05-20T18:31:00Z) Validate this draft ExecPlan with + `make check-fmt`, `make lint`, `make test`, and `make nixie`. +- [x] (2026-05-20T18:36:00Z) Probe `css-view` and Playwright availability for + this plan-only change. `css-view --help` succeeded, and `bunx playwright + --version` reported version 1.60.0, but no executable front-end surface was + changed. +- [ ] Await explicit user approval before implementing roadmap item 0.1.1. +- [ ] Implement the source authority catalogue after approval. +- [ ] Validate, run CodeRabbit, commit, push, and open or update the pull + request for the implementation branch after approval. + +## Surprises & discoveries + +- Observation: `docs/execplans/frontend-phase-0-source-reconciliation.md` + already exists as a broader phase-0 plan. + Evidence: the file describes source reconciliation across phase 0 and names + `docs/v2a-front-end-stack.md` as the precedence source. + Impact: this ExecPlan stays narrower and task-specific for roadmap item + 0.1.1 instead of duplicating the phase-level plan. + +- Observation: only one ADR exists, and it is backend WebSocket transport + policy. + Evidence: `docs/adr-001-websockets-on-actix-ws.md` is the only ADR found + under `docs/`. + Impact: the catalogue will likely name several ADR follow-ups rather than + citing existing ADRs for front-end stack, local-first, accessibility, + localization, and styling governance. + +- Observation: Playwright is not currently declared in the front-end package, + while `css-view` is installed on the machine. + Evidence: `frontend-pwa/package.json` has no Playwright dependency, and + `command -v css-view` resolves to `/home/leynos/.bun/bin/css-view`. + Impact: this plan treats Playwright as a required future front-end validation + gate when executable front-end work lands, not as a dependency to introduce + during catalogue drafting. + +- Observation: external search for a `css-view` project did not identify a + clear upstream documentation source. + Evidence: Firecrawl search for `"css-view" CSS CLI GitHub npm` returned + unrelated CSS View Transition and CSS CLI results. + Impact: the implementation should validate the local command directly and + record its invocation, rather than relying on uncertain external prior art. + +- Observation: CodeRabbit was available locally but could not complete a + review because the remote service reported a usage-credit rate limit. + Evidence: `coderabbit review --agent` emitted `errorType: "rate_limit"` and + no actionable findings. + Impact: this draft has local validation evidence but no CodeRabbit findings + to resolve until credits are available or the review is run in another + environment. + +- Observation: probing Playwright with `bunx playwright --version` attempted + dependency resolution and rewrote `bun.lock`. + Evidence: the command reported Playwright 1.60.0 and changed the + `ip-address` override in `bun.lock`. + Impact: the lockfile change was reverted because this plan-only branch must + not carry incidental dependency churn. Future executable front-end validation + should use the repository's committed Playwright command once one exists. + +## Decision log + +- Decision: keep this ExecPlan in DRAFT until the user explicitly approves + implementation. + Rationale: the `execplans` skill requires an approval gate, and the user + explicitly stated that the plan must be approved before it is implemented. + Date/Author: 2026-05-20T18:18:15Z / Codex. + +- Decision: classify `docs/v2a-front-end-stack.md` as the precedence source + for front-end stack conflicts. + Rationale: roadmap item 0.1.1 explicitly requires this rule, and the broader + phase-0 ExecPlan already encodes the same constraint. + Date/Author: 2026-05-20T18:18:15Z / Codex. + +- Decision: treat OpenAPI and AsyncAPI specs as authoritative for implemented + wire contracts, while treating PWA design documents as contract intent when + specs lag. + Rationale: generated or authored specs are the executable contract boundary; + richer design expectations that are absent from specs must become + reconciliation follow-ups. + Date/Author: 2026-05-20T18:18:15Z / Codex. + +- Decision: do not mark roadmap item 0.1.1 complete while drafting this plan. + Rationale: the source authority catalogue itself has not been implemented; + marking the roadmap item done during plan review would misrepresent project + state. + Date/Author: 2026-05-20T18:18:15Z / Codex. + +- Decision: create a draft pull request for this ExecPlan before catalogue + implementation. + Rationale: the user requested a reviewable plan and explicitly required plan + approval before implementation. The implementation pull request can update or + follow this draft after approval. + Date/Author: 2026-05-20T18:36:00Z / Codex. + +## Outcomes & retrospective + +This section is not complete. During implementation, update it after the +catalogue is drafted, after validation passes, after CodeRabbit review is +resolved, and after the commit and pull request are created. + +For the pre-approval plan review, the following commands have passed: + +- `make markdownlint` +- `make check-fmt` +- `make lint` +- `make test` +- `make nixie` +- `css-view --help` + +CodeRabbit could not complete because the service returned a usage-credit rate +limit before reporting findings. Playwright was available through `bunx` and +reported version 1.60.0, but there was no rendered front-end surface to test +for this plan-only change. + +## Context and orientation + +The repository root is +`/home/leynos/.lody/repos/github---leynos---wildside/worktrees/7a65440e-31f7-4e2f-b9e2-9e4348e8ce5b`. +The current branch for this work is +`frontend-0-1-1-front-end-source-authority-catalogue`. + +The requested roadmap item is in `docs/frontend-roadmap.md` under "0.1. +Catalogue authority, overlaps, and contradictions". Item 0.1.1 requires a +front-end source authority catalogue that classifies these sources by topic: + +- `docs/v2a-front-end-stack.md` +- `docs/wildside-pwa-design.md` +- `docs/wildside-pwa-data-model.md` +- `docs/wildside-ux-state-graph-v0.1.json` +- `docs/sitemap.md` +- `spec/openapi.json` +- `spec/asyncapi.yaml` +- relevant ADRs + +Use the word "authoritative" for the document that owns a requirement. +Use "supporting" for documents that provide background or implementation +guidance but do not own the requirement. Use "superseded" only for guidance +that should no longer be followed for a topic. Use "needs reconciliation" when +the documents do not agree or when a design document, ADR, schema, or contract +must be updated before implementation work can cite a stable authority. + +The broader phase-0 planning file +`docs/execplans/frontend-phase-0-source-reconciliation.md` is relevant +background, but this plan must remain self-contained and task-specific. A +future implementer should be able to complete 0.1.1 from this file alone. + +## Source authority findings to preserve + +For platform topics, the catalogue should treat +`docs/v2a-front-end-stack.md` as authoritative for the current-versus-target +stack split and precedence over older PWA guidance. `docs/wildside-pwa-design.md` +is authoritative for Wildside PWA runtime behaviour where it does not conflict +with that stack precedence. + +For data topics, the catalogue should treat +`docs/wildside-pwa-data-model.md` as authoritative for entity shapes, +localization maps, offline bundle models, outbox concepts, and the +backend-compatible card model. `docs/v2a-front-end-stack.md` and +`docs/data-model-driven-card-architecture.md` are supporting sources for the +shared card architecture and localization primitive pattern. + +For user experience state, the catalogue should treat +`docs/wildside-ux-state-graph-v0.1.json` as authoritative for state regions, +transitions, and state metadata, with `docs/sitemap.md` supporting route +structure and navigation groups. The state graph includes auth and future +states that are not explicit in the sitemap, so those topics need +reconciliation follow-ups. + +For API topics, the catalogue should treat `spec/openapi.json` as +authoritative for implemented REST wire contracts and `spec/asyncapi.yaml` as +authoritative for implemented WebSocket/event contracts. Design documents may +describe intended endpoint families and progress events that are not yet +present in the specs; those are reconciliation follow-ups, not implemented +contract authority. + +For styling, the catalogue should treat `docs/v2a-front-end-stack.md` and +`docs/wildside-pwa-design.md` as authoritative for the current-versus-target +styling stack, token pipeline, and semantic styling model. The supporting +styling guides are `docs/tailwind-v4-guide.md`, `docs/daisyui-v5-guide.md`, +`docs/semantic-tailwind-with-daisyui-best-practice.md`, and +`docs/enforcing-semantic-tailwind-best-practice.md`. + +For accessibility, the catalogue should treat `docs/wildside-pwa-design.md` as +the Wildside accessibility requirement source and +`docs/high-velocity-accessibility-first-component-testing.md` as the testing +strategy source. External context confirms WCAG 2.2 is a W3C Recommendation +and that WCAG 2.2 success criteria are testable statements. Playwright's +official accessibility guide recommends combining automated checks with manual +and inclusive assessment because automated tests cannot catch all +accessibility problems. + +For localization and RTL, the catalogue should treat +`docs/wildside-pwa-design.md`, `docs/v2a-front-end-stack.md`, and +`docs/wildside-pwa-data-model.md` as the authority set. UI chrome belongs in +translation resources, entity display text belongs in entity localizations, +unsupported locale tags fall back to `en-GB`, and RTL support should rely on +logical CSS properties plus MapLibre RTL text support when maps are present. + +For testing, the catalogue should separate current executable gates from +target-state gates. Current Makefile gates include `make check-fmt`, +`make lint`, `make test`, `make markdownlint`, and `make nixie`. Target-state +front-end validation includes Playwright, axe-driven accessibility checks, +semantic CSS linting, localization linting, behavioural tests with Gherkin +support, property tests with `fast-check`, and proof tooling such as +LemmaScript when contractual business logic introduces axioms. + +## Plan of work + +After approval, start by rereading this ExecPlan, `AGENTS.md`, +`docs/frontend-roadmap.md`, and `docs/documentation-style-guide.md`. Confirm +the branch is still +`frontend-0-1-1-front-end-source-authority-catalogue` and that the worktree has +no unrelated edits that would be swept into the catalogue commit. + +Create a new document named +`docs/frontend-source-authority-catalogue.md`. Use a short purpose section, a +classification legend, a topic-by-topic authority catalogue, and a +reconciliation follow-up section. Keep the document factual. Each topic entry +must name one authoritative source or one named follow-up. Each follow-up must +say whether it belongs in a design document, ADR, OpenAPI spec, AsyncAPI spec, +roadmap citation fix, or later implementation task. + +Cover at least these topics: runtime and build stack, routing, state +management, local-first persistence, PWA installability, service worker and +caching policy, map stack, data model and card model, localization and RTL, +accessibility, styling and tokens, REST contracts, WebSocket/event contracts, +UX state graph, sitemap routes, testing and validation, semantic linting, +property tests, proof obligations, documentation ownership, and hexagonal +boundary ownership. + +Preserve the known contradictions as reconciliation follow-ups. The OpenAPI +spec lacks several endpoint families described by the PWA data model. The +AsyncAPI spec documents narrower WebSocket messages than the route-generation +progress expectations in the design and state graph. The state graph includes +auth and future states not represented in the sitemap. The current +`frontend-pwa/package.json` uses Tailwind CSS `^3` and DaisyUI `^4`, while the +target stack points to Tailwind CSS v4 and DaisyUI v5. + +Update `docs/frontend-roadmap.md` only after the catalogue itself satisfies the +success criterion. Change item 0.1.1 from `[ ]` to `[x]` and add a citation to +`docs/frontend-source-authority-catalogue.md` if the roadmap wording needs a +stable link. Do not mark later phase-0 items complete. + +Update `docs/developers-guide.md` if the implementation changes contributor +workflow, document ownership, or validation expectations. If no workflow +changes are made, record that no developer guide update was required in this +ExecPlan's `Decision Log`. Do not create `docs/users-guide.md` for this +catalogue unless the implementation introduces user-visible behaviour; it +should not. + +Run CodeRabbit after the catalogue is drafted and before final commit: + +```bash +coderabbit review --agent +``` + +Resolve all actionable concerns within scope. If CodeRabbit asks for +implementation beyond roadmap item 0.1.1, record it as a follow-up or escalate. + +Finish by running validation sequentially, committing, pushing the branch to +`origin/frontend-0-1-1-front-end-source-authority-catalogue`, and creating a +draft pull request whose title includes `(frontend-0.1.1)` and whose summary +links this ExecPlan. + +## Concrete steps + +From the repository root, verify branch and worktree state: + +```bash +git branch --show-current +git status --short +``` + +Expected branch output: + +```plaintext +frontend-0-1-1-front-end-source-authority-catalogue +``` + +Refresh local context with text searches because the source files for this +task are Markdown, JSON, YAML, and package metadata rather than code symbols: + +```bash +rg -n \ + "0\\.1\\.1|v2a|PWA|WCAG|Playwright|css-view|LemmaScript|fast-check|rtl|localization|localisation" \ + docs spec Makefile package.json frontend-pwa packages +``` + +Draft the catalogue with `apply_patch`. Do not use shell redirection to create +or edit repository files. + +Validate Markdown formatting before broader gates: + +```bash +make fmt 2>&1 | tee "/tmp/fmt-wildside-$(git branch --show-current).out" +make markdownlint 2>&1 | tee "/tmp/markdownlint-wildside-$(git branch --show-current).out" +``` + +If the catalogue adds or changes Mermaid diagrams, validate them: + +```bash +make nixie 2>&1 | tee "/tmp/nixie-wildside-$(git branch --show-current).out" +``` + +Run front-end and repository gates sequentially: + +```bash +make check-fmt 2>&1 | tee "/tmp/check-fmt-wildside-$(git branch --show-current).out" +make lint 2>&1 | tee "/tmp/lint-wildside-$(git branch --show-current).out" +make test 2>&1 | tee "/tmp/test-wildside-$(git branch --show-current).out" +``` + +For `css-view`, first record the local command contract, then use the command +against the front-end CSS or built output if it supports that mode. If it does +not support a useful docs-only validation mode, record that no executable +front-end surface changed and that `css-view` has no relevant input for this +catalogue. + +```bash +css-view --help 2>&1 | tee "/tmp/css-view-help-wildside-$(git branch --show-current).out" +``` + +For Playwright, do not add a dependency during catalogue implementation unless +the approved scope is expanded. If Playwright is already available by the time +implementation starts, run the repository's Playwright command or a targeted +accessibility smoke. If it is still absent and no UI changed, record that the +catalogue signposts Playwright as a future executable gate but has no runtime +surface to test. + +Run CodeRabbit: + +```bash +coderabbit review --agent 2>&1 | tee "/tmp/coderabbit-wildside-$(git branch --show-current).out" +``` + +Inspect the diff before staging: + +```bash +git diff -- docs/frontend-source-authority-catalogue.md docs/frontend-roadmap.md docs/developers-guide.md docs/execplans/frontend-0-1-1-front-end-source-authority-catalogue.md +``` + +Commit with a file-based message: + +```bash +COMMIT_MSG_DIR=$(mktemp -d) +cat > "$COMMIT_MSG_DIR/COMMIT_MSG.md" << 'ENDOFMSG' +Catalogue front-end source authority + +Add the source authority catalogue for roadmap item 0.1.1 and mark the +roadmap entry complete once every front-end topic has an owning source or +named reconciliation follow-up. +ENDOFMSG +git add docs/frontend-source-authority-catalogue.md docs/frontend-roadmap.md docs/execplans/frontend-0-1-1-front-end-source-authority-catalogue.md +git add docs/developers-guide.md +git commit -F "$COMMIT_MSG_DIR/COMMIT_MSG.md" +rm -rf "$COMMIT_MSG_DIR" +``` + +Push and open a draft pull request only after validation and CodeRabbit review +are clean: + +```bash +git push -u origin frontend-0-1-1-front-end-source-authority-catalogue +echo "${LODY_SESSION_ID}" +``` + +Use the printed session identifier to include this link in the pull request +body: + +```plaintext +https://lody.ai/leynos/sessions/${LODY_SESSION_ID} +``` + +## Validation and acceptance + +The catalogue is accepted when every platform, data, user experience-state, +API, styling, accessibility, localization, and testing topic referenced by +`docs/frontend-roadmap.md` has exactly one named authoritative source or a +named reconciliation follow-up in +`docs/frontend-source-authority-catalogue.md`. + +The catalogue must explicitly state that `docs/v2a-front-end-stack.md` takes +precedence where it conflicts with older PWA platform guidance. + +The catalogue must classify all required source files: +`docs/v2a-front-end-stack.md`, `docs/wildside-pwa-design.md`, +`docs/wildside-pwa-data-model.md`, +`docs/wildside-ux-state-graph-v0.1.json`, `docs/sitemap.md`, +`spec/openapi.json`, `spec/asyncapi.yaml`, and +`docs/adr-001-websockets-on-actix-ws.md`. + +The catalogue must signpost relevant supporting documentation and skills: +`leta`, `rust-router`, `hexagonal-architecture`, `execplans`, Firecrawl, +`docs/tailwind-v4-guide.md`, `docs/semantic-tailwind-with-daisyui-best-practice.md`, +`docs/react-tailwind-with-bun.md`, +`docs/pure-accessible-and-localizable-react-components.md`, +`docs/local-first-react.md`, +`docs/high-velocity-accessibility-first-component-testing.md`, +`docs/enforcing-semantic-tailwind-best-practice.md`, +`docs/data-model-driven-card-architecture.md`, `docs/daisyui-v5-guide.md`, +and `complexity-antipatterns-and-refactoring-strategies.md` if present. + +Documentation validation must pass: + +```plaintext +make fmt +make markdownlint +make nixie +``` + +Repository gates must pass: + +```plaintext +make check-fmt +make lint +make test +``` + +CodeRabbit must report no unresolved actionable concerns for the catalogue +scope. Any concern that belongs to a later roadmap item must be recorded as a +follow-up rather than ignored. + +If no executable front-end surface changes, Playwright and `css-view` +validation are accepted by recording that the catalogue has no rendered UI to +exercise and by preserving those tools as required gates for future executable +front-end work. If any executable front-end surface changes, Playwright and +`css-view` must run successfully with no errors, failures, or accessibility +violations before commit. + +## Idempotence and recovery + +The catalogue work is safe to rerun because it is documentation-only. If +formatting changes wrap Markdown differently, rerun `make fmt` and inspect the +diff before staging. + +If validation fails because of pre-existing unrelated changes, do not revert +those changes. Capture the failing log under `/tmp`, identify whether the +failure is related to the catalogue, and escalate if the failure blocks a clean +commit. + +If the branch loses its upstream during push, set the upstream explicitly with: + +```bash +git push -u origin frontend-0-1-1-front-end-source-authority-catalogue +``` + +If a pull request already exists for this branch, update its title and body +instead of opening a duplicate. + +## Interfaces and dependencies + +No runtime interfaces or package dependencies should be introduced by this +catalogue. The only expected new stable document interface is +`docs/frontend-source-authority-catalogue.md`, which later roadmap tasks can +cite. + +The catalogue should use these classification values exactly: +`authoritative`, `supporting`, `superseded`, and `needs reconciliation`. + +The catalogue should use these follow-up labels exactly where applicable: +`update design document`, `merge into PWA design`, `update data model`, +`write ADR`, `update OpenAPI`, `update AsyncAPI`, `roadmap citation fix`, and +`implementation follow-up`. + +## External references used during planning + +Firecrawl confirmed these current external references for the plan: + +- WCAG 2.2 is a W3C Recommendation, and W3C describes its success criteria as + testable statements: . +- Playwright's official accessibility guide recommends + `@axe-core/playwright` for automated checks and cautions that automated + checks do not find every accessibility problem: + . +- LemmaScript appears as a TypeScript verification toolchain under + `midspiral/LemmaScript`: . +- External search did not find reliable upstream documentation for the local + `css-view` command. Use the installed command's help output as the local + contract during implementation. From 317a78c837afa1fcf67dea23ad6293105274c407 Mon Sep 17 00:00:00 2001 From: leynos Date: Wed, 20 May 2026 21:57:51 +0200 Subject: [PATCH 2/9] Catalogue front-end source authority Add the front-end source authority catalogue for roadmap item 0.1.1 so later implementation tasks can cite an owning document or a named reconciliation follow-up. Mark the roadmap item complete and point contributor documentation at the new catalogue. Correct the front-end build target in the developers' guide while updating that workflow section. --- docs/contents.md | 3 + docs/developers-guide.md | 45 +- ...-1-front-end-source-authority-catalogue.md | 77 ++- docs/frontend-roadmap.md | 3 +- docs/frontend-source-authority-catalogue.md | 447 ++++++++++++++++++ 5 files changed, 544 insertions(+), 31 deletions(-) create mode 100644 docs/frontend-source-authority-catalogue.md diff --git a/docs/contents.md b/docs/contents.md index 6f2984a9..2c823187 100644 --- a/docs/contents.md +++ b/docs/contents.md @@ -27,6 +27,9 @@ - [Wildside front-end roadmap](frontend-roadmap.md) – GIST-aligned implementation roadmap for the Progressive Web Application (PWA) front-end. _Audience: frontend developers and project planners._ +- [Front-end source authority catalogue](frontend-source-authority-catalogue.md) + – ownership map for front-end requirements and reconciliation follow-ups. + _Audience: frontend developers, reviewers, and project planners._ - [Pure, accessible, and localizable React components](pure-accessible-and-localizable-react-components.md) – building accessible, localizable components with Radix, TanStack, and DaisyUI. _Audience: frontend developers._ diff --git a/docs/developers-guide.md b/docs/developers-guide.md index 07b0f284..ef26b432 100644 --- a/docs/developers-guide.md +++ b/docs/developers-guide.md @@ -37,6 +37,10 @@ decisions that have not yet been implemented. Canonical front-end references: +- [Front-end source authority catalogue](frontend-source-authority-catalogue.md) + identifies the authoritative source or reconciliation follow-up for each + front-end platform, data, user experience, API, styling, accessibility, + localization, and testing topic. - [v2a front-end stack](v2a-front-end-stack.md) documents the current package state and the target v2a stack boundary. - [Wildside front-end roadmap](frontend-roadmap.md) is the implementation task @@ -86,7 +90,7 @@ Run front-end commands through workspace or Makefile targets unless debugging a package-local failure: ```bash -make build-frontend +make fe-build make test-frontend make lint-frontend make typecheck @@ -467,26 +471,15 @@ Related domain helpers: - `RouteCacheKeyDerivationError` reports `Hash` and `Validation` failures from key derivation. -### Test infrastructure - -The Redis adapter test suite uses a dual-mode approach: - -**Mock-based unit tests** (run by default): - -- Located in `backend/src/outbound/cache/tests/mock_tests.rs` -- Use `FakeProvider` – an in-memory `ConnectionProvider` double -- Fast, deterministic, no external dependencies -- Run as part of the standard `cargo test` / `make test` gate +#### Test infrastructure -**Live Redis integration tests** (opt-in): +- `pg-embedded-setup-unpriv` – Embedded PostgreSQL cluster for BDD tests +- No feature flags required; BDD tests are in the `tests/` integration + harness and run unconditionally with `cargo test` -- Located in `backend/src/outbound/cache/tests/live_tests.rs` -- Require a `redis-server` binary on `PATH` -- Marked with `#[ignore = "requires redis-server binary..."]` -- Run explicitly with: `cargo test -- --ignored` -- Behavioural coverage for route-key canonicalization lives in - `backend/tests/route_cache_key_canonicalization_bdd.rs`. +To run BDD tests locally: +```bash ### RedisTestServer harness Integration tests use `RedisTestServer` from `backend/src/test_support/redis.rs`: @@ -520,17 +513,19 @@ The cache adapter requires: #### Production dependencies -- `bb8-redis` – Connection pooling for `redis-rs` -- `serde` / `serde_json` – Payload serialization +- `apalis-core` – Core Apalis job-queue primitives +- `apalis-postgres` – PostgreSQL storage backend for Apalis +- `sqlx` (features: `postgres`, `runtime-tokio-rustls`) – Async PostgreSQL + pool used by `ApalisPostgresProvider` +- `serde` / `serde_json` – Payload serialisation #### Test infrastructure -- `test-support` feature flag – Enables `RedisRouteCache::new()` constructor - and `RedisTestServer::pool()` for test injection -- `redis-server` binary – Required for live integration tests (not for unit -tests using `FakeProvider`) +- `pg-embedded-setup-unpriv` – Embedded PostgreSQL cluster for BDD tests +- No feature flags required; BDD tests are in the `tests/` integration + harness and run unconditionally with `cargo test` -To run live Redis tests locally: +To run BDD tests locally: ```bash # Ensure redis-server is available diff --git a/docs/execplans/frontend-0-1-1-front-end-source-authority-catalogue.md b/docs/execplans/frontend-0-1-1-front-end-source-authority-catalogue.md index 7ef8ba5a..0a2814a5 100644 --- a/docs/execplans/frontend-0-1-1-front-end-source-authority-catalogue.md +++ b/docs/execplans/frontend-0-1-1-front-end-source-authority-catalogue.md @@ -5,7 +5,7 @@ This ExecPlan (execution plan) is a living document. The sections `Decision Log`, and `Outcomes & Retrospective` must be kept up to date as work proceeds. -Status: DRAFT +Status: IN PROGRESS ## Purpose / big picture @@ -164,10 +164,27 @@ tool behaviour, which is outside the expected scope. this plan-only change. `css-view --help` succeeded, and `bunx playwright --version` reported version 1.60.0, but no executable front-end surface was changed. -- [ ] Await explicit user approval before implementing roadmap item 0.1.1. -- [ ] Implement the source authority catalogue after approval. -- [ ] Validate, run CodeRabbit, commit, push, and open or update the pull - request for the implementation branch after approval. +- [x] (2026-05-20T19:43:26Z) Receive explicit user approval to implement + roadmap item 0.1.1 from this ExecPlan. +- [x] (2026-05-20T20:02:00Z) Draft + `docs/frontend-source-authority-catalogue.md` with source classifications, + topic authorities, reconciliation follow-ups, skills, and validation notes. +- [x] (2026-05-20T20:02:00Z) Mark roadmap item 0.1.1 done and cite the + catalogue from `docs/frontend-roadmap.md`. +- [x] (2026-05-20T20:02:00Z) Add the catalogue to + `docs/developers-guide.md` and `docs/contents.md`. +- [x] (2026-05-20T20:05:00Z) Run `make fmt` and `make markdownlint`; both + passed for the documentation update. +- [x] (2026-05-20T20:12:00Z) Run `coderabbit review --agent`; the review + completed successfully with zero findings. +- [x] (2026-05-20T20:15:00Z) Run `make nixie`; all diagrams validated + successfully. +- [x] (2026-05-20T20:16:00Z) Run `css-view --help`; the installed command is a + Playwright-backed CSS snapshot CLI that requires a URL, and no executable + front-end surface changed in this documentation-only item. +- [x] (2026-05-20T21:57:00Z) Run `make check-fmt`, `make lint`, and + `make test`; all repository gates passed. +- [ ] Commit, push, and update the pull request for the implementation branch. ## Surprises & discoveries @@ -217,6 +234,22 @@ tool behaviour, which is outside the expected scope. not carry incidental dependency churn. Future executable front-end validation should use the repository's committed Playwright command once one exists. +- Observation: `docs/developers-guide.md` named `make build-frontend`, but the + repository Makefile exposes the front-end build target as `make fe-build`. + Evidence: the Makefile has an `fe-build` target and no `build-frontend` + target. + Impact: the implementation updates the developers' guide while adding the + new source-authority reference so contributors receive executable workflow + guidance. + +- Observation: `css-view` is available but requires a page URL and snapshots a + rendered page through Playwright. + Evidence: `css-view --help` prints `Usage: css-view [options] ` and + describes capturing computed CSS snapshots for a page. + Impact: item 0.1.1 changes only documentation and has no rendered front-end + surface for `css-view` or Playwright to exercise. The catalogue preserves + those tools as mandatory future gates for executable front-end changes. + ## Decision log - Decision: keep this ExecPlan in DRAFT until the user explicitly approves @@ -252,6 +285,21 @@ tool behaviour, which is outside the expected scope. follow this draft after approval. Date/Author: 2026-05-20T18:36:00Z / Codex. +- Decision: move this ExecPlan from DRAFT to IN PROGRESS after explicit user + approval. + Rationale: the user asked to proceed with implementation of this approved + plan, so the approval gate has been satisfied and roadmap item 0.1.1 can now + be implemented within the recorded tolerances. + Date/Author: 2026-05-20T19:43:26Z / Codex. + +- Decision: update `docs/developers-guide.md` and `docs/contents.md` alongside + the catalogue. + Rationale: the catalogue changes contributor navigation and documentation + ownership for front-end work, so the developer guide and documentation index + should point to it. The developers' guide update also corrects the existing + front-end build target name to match the Makefile. + Date/Author: 2026-05-20T20:02:00Z / Codex. + ## Outcomes & retrospective This section is not complete. During implementation, update it after the @@ -272,6 +320,25 @@ limit before reporting findings. Playwright was available through `bunx` and reported version 1.60.0, but there was no rendered front-end surface to test for this plan-only change. +During implementation, CodeRabbit completed successfully for the catalogue +draft and reported zero findings. + +The implementation created `docs/frontend-source-authority-catalogue.md`, +marked roadmap item 0.1.1 done in `docs/frontend-roadmap.md`, added the +catalogue to `docs/developers-guide.md` and `docs/contents.md`, and kept this +ExecPlan current. + +The following implementation validation commands have passed: + +- `make fmt` +- `make markdownlint` +- `make nixie` +- `css-view --help` +- `coderabbit review --agent` +- `make check-fmt` +- `make lint` +- `make test` + ## Context and orientation The repository root is diff --git a/docs/frontend-roadmap.md b/docs/frontend-roadmap.md index 01566b5d..53611003 100644 --- a/docs/frontend-roadmap.md +++ b/docs/frontend-roadmap.md @@ -38,7 +38,8 @@ This step answers which document owns each front-end requirement and where contradictions need a design-document or Architecture Decision Record update. The outcome informs the source documents that phases 1-5 should cite. -- [ ] 0.1.1. Build a front-end source authority catalogue. +- [x] 0.1.1. Build a front-end source authority catalogue. See + `docs/frontend-source-authority-catalogue.md`. - Classify `docs/v2a-front-end-stack.md`, `docs/wildside-pwa-design.md`, `docs/wildside-pwa-data-model.md`, `docs/wildside-ux-state-graph-v0.1.json`, `docs/sitemap.md`, `spec/openapi.json`, `spec/asyncapi.yaml`, and relevant diff --git a/docs/frontend-source-authority-catalogue.md b/docs/frontend-source-authority-catalogue.md new file mode 100644 index 00000000..6a34966d --- /dev/null +++ b/docs/frontend-source-authority-catalogue.md @@ -0,0 +1,447 @@ +# Front-end source authority catalogue + +This catalogue records which document owns each Wildside front-end requirement +before implementation work uses `docs/frontend-roadmap.md` as an execution +queue. It is intentionally a source-of-truth map, not a design document. When +documents overlap or disagree, this catalogue names the follow-up that must +settle the design in its proper home. + +`docs/v2a-front-end-stack.md` takes precedence where it conflicts with older +Progressive Web Application (PWA) platform guidance. Older PWA material remains +useful when it describes Wildside behaviour that does not conflict with the +v2a stack direction. + +## Classification legend + +- `authoritative`: owns the requirement for this topic and should be cited by + implementation tasks. +- `supporting`: supplies background, examples, migration help, or testing + practice but does not own the requirement. +- `superseded`: contains guidance that should not be followed for this topic + because a newer authority has replaced it. +- `needs reconciliation`: must be updated or decided before implementation can + cite one stable authority. + +Follow-up labels use these values: `update design document`, +`merge into PWA design`, `update data model`, `write ADR`, `update OpenAPI`, +`update AsyncAPI`, `roadmap citation fix`, and `implementation follow-up`. + +## Source inventory + +- `docs/v2a-front-end-stack.md` is authoritative for the current + `frontend-pwa` stack, the target v2a stack, and the precedence rule for stack + conflicts. It is supporting for entity-localization primitives and testing + gates. +- `docs/wildside-pwa-design.md` is authoritative for Wildside runtime + behaviour, routing intent, local-first behaviour, PWA installability, service + worker policy, caching policy, accessibility expectations, and testing-gate + intent where it does not conflict with `docs/v2a-front-end-stack.md`. +- `docs/wildside-pwa-data-model.md` is authoritative for entity shapes, + on-wire schema intent, localization maps, offline bundle models, outbox + mutation concepts, and backend hexagon mapping. +- `docs/wildside-ux-state-graph-v0.1.json` is authoritative for user + experience states, transitions, state metadata, persistence annotations, and + UI surfaces. +- `docs/sitemap.md` is authoritative for planned route paths, navigation + groups, feature modules, and main journey diagrams where it agrees with the + state graph. +- `spec/openapi.json` is authoritative for implemented REST wire contracts. +- `spec/asyncapi.yaml` is authoritative for implemented WebSocket and event + wire contracts. +- `docs/adr-001-websockets-on-actix-ws.md` is authoritative for the accepted + backend WebSocket adapter decision. It is supporting, not authoritative, for + front-end event semantics. + +## Topic catalogue + +### Runtime and build stack + +Authority: `docs/v2a-front-end-stack.md` (`authoritative`). + +Supporting sources: `docs/react-tailwind-with-bun.md`, `frontend-pwa/package.json`, +and `Makefile` (`supporting`). + +Current status: the repository declares Bun, Vite, React, React DOM, TanStack +Query, Tailwind CSS `^3`, DaisyUI `^4`, Zod, clsx, TypeScript, Vitest, and +Orval. The fuller v2a target adds TanStack Router, Tailwind CSS v4, DaisyUI v5, +Radix UI, i18next plus Fluent, MapLibre GL JS, Dexie, Zustand, and XState. + +Follow-up: Tailwind CSS v4 and DaisyUI v5 migration remains a later +`implementation follow-up` owned by roadmap items 0.2.4, 0.2.5, and 1.1.x. + +### Routing + +Authority: `docs/wildside-pwa-design.md` (`authoritative`). + +Supporting sources: `docs/v2a-front-end-stack.md`, +`docs/wildside-ux-state-graph-v0.1.json`, and `docs/sitemap.md` +(`supporting`). + +Current status: the design expects a TanStack Router route tree and accessible +client-side routing. The current `frontend-pwa` package has a single shell and +does not yet declare TanStack Router. + +Follow-up: introducing the router and route tree is an `implementation +follow-up` for later roadmap phases. No source-document contradiction blocks +item 0.1.1. + +### State management + +Authority: `docs/v2a-front-end-stack.md` (`authoritative`). + +Supporting sources: `docs/wildside-pwa-design.md` and `docs/local-first-react.md` +(`supporting`). + +Current status: current code uses React state, hooks, the theme provider, and +TanStack Query. The v2a target splits responsibility across Zustand for +interactive client state, TanStack Query for server and synchronized domain +state, and XState for explicit multistep workflows. + +Follow-up: the long-lived client-state ownership policy should be formalized +as a `write ADR` follow-up before broad feature implementation depends on it. + +### Local-first persistence + +Authority: `docs/wildside-pwa-design.md` (`authoritative`). + +Supporting sources: `docs/wildside-pwa-data-model.md`, +`docs/v2a-front-end-stack.md`, and `docs/local-first-react.md` (`supporting`). + +Current status: normal domain data belongs in TanStack Query and should be +persisted to IndexedDB. Heavy assets and mutation queues belong in Dexie or +Cache Storage according to the asset type and operational requirements. + +Follow-up: cache versioning, outbox retry semantics, and conflict-resolution +policy need a `write ADR` follow-up if they become shared platform policy +rather than feature-local implementation details. + +### PWA installability + +Authority: `docs/wildside-pwa-design.md` (`authoritative`). + +Supporting source: +`docs/building-accessible-and-responsive-progressive-web-applications.md` +(`supporting`). + +Current status: Wildside is intended to be installable with a Web App +Manifest, app-like display mode, icons, `start_url`, and theme colours. + +Follow-up: manifest implementation is an `implementation follow-up` for the +PWA hardening workstream. No additional source reconciliation is needed for +item 0.1.1. + +### Service worker and caching policy + +Authority: `docs/wildside-pwa-design.md` (`authoritative`). + +Supporting source: `docs/wildside-pwa-data-model.md` (`supporting`). + +Current status: the app shell uses cache-first precaching, catalogue reads use +network-first with cached fallback, route generation status uses network-only +or network-first plus WebSocket events, and tile caching differs between normal +browsing and offline bundles. + +Follow-up: service-worker update strategy and cache-version governance should +be captured by a `write ADR` follow-up before implementation hardens the +runtime policy. + +### Map stack + +Authority: `docs/wildside-pwa-design.md` (`authoritative`). + +Supporting sources: `docs/v2a-front-end-stack.md`, +`docs/wildside-pwa-data-model.md`, and `docs/sitemap.md` (`supporting`). + +Current status: Wildside's target map stack uses MapLibre GL JS, a stable map +canvas per map route view, OpenMapTiles-backed styling, a MapLibre +right-to-left (RTL) text plugin when RTL locales are active, and shared map +state for viewport and overlays. + +Follow-up: MapLibre dependency introduction and tile-provider strategy are +later `implementation follow-up` and `write ADR` candidates. Current +`frontend-pwa/package.json` does not declare MapLibre. + +### Data model and card model + +Authority: `docs/wildside-pwa-data-model.md` (`authoritative`). + +Supporting sources: `docs/data-model-driven-card-architecture.md` and +`docs/v2a-front-end-stack.md` (`supporting`). + +Current status: entity and value-object shapes cover Explore, Discover, +Customize, Map, Safety, offline downloads, user preferences, notes, progress, +walk sessions, route plans, and card-level projections. The backend must not +ship CSS classes; it provides semantic identifiers that the client maps to +presentation. + +Follow-up: richer card-model fixture migration is an `implementation +follow-up`. If data-shape gaps are found while reconciling contracts, they +belong in an `update data model` follow-up. + +### Localization and RTL + +Authority: `docs/wildside-pwa-design.md` and +`docs/wildside-pwa-data-model.md` (`authoritative` as an authority set). + +Supporting sources: `docs/v2a-front-end-stack.md` and +`docs/pure-accessible-and-localizable-react-components.md` (`supporting`). + +Current status: UI chrome belongs in translation resources, entity display text +belongs in entity localization maps, unsupported locale tags fall back to +`en-GB`, and RTL behaviour should use CSS logical properties plus MapLibre RTL +text support when map labels are present. + +Follow-up: actual i18next, Fluent, supported-locale metadata, and RTL test +harness introduction are later `implementation follow-up` items. Any conflict +between UI-resource and entity-owned strings should be settled through +`merge into PWA design` or `update data model`, depending on which surface owns +the string. + +### Accessibility + +Authority: `docs/wildside-pwa-design.md` (`authoritative`). + +Supporting sources: +`docs/high-velocity-accessibility-first-component-testing.md`, +`docs/pure-accessible-and-localizable-react-components.md`, and +`docs/building-accessible-and-responsive-progressive-web-applications.md` +(`supporting`). + +Current status: Wildside targets Web Content Accessibility Guidelines (WCAG) +2.2 Level AA. Requirements include semantic HyperText Markup Language (HTML), +Radix primitives for complex widgets, visible focus, focus-not-obscured +behaviour, skip links, route focus management, and route-change announcements. + +Follow-up: executable Playwright, axe, and manual accessibility gates are +later `implementation follow-up` items. The policy source is stable for 0.1.1. + +### Styling and tokens + +Authority: `docs/v2a-front-end-stack.md` and `docs/wildside-pwa-design.md` +(`authoritative` as an authority set). + +Supporting sources: `docs/tailwind-v4-guide.md`, `docs/daisyui-v5-guide.md`, +`docs/semantic-tailwind-with-daisyui-best-practice.md`, and +`docs/enforcing-semantic-tailwind-best-practice.md` (`supporting`). + +Current status: current implementation uses Tailwind CSS `^3`, DaisyUI `^4`, +semantic project classes, and generated repository tokens. The target +architecture uses Tailwind CSS v4, DaisyUI v5, Radix state attributes, semantic +HTML, semantic classes, and generated design tokens. + +Follow-up: current Tailwind v3 and DaisyUI v4 package state conflicts with the +target v4/v5 platform direction. The repository already treats +`docs/v2a-front-end-stack.md` as the precedence source, and migration belongs +to `implementation follow-up` plus 0.2.4 and 0.2.5. + +### REST contracts + +Authority: `spec/openapi.json` (`authoritative`). + +Supporting sources: `docs/wildside-pwa-data-model.md` and +`docs/wildside-pwa-design.md` (`supporting`). + +Current status: the implemented REST spec currently covers login, users, +current-user reads, interest and preference writes, route annotations, route +notes, route progress, and health checks. + +Follow-up: the data model describes endpoint families that are absent from the +implemented OpenAPI spec, including catalogue explore snapshots, interest-theme +listing as a catalogue endpoint, route generation creation/status/detail, and +offline bundle management. These gaps need `update OpenAPI` before feature +implementation can cite implemented wire contracts. + +### WebSocket and event contracts + +Authority: `spec/asyncapi.yaml` (`authoritative`). + +Supporting sources: `docs/adr-001-websockets-on-actix-ws.md`, +`docs/wildside-pwa-design.md`, and +`docs/wildside-ux-state-graph-v0.1.json` (`supporting`). + +Current status: the implemented AsyncAPI contract describes `/ws` display-name +submission, invalid-display-name replies, and user-created events. ADR 001 +owns the backend adapter choice and confirms that WebSocket handling remains an +inbound adapter. + +Follow-up: route-generation progress and offline-bundle progress events are +described by the PWA design and state graph but not by `spec/asyncapi.yaml`. +Those gaps need `update AsyncAPI` before implementation treats them as +implemented event contracts. + +### UX state graph + +Authority: `docs/wildside-ux-state-graph-v0.1.json` (`authoritative`). + +Supporting sources: `docs/wildside-pwa-design.md`, +`docs/wildside-pwa-data-model.md`, and `docs/sitemap.md` (`supporting`). + +Current status: the graph owns state identifiers, user-visible state metadata, +transition intent, state-specific local and server state, persistence notes, +and accessibility annotations. It covers runtime, sync, welcome, discover, +explore, customize, wizard, map, saved, offline, safety, completion, and auth +states. + +Follow-up: auth and future route states exist in the graph but are not +represented as current sitemap routes. This needs `roadmap citation fix` or +`merge into PWA design`, depending on whether they remain future states or +become planned route work. + +### Sitemap routes + +Authority: `docs/sitemap.md` (`authoritative`). + +Supporting sources: `docs/wildside-ux-state-graph-v0.1.json` and +`docs/wildside-pwa-design.md` (`supporting`). + +Current status: the sitemap owns planned route paths, bottom navigation groups, +nested map routes, feature modules, and high-level user flows. + +Follow-up: route names and future auth states should be reconciled against the +state graph through `roadmap citation fix` or `merge into PWA design` before +later roadmap phases use them as implementation tickets. + +### Testing and validation + +Authority: `docs/wildside-pwa-design.md` and +`docs/high-velocity-accessibility-first-component-testing.md` +(`authoritative` as an authority set). + +Supporting sources: `docs/v2a-front-end-stack.md`, `docs/wildside-testing-guide.md`, +`docs/rstest-bdd-users-guide.md`, `Makefile`, and `frontend-pwa/package.json` +(`supporting`). + +Current status: executable repository gates are `make check-fmt`, `make lint`, +`make test`, `make markdownlint`, and `make nixie`. Current front-end unit +tests use Vitest. Target-state front-end validation includes component tests, +accessibility tests, Playwright browser checks, keyboard flows, localization +regression tests, Gherkin behavioural tests where applicable, property tests +where invariants are introduced, and proof tooling where axioms or contractual +business logic require it. + +Follow-up: importing the richer v2a lint, Playwright, axe, Gherkin, +`fast-check`, and proof gates is an `implementation follow-up` for 0.2.4 and +later implementation phases. + +### Semantic linting + +Authority: `docs/enforcing-semantic-tailwind-best-practice.md` +(`authoritative`). + +Supporting sources: `docs/semantic-tailwind-with-daisyui-best-practice.md` and +`docs/v2a-front-end-stack.md` (`supporting`). + +Current status: the semantic lint policy covers GritQL, Semgrep, Stylelint, +Biome integration, semantic landmarks, heading structure, daisyUI component +class misuse, state-slot classes, class-list length, and raw colour rules. + +Follow-up: the rules from the v2a mockup need `implementation follow-up` import +work under roadmap item 0.2.4 before they become executable local gates. + +### Property tests + +Authority: `docs/wildside-pwa-design.md` (`authoritative` for when validation +is required). + +Supporting sources: `docs/v2a-front-end-stack.md` and +`docs/high-velocity-accessibility-first-component-testing.md` (`supporting`). + +Current status: item 0.1.1 is documentation-only and introduces no invariant +over inputs, states, orderings, or transitions. Property tests are therefore +not applicable to this catalogue. + +Follow-up: when later phases introduce invariants, add `fast-check` property +tests as an `implementation follow-up` in the same feature slice. + +### Proof obligations + +Authority: `docs/wildside-pwa-design.md` (`authoritative` for when proof is +required). + +Supporting sources: `docs/v2a-front-end-stack.md` and the LemmaScript upstream +project (`supporting`). + +Current status: item 0.1.1 introduces no axiom or contractual business logic. +An exhaustive proof is therefore not applicable to this catalogue. + +Follow-up: if later phases introduce business axioms, protocol invariants, or +contractual state-machine properties, add a substantive proof as an +`implementation follow-up` in that feature slice. + +### Documentation ownership + +Authority: `docs/documentation-style-guide.md` (`authoritative`). + +Supporting sources: `AGENTS.md`, `docs/contents.md`, and this catalogue +(`supporting`). + +Current status: design decisions belong in design documents or ADRs, +implementation queues belong in roadmaps, and source-authority classification +belongs in this catalogue. Markdown uses en-GB-oxendict spelling and the +project Markdown rules. + +Follow-up: if later reconciliation turns catalogue follow-ups into accepted +policy, update the owning design document or write an ADR. Do not leave policy +only in `docs/frontend-roadmap.md`. + +### Hexagonal boundary ownership + +Authority: `docs/wildside-pwa-data-model.md` (`authoritative`). + +Supporting sources: `docs/wildside-backend-architecture.md`, +`docs/adr-001-websockets-on-actix-ws.md`, and the `hexagonal-architecture` +skill (`supporting`). + +Current status: domain types and ports remain framework-independent, adapters +translate at boundaries, and front-end code consumes backend contracts through +OpenAPI, AsyncAPI, generated clients, or documented ports. The WebSocket ADR +confirms that backend event handling remains an inbound adapter concern. + +Follow-up: any front-end implementation that would depend directly on backend +adapter internals must stop and create an `update design document` or +`write ADR` follow-up instead. + +## Reconciliation follow-ups + +- `update OpenAPI`: add or explicitly defer catalogue explore snapshots, + interest-theme listing as a catalogue contract, route generation + creation/status/detail, route annotations parity, and offline bundle + management before front-end feature work treats them as implemented REST + contracts. +- `update AsyncAPI`: add or explicitly defer route-generation progress, + offline-bundle progress, and other progress events described by the PWA + design and state graph before front-end feature work treats them as + implemented WebSocket contracts. +- `write ADR`: settle durable front-end platform policy for client-state + ownership, service-worker update strategy, cache versioning, outbox retry + semantics, conflict handling, and map tile provider strategy. +- `merge into PWA design`: reconcile auth and future route states between the + state graph, sitemap, and staged PWA design. +- `roadmap citation fix`: once the design and contract owners are reconciled, + replace later roadmap prose that reads as policy with citations to the + authoritative source. +- `implementation follow-up`: import target-state validation gates, semantic + linting, Playwright accessibility checks, localization checks, Gherkin + behavioural tests, `fast-check` property tests, and substantive proof tooling + only in the feature slices that require them. + +## Relevant skills and tooling + +- Use the `leta` skill for code navigation before any code or symbol-level + refactor. +- Use the `rust-router` skill before Rust implementation work, then load the + smallest relevant Rust follow-on skill. +- Use the `hexagonal-architecture` skill when changing or reviewing boundaries + between domain policy, ports, and adapters. +- Use the `execplans` skill for non-trivial implementation plans and keep the + plan's living sections current. +- Use Firecrawl when current external information is needed for open-source + tooling, common protocols, specifications, or prior art. + +## Validation note for item 0.1.1 + +This catalogue changes documentation only. It introduces no executable +front-end surface, runtime strings, card model fixture data, APIs, persistence, +or CSS. Playwright and `css-view` therefore have no rendered UI to validate for +this item. They remain required gates for later executable front-end work, with +no errors, failures, or accessibility violations permitted when applicable. From c84a702ce172d8b40f714b90cdfc571b1d408c77 Mon Sep 17 00:00:00 2001 From: leynos Date: Wed, 20 May 2026 21:59:14 +0200 Subject: [PATCH 3/9] Close front-end authority catalogue plan Mark the implementation ExecPlan complete after the catalogue commit, branch push, and draft pull request update finished. --- ...-1-front-end-source-authority-catalogue.md | 28 +++++++++---------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/execplans/frontend-0-1-1-front-end-source-authority-catalogue.md b/docs/execplans/frontend-0-1-1-front-end-source-authority-catalogue.md index 0a2814a5..dec36462 100644 --- a/docs/execplans/frontend-0-1-1-front-end-source-authority-catalogue.md +++ b/docs/execplans/frontend-0-1-1-front-end-source-authority-catalogue.md @@ -5,7 +5,7 @@ This ExecPlan (execution plan) is a living document. The sections `Decision Log`, and `Outcomes & Retrospective` must be kept up to date as work proceeds. -Status: IN PROGRESS +Status: COMPLETE ## Purpose / big picture @@ -166,25 +166,26 @@ tool behaviour, which is outside the expected scope. changed. - [x] (2026-05-20T19:43:26Z) Receive explicit user approval to implement roadmap item 0.1.1 from this ExecPlan. -- [x] (2026-05-20T20:02:00Z) Draft +- [x] (2026-05-20T19:45:00Z) Draft `docs/frontend-source-authority-catalogue.md` with source classifications, topic authorities, reconciliation follow-ups, skills, and validation notes. -- [x] (2026-05-20T20:02:00Z) Mark roadmap item 0.1.1 done and cite the +- [x] (2026-05-20T19:45:00Z) Mark roadmap item 0.1.1 done and cite the catalogue from `docs/frontend-roadmap.md`. -- [x] (2026-05-20T20:02:00Z) Add the catalogue to +- [x] (2026-05-20T19:45:00Z) Add the catalogue to `docs/developers-guide.md` and `docs/contents.md`. -- [x] (2026-05-20T20:05:00Z) Run `make fmt` and `make markdownlint`; both +- [x] (2026-05-20T19:46:00Z) Run `make fmt` and `make markdownlint`; both passed for the documentation update. -- [x] (2026-05-20T20:12:00Z) Run `coderabbit review --agent`; the review +- [x] (2026-05-20T19:52:00Z) Run `coderabbit review --agent`; the review completed successfully with zero findings. -- [x] (2026-05-20T20:15:00Z) Run `make nixie`; all diagrams validated +- [x] (2026-05-20T19:53:00Z) Run `make nixie`; all diagrams validated successfully. -- [x] (2026-05-20T20:16:00Z) Run `css-view --help`; the installed command is a +- [x] (2026-05-20T19:53:00Z) Run `css-view --help`; the installed command is a Playwright-backed CSS snapshot CLI that requires a URL, and no executable front-end surface changed in this documentation-only item. -- [x] (2026-05-20T21:57:00Z) Run `make check-fmt`, `make lint`, and +- [x] (2026-05-20T19:57:00Z) Run `make check-fmt`, `make lint`, and `make test`; all repository gates passed. -- [ ] Commit, push, and update the pull request for the implementation branch. +- [x] (2026-05-20T19:58:28Z) Commit the implementation, push the branch, and + update draft pull request #355. ## Surprises & discoveries @@ -298,13 +299,12 @@ tool behaviour, which is outside the expected scope. ownership for front-end work, so the developer guide and documentation index should point to it. The developers' guide update also corrects the existing front-end build target name to match the Makefile. - Date/Author: 2026-05-20T20:02:00Z / Codex. + Date/Author: 2026-05-20T19:45:00Z / Codex. ## Outcomes & retrospective -This section is not complete. During implementation, update it after the -catalogue is drafted, after validation passes, after CodeRabbit review is -resolved, and after the commit and pull request are created. +The catalogue implementation is complete. The branch has been pushed and draft +pull request #355 has been updated for implementation review. For the pre-approval plan review, the following commands have passed: From 651135a992adfb664c871feee69a8727a6a84883 Mon Sep 17 00:00:00 2001 From: leynos Date: Sun, 24 May 2026 23:49:52 +0200 Subject: [PATCH 4/9] Stabilize embedded Postgres bootstrap Pin the embedded PostgreSQL binary release source used by backend test bootstrap to the Theseus binary repository when no override is already set. Keep existing `POSTGRESQL_RELEASES_URL` values intact so CI and local runs can still opt into a different source deliberately. Update `postgresql_embedded`, `pg-embed-setup-unpriv`, and the Makefile `pg_worker` install version together so the test runner does not keep using an older bootstrap helper while the backend depends on the newer crates. --- Cargo.lock | 8 ++--- Makefile | 2 +- backend/Cargo.toml | 2 +- backend/tests/support/atexit_cleanup.rs | 40 ++++++++++++++++++++++++- 4 files changed, 45 insertions(+), 7 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index feb9f2cc..8241cbc5 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2647,7 +2647,7 @@ dependencies = [ "libc", "percent-encoding", "pin-project-lite", - "socket2 0.6.0", + "socket2 0.5.10", "tokio", "tower-service", "tracing", @@ -4340,7 +4340,7 @@ dependencies = [ "quinn-udp", "rustc-hash", "rustls", - "socket2 0.6.0", + "socket2 0.5.10", "thiserror 2.0.18", "tokio", "tracing", @@ -4377,7 +4377,7 @@ dependencies = [ "cfg_aliases", "libc", "once_cell", - "socket2 0.6.0", + "socket2 0.5.10", "tracing", "windows-sys 0.60.2", ] @@ -5914,7 +5914,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd" dependencies = [ "fastrand", - "getrandom 0.3.3", + "getrandom 0.4.2", "once_cell", "rustix 1.1.4", "windows-sys 0.61.2", diff --git a/Makefile b/Makefile index ede56749..b2fe3dc1 100644 --- a/Makefile +++ b/Makefile @@ -158,7 +158,7 @@ lint-actions: PG_WORKER_PATH ?= $(CURDIR)/target/pg_worker PG_WORKER_INSTALL_ROOT ?= $(CURDIR)/target/pg-worker-root -PG_EMBED_SETUP_UNPRIV_VERSION ?= 0.5.0 +PG_EMBED_SETUP_UNPRIV_VERSION ?= 0.5.1 NEXTEST_TEST_THREADS ?= 1 test: test-rust test-frontend diff --git a/backend/Cargo.toml b/backend/Cargo.toml index eec787a8..74f84190 100644 --- a/backend/Cargo.toml +++ b/backend/Cargo.toml @@ -23,7 +23,7 @@ clap = { version = "4", features = ["derive"] } async-trait = "0.1.89" color-eyre = "0.6.5" pg_embedded_setup_unpriv = { package = "pg-embed-setup-unpriv", version = "0.5.1" } -postgresql_embedded = { version = "0.20.0", features = ["tokio"] } +postgresql_embedded = { version = "0.20.2", features = ["tokio"] } postgres = { version = "0.19.12", features = ["with-uuid-1"] } paste = "1.0.15" thiserror = "2.0.17" diff --git a/backend/tests/support/atexit_cleanup.rs b/backend/tests/support/atexit_cleanup.rs index 371bfebe..80401358 100644 --- a/backend/tests/support/atexit_cleanup.rs +++ b/backend/tests/support/atexit_cleanup.rs @@ -160,6 +160,18 @@ fn ensure_stable_password() { std::env::set_var("PG_PASSWORD", "wildside_embedded_test"); } } + if std::env::var_os("POSTGRESQL_RELEASES_URL").is_none() { + // SAFETY: called before the library spawns any threads. The shared + // cluster singleton serializes access with a `Mutex`, so this runs at + // most once per process. + unsafe { + // Pin the release source to Theseus to avoid transient fetch issues in CI + std::env::set_var( + "POSTGRESQL_RELEASES_URL", + "https://github.com/theseus-rs/postgresql-binaries", + ); + } + } } /// Reads the postmaster PID from the `postmaster.pid` file in `data_dir`. @@ -287,13 +299,39 @@ mod tests { #[test] fn ensure_stable_password_does_not_overwrite_existing_value() { - let _guard = env_lock::lock_env([("PG_PASSWORD", Some("custom_value"))]); + let _guard = env_lock::lock_env([ + ("PG_PASSWORD", Some("custom_value")), + ( + "POSTGRESQL_RELEASES_URL", + Some("https://example.invalid/postgresql-binaries"), + ), + ]); super::ensure_stable_password(); assert_eq!( std::env::var("PG_PASSWORD").expect("PG_PASSWORD should be set"), "custom_value", "ensure_stable_password should not overwrite an existing PG_PASSWORD" ); + assert_eq!( + std::env::var("POSTGRESQL_RELEASES_URL") + .expect("POSTGRESQL_RELEASES_URL should be set"), + "https://example.invalid/postgresql-binaries", + "ensure_stable_password should not overwrite an existing release URL" + ); + } + + #[test] + fn ensure_stable_password_sets_release_url_when_missing() { + let _guard = env_lock::lock_env([ + ("PG_PASSWORD", Some("custom_value")), + ("POSTGRESQL_RELEASES_URL", None), + ]); + super::ensure_stable_password(); + assert_eq!( + std::env::var("POSTGRESQL_RELEASES_URL") + .expect("POSTGRESQL_RELEASES_URL should be set"), + "https://github.com/theseus-rs/postgresql-binaries" + ); } #[test] From 9212ddadf74eb7003acc7dacc1591207b2cc1d11 Mon Sep 17 00:00:00 2001 From: leynos Date: Mon, 25 May 2026 21:00:58 +0200 Subject: [PATCH 5/9] Separate embedded Postgres binary cache Split the PostgreSQL embedded binary cache out of the Cargo cache in CI so `Cargo.lock` churn does not evict the downloaded server archive. Point `POSTGRESQL_RELEASES_URL` at the Theseus releases endpoint to keep test bootstrap downloads on the intended source. Keep the existing `GITHUB_TOKEN` and `PG_EMBEDDED_WORKER` test environment intact. --- .github/workflows/ci.yml | 7 ++++++- backend/tests/support/atexit_cleanup.rs | 7 ++++--- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 4229c332..4605b459 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -138,10 +138,15 @@ jobs: path: | ~/.cargo/registry ~/.cargo/git - ~/.theseus/postgresql target key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }} restore-keys: ${{ runner.os }}-cargo- + - name: Cache PostgreSQL embedded binaries + uses: ubicloud/cache@v4 + with: + path: ~/.theseus/postgresql + key: ${{ runner.os }}-pg-embedded-${{ hashFiles('**/Cargo.lock') }} + restore-keys: ${{ runner.os }}-pg-embedded- - name: Rust build run: cargo check --locked --manifest-path backend/Cargo.toml --all-targets --all-features - name: Rust fmt check diff --git a/backend/tests/support/atexit_cleanup.rs b/backend/tests/support/atexit_cleanup.rs index 80401358..817648fa 100644 --- a/backend/tests/support/atexit_cleanup.rs +++ b/backend/tests/support/atexit_cleanup.rs @@ -161,14 +161,15 @@ fn ensure_stable_password() { } } if std::env::var_os("POSTGRESQL_RELEASES_URL").is_none() { + // Pin to Theseus binaries to prevent transient fetch failures in CI + // (misreported by reqwest as "error decoding response body"). // SAFETY: called before the library spawns any threads. The shared // cluster singleton serializes access with a `Mutex`, so this runs at // most once per process. unsafe { - // Pin the release source to Theseus to avoid transient fetch issues in CI std::env::set_var( "POSTGRESQL_RELEASES_URL", - "https://github.com/theseus-rs/postgresql-binaries", + "https://github.com/theseus-rs/postgresql-binaries/releases", ); } } @@ -330,7 +331,7 @@ mod tests { assert_eq!( std::env::var("POSTGRESQL_RELEASES_URL") .expect("POSTGRESQL_RELEASES_URL should be set"), - "https://github.com/theseus-rs/postgresql-binaries" + "https://github.com/theseus-rs/postgresql-binaries/releases" ); } From b9409cb3e04e478552b6ac03bb41eab55286f5de Mon Sep 17 00:00:00 2001 From: leynos Date: Tue, 26 May 2026 18:38:54 +0200 Subject: [PATCH 6/9] Clarify embedded Postgres release pin Align the `POSTGRESQL_RELEASES_URL` bootstrap comment with the CI timeout failure mode and keep the pinned Theseus releases endpoint explicit. Refresh `Cargo.lock` after rerunning the embedded PostgreSQL dependency update so the lockfile records the current resolver output. --- Cargo.lock | 6 +++--- backend/tests/support/atexit_cleanup.rs | 8 ++++---- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 8241cbc5..3f443642 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2647,7 +2647,7 @@ dependencies = [ "libc", "percent-encoding", "pin-project-lite", - "socket2 0.5.10", + "socket2 0.6.0", "tokio", "tower-service", "tracing", @@ -4340,7 +4340,7 @@ dependencies = [ "quinn-udp", "rustc-hash", "rustls", - "socket2 0.5.10", + "socket2 0.6.0", "thiserror 2.0.18", "tokio", "tracing", @@ -4377,7 +4377,7 @@ dependencies = [ "cfg_aliases", "libc", "once_cell", - "socket2 0.5.10", + "socket2 0.6.0", "tracing", "windows-sys 0.60.2", ] diff --git a/backend/tests/support/atexit_cleanup.rs b/backend/tests/support/atexit_cleanup.rs index 817648fa..e0edf7c5 100644 --- a/backend/tests/support/atexit_cleanup.rs +++ b/backend/tests/support/atexit_cleanup.rs @@ -161,10 +161,10 @@ fn ensure_stable_password() { } } if std::env::var_os("POSTGRESQL_RELEASES_URL").is_none() { - // Pin to Theseus binaries to prevent transient fetch failures in CI - // (misreported by reqwest as "error decoding response body"). - // SAFETY: called before the library spawns any threads. The shared - // cluster singleton serializes access with a `Mutex`, so this runs at + // Pin to Theseus binaries to avoid transient fetch failures in CI that + // reqwest misreports as "error decoding response body". + // SAFETY: called before the library spawns any threads. The shared- + // cluster singleton serialises access with a Mutex, so this runs at // most once per process. unsafe { std::env::set_var( From c21ed6f32a27cdb39c9a2b233bf76c39e9e2a60b Mon Sep 17 00:00:00 2001 From: leynos Date: Tue, 26 May 2026 20:51:32 +0200 Subject: [PATCH 7/9] Refresh embedded Postgres lockfile Record the lockfile normalization produced by the required embedded Postgres dependency update so CI and local builds use the same resolved graph. --- Cargo.lock | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 3f443642..8241cbc5 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2647,7 +2647,7 @@ dependencies = [ "libc", "percent-encoding", "pin-project-lite", - "socket2 0.6.0", + "socket2 0.5.10", "tokio", "tower-service", "tracing", @@ -4340,7 +4340,7 @@ dependencies = [ "quinn-udp", "rustc-hash", "rustls", - "socket2 0.6.0", + "socket2 0.5.10", "thiserror 2.0.18", "tokio", "tracing", @@ -4377,7 +4377,7 @@ dependencies = [ "cfg_aliases", "libc", "once_cell", - "socket2 0.6.0", + "socket2 0.5.10", "tracing", "windows-sys 0.60.2", ] From b00dd5cbf2dabe98fbdb9bbe08aa266182d8c2ab Mon Sep 17 00:00:00 2001 From: leynos Date: Wed, 27 May 2026 00:46:42 +0200 Subject: [PATCH 8/9] Warm embedded Postgres cache in CI Add a CI cache warm-up script that downloads the exact Theseus PostgreSQL archive, verifies its checksum, and populates the pg-embed-setup-unpriv binary cache before nextest starts. Cache both postgresql_embedded and pg-embed-setup-unpriv binary directories, and align CI nextest scheduling with the repository Makefile so first-use cluster bootstrap is serialized. Extend the pg-embed nextest group to cover the remaining PostgreSQL-backed integration binaries. Stabilize the audit exception property test by excluding Fast Check's intentional `Invalid Date` values from a property that exercises real calendar dates. --- .config/nextest.toml | 11 + .github/workflows/ci.yml | 16 +- docs/developers-guide.md | 4 + scripts/security-audit-reporting.test.mjs | 41 ++- scripts/warm-pg-embedded-cache.sh | 374 ++++++++++++++++++++++ 5 files changed, 428 insertions(+), 18 deletions(-) create mode 100644 scripts/warm-pg-embedded-cache.sh diff --git a/.config/nextest.toml b/.config/nextest.toml index 749ea174..898ca22c 100644 --- a/.config/nextest.toml +++ b/.config/nextest.toml @@ -19,6 +19,8 @@ max-threads = 1 filter = """ binary(diesel_user_repository) | binary(diesel_user_preferences_repository) + | binary(diesel_login_users_adapters) + | binary(diesel_profile_interests_adapters) | binary(diesel_route_annotation_repository) | binary(diesel_example_data_runs_repository) | binary(er_snapshots_bdd) @@ -29,9 +31,18 @@ filter = """ | binary(catalogue_descriptor_ingestion_bdd) | binary(catalogue_descriptor_read_models_bdd) | binary(offline_bundle_walk_session_bdd) + | binary(osm_ingestion_bdd) + | binary(overpass_enrichment_bdd) | binary(ports_behaviour) | binary(pg_embed_isolation) + | binary(route_queue_apalis_bdd) | binary(schema_baseline_bdd) + | binary(startup_mode_composition_bdd) + | binary(user_interests_revision_conflicts_bdd) + | binary(user_state_profile_interests_startup_modes_bdd) + | binary(user_state_schema_audit_bdd) + | binary(user_state_startup_modes_bdd) + | binary(users_list_pagination_bdd) """ test-group = "pg-embed" threads-required = 4 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 4605b459..d2863d8d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -144,7 +144,9 @@ jobs: - name: Cache PostgreSQL embedded binaries uses: ubicloud/cache@v4 with: - path: ~/.theseus/postgresql + path: | + ~/.theseus/postgresql + ~/.cache/pg-embedded/binaries key: ${{ runner.os }}-pg-embedded-${{ hashFiles('**/Cargo.lock') }} restore-keys: ${{ runner.os }}-pg-embedded- - name: Rust build @@ -173,6 +175,16 @@ jobs: run: | make prepare-pg-worker + - name: Warm PostgreSQL embedded binary cache + env: + # Increase GitHub API rate limits for postgresql_embedded downloads. + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + # Pin the embedded PostgreSQL version so postgresql_archive skips release-listing queries. + # Use the semver "exact" prefix so no wildcard resolution is attempted. + POSTGRESQL_VERSION: "=16.10.0" + POSTGRESQL_RELEASES_URL: https://github.com/theseus-rs/postgresql-binaries/releases + run: bash scripts/warm-pg-embedded-cache.sh + - name: Rust tests env: # Increase GitHub API rate limits for postgresql_embedded downloads. @@ -184,6 +196,8 @@ jobs: PG_TEST_BACKEND: postgresql_embedded # Root-path bootstrap requires the worker binary for privilege demotion. PG_EMBEDDED_WORKER: ${{ github.workspace }}/target/pg_worker + # Match the repository Makefile and avoid concurrent first-use cluster bootstrap. + NEXTEST_TEST_THREADS: "1" run: | # Clean stale pg-embed directories that may conflict with new runs. find target/ -maxdepth 1 -type d -name 'pg-embed-*' -exec rm -rf {} + 2>/dev/null || true diff --git a/docs/developers-guide.md b/docs/developers-guide.md index ef26b432..aa918904 100644 --- a/docs/developers-guide.md +++ b/docs/developers-guide.md @@ -230,6 +230,10 @@ test usage remains coherent: rerunning migrations per test. - Use `CleanupMode::None` only for explicit debugging sessions where retained files are required; keep deterministic cleanup defaults for normal runs. +- Continuous Integration (CI) warms the `pg-embed-setup-unpriv` binary cache + with `scripts/warm-pg-embedded-cache.sh` before running `cargo nextest`. + Cache both `~/.theseus/postgresql` and `~/.cache/pg-embedded/binaries` when + changing CI runners or cache providers. ## Rust behavioural tests with `rstest-bdd` v0.5.0 diff --git a/scripts/security-audit-reporting.test.mjs b/scripts/security-audit-reporting.test.mjs index f2c44fbf..c8f16b4f 100644 --- a/scripts/security-audit-reporting.test.mjs +++ b/scripts/security-audit-reporting.test.mjs @@ -123,23 +123,30 @@ describe('assertNoExpired', () => { it('allows exceptions expiring on or after the current date', () => { fc.assert( - fc.property(fc.date({ min: new Date('2024-01-01'), max: new Date('2030-12-31') }), (date) => { - const today = date.toISOString().slice(0, 10); - const policyIo = throwingPolicyIo(); - - assertNoExpired( - [ - exceptionEntry({ - addedAt: '2024-01-01', - expiresAt: today, - }), - ], - date, - policyIo, - ); - - expect(policyIo.exit).not.toHaveBeenCalled(); - }), + fc.property( + fc.date({ + max: new Date('2030-12-31'), + min: new Date('2024-01-01'), + noInvalidDate: true, + }), + (date) => { + const today = date.toISOString().slice(0, 10); + const policyIo = throwingPolicyIo(); + + assertNoExpired( + [ + exceptionEntry({ + addedAt: '2024-01-01', + expiresAt: today, + }), + ], + date, + policyIo, + ); + + expect(policyIo.exit).not.toHaveBeenCalled(); + }, + ), ); }); diff --git a/scripts/warm-pg-embedded-cache.sh b/scripts/warm-pg-embedded-cache.sh new file mode 100644 index 00000000..5b4be06c --- /dev/null +++ b/scripts/warm-pg-embedded-cache.sh @@ -0,0 +1,374 @@ +#!/usr/bin/env bash +# Warm the PostgreSQL binary cache used by embedded database tests. +# +# CI invokes this before cargo nextest, and developers may run it locally when +# they need deterministic PostgreSQL bootstrap without repeated downloads. The +# script populates the pg-embed-setup-unpriv cache and can copy from the +# postgresql_embedded Theseus cache when that cache is already present. +# +# Environment: +# - PG_EMBEDDED_VERSION or POSTGRESQL_VERSION: exact PostgreSQL version. +# - PG_BINARY_CACHE_DIR: override the pg-embed-setup-unpriv cache root. +# - XDG_CACHE_HOME: fallback cache root before ~/.cache when set. +# - POSTGRESQL_RELEASES_URL: override the Theseus release repository URL. +set -euo pipefail + +CACHE_LOCK_DIR='' +DOWNLOAD_WORK_DIR='' +EXIT_CLEANUP_REGISTERED=false +PREVIOUS_EXIT_HANDLER='' + +log() { + printf '[pg-embedded-cache] %s\n' "$*" >&2 +} + +warn() { + printf '[pg-embedded-cache] warning: %s\n' "$*" >&2 +} + +fail() { + printf '[pg-embedded-cache] error: %s\n' "$*" >&2 + exit 1 +} + +cleanup_download_work_dir() { + if [[ -n "$DOWNLOAD_WORK_DIR" ]]; then + rm -rf "$DOWNLOAD_WORK_DIR" + DOWNLOAD_WORK_DIR='' + fi +} + +normalise_version() { + local raw_version="${PG_EMBEDDED_VERSION:-${POSTGRESQL_VERSION:-16.10.0}}" + local version_pattern='^[0-9]+([.][0-9]+)*$' + raw_version="${raw_version#=}" + + if [[ ! "$raw_version" =~ $version_pattern ]]; then + fail "expected an exact PostgreSQL version, got '${raw_version}'" + fi + + printf '%s\n' "$raw_version" +} + +platform_triple() { + case "$(uname -s):$(uname -m)" in + Linux:x86_64) + printf 'x86_64-unknown-linux-gnu\n' + ;; + Linux:aarch64) + printf 'aarch64-unknown-linux-gnu\n' + ;; + Darwin:x86_64) + printf 'x86_64-apple-darwin\n' + ;; + Darwin:arm64) + printf 'aarch64-apple-darwin\n' + ;; + *) + fail "unsupported platform for PostgreSQL cache warm-up: $(uname -s) $(uname -m)" + ;; + esac +} + +cache_dir() { + if [[ -n "${PG_BINARY_CACHE_DIR:-}" ]]; then + printf '%s\n' "$PG_BINARY_CACHE_DIR" + elif [[ -n "${XDG_CACHE_HOME:-}" ]]; then + printf '%s/pg-embedded/binaries\n' "$XDG_CACHE_HOME" + elif [[ -n "${HOME:-}" ]]; then + printf '%s/.cache/pg-embedded/binaries\n' "$HOME" + else + printf '%s/pg-embedded/binaries\n' "${TMPDIR:-/tmp}" + fi +} + +release_base_url() { + local raw_url="${POSTGRESQL_RELEASES_URL:-https://github.com/theseus-rs/postgresql-binaries}" + raw_url="${raw_url%/}" + raw_url="${raw_url%/releases}" + printf '%s\n' "$raw_url" +} + +cache_is_complete() { + local version_dir="$1" + [[ -f "${version_dir}/.complete" && -x "${version_dir}/bin/postgres" ]] +} + +capture_existing_exit_handler() { + local trap_spec + + trap_spec="$(trap -p EXIT || true)" + + if [[ -z "$trap_spec" ]]; then + return + fi + + trap_spec="${trap_spec#trap -- }" + trap_spec="${trap_spec% EXIT}" + + # Bash reports trap bodies as shell-quoted strings. Evaluate only that + # already-installed handler so the new cleanup can compose with it. This + # assumes earlier EXIT handlers came from trusted code; an untrusted + # preinstalled EXIT trap would make this evaluation unsafe. + eval "PREVIOUS_EXIT_HANDLER=${trap_spec}" +} + +run_exit_cleanup() { + local status=$? + + cleanup_download_work_dir + + if [[ -n "$CACHE_LOCK_DIR" ]]; then + rm -rf "$CACHE_LOCK_DIR" + fi + + if [[ -n "$PREVIOUS_EXIT_HANDLER" ]]; then + eval "$PREVIOUS_EXIT_HANDLER" + fi + + exit "$status" +} + +register_exit_cleanup() { + if [[ "$EXIT_CLEANUP_REGISTERED" == true ]]; then + return + fi + + capture_existing_exit_handler + trap run_exit_cleanup EXIT + EXIT_CLEANUP_REGISTERED=true +} + +lock_owner_pid() { + local lock_dir="$1" + local pid + + if [[ ! -r "${lock_dir}/pid" ]]; then + return 1 + fi + + if ! IFS= read -r pid <"${lock_dir}/pid"; then + return 1 + fi + + if [[ ! "$pid" =~ ^[0-9]+$ ]]; then + return 1 + fi + + printf '%s\n' "$pid" +} + +remove_stale_cache_lock() { + local lock_dir="$1" + local owner_pid + local current_pid + + if ! owner_pid="$(lock_owner_pid "$lock_dir")"; then + return 1 + fi + + if kill -0 "$owner_pid" 2>/dev/null; then + return 1 + fi + + if ! current_pid="$(lock_owner_pid "$lock_dir")"; then + return 1 + fi + + if [[ "$current_pid" != "$owner_pid" ]]; then + return 1 + fi + + warn "removing stale PostgreSQL cache lock at ${lock_dir} for pid ${owner_pid}" + rm -f "${lock_dir}/pid" + rmdir "$lock_dir" 2>/dev/null +} + +acquire_cache_lock() { + local root_dir="$1" + local lock_dir="${root_dir}/.warm-pg-embedded-cache.lock" + local has_logged_wait=false + local waited_seconds=0 + + while ! mkdir "$lock_dir" 2>/dev/null; do + if remove_stale_cache_lock "$lock_dir"; then + continue + fi + + if ((waited_seconds >= 600)); then + fail "timed out waiting for PostgreSQL cache lock at ${lock_dir}" + fi + + if [[ "$has_logged_wait" == false ]]; then + log "waiting for cache lock at ${lock_dir} (pid $$, ${waited_seconds}s elapsed)" + has_logged_wait=true + elif ((waited_seconds > 0 && waited_seconds % 30 == 0)); then + log "still waiting for cache lock at ${lock_dir} (${waited_seconds}s elapsed)" + fi + + sleep 1 + waited_seconds=$((waited_seconds + 1)) + done + + if ! printf '%s\n' "$$" >"${lock_dir}/pid"; then + rm -rf "$lock_dir" + fail "failed to record PostgreSQL cache lock owner at ${lock_dir}" + fi + + CACHE_LOCK_DIR="$lock_dir" + register_exit_cleanup +} + +install_cache_dir() { + local prepared_dir="$1" + local version_dir="$2" + local previous_dir + local rollback_error + local rollback_status + + previous_dir="$(mktemp -d "${version_dir}.previous.XXXXXX")" + rmdir "$previous_dir" + + if [[ -e "$version_dir" ]]; then + if ! mv "$version_dir" "$previous_dir"; then + rm -rf "$prepared_dir" "$previous_dir" + fail "failed to move existing PostgreSQL cache at ${version_dir}" + fi + fi + + if mv "$prepared_dir" "$version_dir"; then + rm -rf "$previous_dir" + return + fi + + if [[ -e "$previous_dir" && ! -e "$version_dir" ]]; then + rollback_status=0 + rollback_error="$(mv "$previous_dir" "$version_dir" 2>&1)" || rollback_status=$? + if ((rollback_status != 0)); then + warn "failed to restore PostgreSQL cache from ${previous_dir} to ${version_dir}: ${rollback_error} (exit ${rollback_status})" + fi + fi + + rm -rf "$prepared_dir" + fail "failed to install PostgreSQL cache at ${version_dir}" +} + +populate_from_theseus_cache() { + local version="$1" + local version_dir="$2" + local prepared_dir + local source_dir + + if [[ -z "${HOME:-}" ]]; then + return 1 + fi + + source_dir="${HOME}/.theseus/postgresql/${version}" + + if [[ ! -x "${source_dir}/bin/postgres" ]]; then + return 1 + fi + + log "copying PostgreSQL ${version} from ${source_dir}" + prepared_dir="$(mktemp -d "${version_dir}.tmp.XXXXXX")" + if ! cp -a "${source_dir}/." "$prepared_dir/"; then + rm -rf "$prepared_dir" + fail "failed to copy PostgreSQL cache from ${source_dir}" + fi + + if ! touch "${prepared_dir}/.complete"; then + rm -rf "$prepared_dir" + fail "failed to mark copied PostgreSQL cache as complete" + fi + + install_cache_dir "$prepared_dir" "$version_dir" +} + +verify_checksum() { + local work_dir="$1" + local asset="$2" + local triple="$3" + local checksum_output + local checksum_status=0 + + if [[ "$triple" == *-apple-darwin ]]; then + checksum_output="$(cd "$work_dir" && shasum -a 256 -c "${asset}.sha256" 2>&1)" || checksum_status=$? + else + checksum_output="$(cd "$work_dir" && sha256sum -c "${asset}.sha256" 2>&1)" || checksum_status=$? + fi + + if ((checksum_status != 0)); then + fail "checksum verification failed for ${asset} using ${asset}.sha256 (exit ${checksum_status}); output: ${checksum_output}. The .sha256 may be malformed or mismatched." + fi +} + +download_and_extract() { + local version="$1" + local version_dir="$2" + local triple="$3" + local base_url="$4" + + local asset="postgresql-${version}-${triple}.tar.gz" + local work_dir + work_dir="$(mktemp -d)" + DOWNLOAD_WORK_DIR="$work_dir" + + log "downloading ${asset}" + curl --fail --location --retry 5 --retry-all-errors \ + --connect-timeout 30 --max-time 600 --speed-time 60 --speed-limit 1024 \ + --output "${work_dir}/${asset}" \ + "${base_url}/releases/download/${version}/${asset}" + curl --fail --location --retry 5 --retry-all-errors \ + --connect-timeout 30 --max-time 600 --speed-time 60 --speed-limit 1024 \ + --output "${work_dir}/${asset}.sha256" \ + "${base_url}/releases/download/${version}/${asset}.sha256" + + verify_checksum "$work_dir" "$asset" "$triple" + + local prepared_dir + prepared_dir="$(mktemp -d "${version_dir}.tmp.XXXXXX")" + if ! tar -xzf "${work_dir}/${asset}" -C "$prepared_dir" --strip-components=1; then + rm -rf "$prepared_dir" + fail "failed to extract PostgreSQL archive" + fi + + if [[ ! -x "${prepared_dir}/bin/postgres" ]]; then + rm -rf "$prepared_dir" + fail "archive did not contain bin/postgres" + fi + + if ! touch "${prepared_dir}/.complete"; then + rm -rf "$prepared_dir" + fail "failed to mark downloaded PostgreSQL cache as complete" + fi + + install_cache_dir "$prepared_dir" "$version_dir" + cleanup_download_work_dir +} + +main() { + local version + local root_dir + local version_dir + + version="$(normalise_version)" + root_dir="$(cache_dir)" + version_dir="${root_dir}/${version}" + mkdir -p "$root_dir" + acquire_cache_lock "$root_dir" + + if cache_is_complete "$version_dir"; then + log "cache hit for PostgreSQL ${version} at ${version_dir}" + return + fi + + if populate_from_theseus_cache "$version" "$version_dir"; then + log "cache warmed for PostgreSQL ${version} at ${version_dir}" + return + fi + + download_and_extract "$version" "$version_dir" "$(platform_triple)" "$(release_base_url)" + log "cache warmed for PostgreSQL ${version} at ${version_dir}" +} + +main "$@" From 0576b57c53c7f2af7b93d96cc32face44975ef77 Mon Sep 17 00:00:00 2001 From: leynos Date: Wed, 27 May 2026 11:18:17 +0200 Subject: [PATCH 9/9] Document embedded PostgreSQL CI cache hardening Refresh the vendored pg-embed-setup-unpriv user guide from the v0.5.1 upstream tag and record the CI mitigation for stalled PostgreSQL release downloads. --- docs/developers-guide.md | 43 +++++++- docs/pg-embed-setup-unpriv-users-guide.md | 121 ++++++++++++++++------ 2 files changed, 128 insertions(+), 36 deletions(-) diff --git a/docs/developers-guide.md b/docs/developers-guide.md index aa918904..d8f54625 100644 --- a/docs/developers-guide.md +++ b/docs/developers-guide.md @@ -230,10 +230,45 @@ test usage remains coherent: rerunning migrations per test. - Use `CleanupMode::None` only for explicit debugging sessions where retained files are required; keep deterministic cleanup defaults for normal runs. -- Continuous Integration (CI) warms the `pg-embed-setup-unpriv` binary cache - with `scripts/warm-pg-embedded-cache.sh` before running `cargo nextest`. - Cache both `~/.theseus/postgresql` and `~/.cache/pg-embedded/binaries` when - changing CI runners or cache providers. + +### Embedded PostgreSQL CI bootstrap stability + +Continuous Integration (CI) warms the `pg-embed-setup-unpriv` binary cache with +`scripts/warm-pg-embedded-cache.sh` before running `cargo nextest`. Keep this +warm-up step before `Rust tests`; it turns PostgreSQL binary acquisition into a +short, explicit CI step instead of letting the first integration test perform a +cold download inside `postgresql_embedded::setup()`. + +The CI cache step must include both binary-cache locations used by the two +embedded PostgreSQL layers: + +- `~/.theseus/postgresql` for `postgresql_embedded` runtime installations. +- `~/.cache/pg-embedded/binaries` for `pg-embed-setup-unpriv` release archives. + +Do not co-locate those paths inside the Cargo registry/cache archive. Cargo +dependency updates and `Cargo.lock` churn otherwise evict the PostgreSQL +binary cache and force a fresh download during unrelated test changes. + +The warm-up step pins: + +- `POSTGRESQL_VERSION="=16.10.0"` so archive resolution does not need wildcard + release discovery. +- `POSTGRESQL_RELEASES_URL=https://github.com/theseus-rs/postgresql-binaries/releases` + so the binary source cannot drift when crate defaults change. +- `GITHUB_TOKEN` so GitHub release requests use the Actions token and avoid + anonymous rate limits. + +Keep PostgreSQL-backed nextest binaries in the `pg-embedded` test group in +`.config/nextest.toml`, and keep that group serialised. First-use cluster +bootstrap is process-local and expensive; serial execution avoids concurrent +setup attempts competing for the same warmed cache, filesystem paths, and +worker process. + +If CI reports `error decoding response body`, treat it as a likely download +stall or timeout from `reqwest` rather than as JSON/body corruption. Check the +`Cache PostgreSQL embedded binaries` and `Warm PostgreSQL embedded binary +cache` steps first, then verify that the `Rust tests` step is still exporting +`PG_EMBEDDED_WORKER`, `GITHUB_TOKEN`, and `NEXTEST_TEST_THREADS=1`. ## Rust behavioural tests with `rstest-bdd` v0.5.0 diff --git a/docs/pg-embed-setup-unpriv-users-guide.md b/docs/pg-embed-setup-unpriv-users-guide.md index 0ec0e38c..429a578c 100644 --- a/docs/pg-embed-setup-unpriv-users-guide.md +++ b/docs/pg-embed-setup-unpriv-users-guide.md @@ -30,7 +30,7 @@ tool and integrate it into automated test flows. `PG_TEST_BACKEND` selects the backend used by `bootstrap_for_tests()` and `TestCluster`. Supported values are: -- unset: `postgresql_embedded` +- unset or empty: `postgresql_embedded` - `postgresql_embedded`: run the embedded PostgreSQL backend Any other value triggers a `SKIP-TEST-CLUSTER` error, so test harnesses can @@ -40,7 +40,7 @@ The embedded backend downloads PostgreSQL binaries, initializes the data directory, and writes to the configured runtime and data paths. It requires outbound network access. On Linux, root workflows must supply `PG_EMBEDDED_WORKER` so the helper can drop privileges. On macOS, root -execution is unsupported and expected to fail fast; on Windows, the backend +execution is unsupported and expected to fail fast; on Windows the backend always runs in-process. Troubleshooting guidance: @@ -52,6 +52,13 @@ Troubleshooting guidance: ## Quick start +On Linux `x86_64` and `aarch64`, tagged releases publish both CLI binaries in a +`cargo binstall` archive. Install them with: + +```bash +cargo binstall pg-embed-setup-unpriv +``` + 1. Choose directories for the staged PostgreSQL distribution and the cluster’s data files. They must be writable by whichever user will run the helper; the tool reapplies ownership and permissions on every invocation. @@ -73,16 +80,18 @@ Troubleshooting guidance: 3. Run the helper (`cargo run --release --bin pg_embedded_setup_unpriv`). The command downloads the specified PostgreSQL release, ensures the directories exist, applies PostgreSQL-compatible permissions (0755 for the installation - cache, 0700 for the runtime and data directories), and initializes the - cluster with the provided credentials. Invocations that begin as `root` - prepare directories for `nobody` and execute lifecycle commands through the - worker helper, so the privileged operations run entirely under the sandbox - user. Ownership fix-ups occur on every call, so running the tool twice - remains idempotent. - -4. Pass the resulting paths and credentials to the test suite. If - `postgresql_embedded` is used directly after the setup step, it can reuse - the staged binaries and data directory without needing `root`. + cache, 0700 for the runtime and data directories), and initialises the + cluster with the provided credentials via `initdb`. The PostgreSQL server is + **not** started — the installation is left ready for subsequent use by + `TestCluster` or other tools. Invocations that begin as `root` prepare + directories for `nobody` and execute lifecycle commands through the worker + helper, so the privileged operations run entirely under the sandbox user. + Ownership fix-ups occur on every call so running the tool twice remains + idempotent. + +4. Pass the resulting paths and credentials to your tests. If you use + `postgresql_embedded` directly after the setup step, it can reuse the staged + binaries and data directory without needing `root`. ## Bootstrap for test suites @@ -122,7 +131,7 @@ rather than when PostgreSQL launches. configuration entries into `bootstrap.settings.configuration` to minimize background and parallel worker processes for ephemeral test clusters. Override these values by mutating the configuration map before starting the cluster if -the test suite needs different behaviour. +your tests need different behaviour. ## Resource Acquisition Is Initialization (RAII) test clusters @@ -140,7 +149,7 @@ fn exercise_cluster() -> BootstrapResult<()> { let cluster = TestCluster::new()?; let url = cluster.settings().url("app_db"); // Issue queries using any preferred client here. - drop(cluster); // PostgreSQL shuts down automatically. +drop(cluster); // PostgreSQL shuts down automatically. Ok(()) } ``` @@ -164,9 +173,8 @@ drop(cluster); ``` Shared clusters created with `test_support::shared_test_cluster()` are -intentionally leaked for the process lifetime via `std::mem::forget` in the -test helper, and therefore do not perform cleanup on drop. This behaviour is -intentional because process-wide reuse is required for shared fixtures. +intentionally leaked for the process lifetime and therefore do not perform +cleanup on drop. ### Async API for `#[tokio::test]` contexts @@ -176,14 +184,14 @@ from within a runtime" because it creates its own internal Tokio runtime. Async contexts require enabling the `async-api` feature and using the async constructor and shutdown methods. -Enable the feature in the test crate's `Cargo.toml`: +Enable the feature in your `Cargo.toml`: ```toml [dev-dependencies] -pg-embed-setup-unpriv = { version = "0.5.0", features = ["async-api"] } +pg-embed-setup-unpriv = { version = "0.2", features = ["async-api"] } ``` -Then use `start_async()` and `stop_async()` in async tests: +Then use `start_async()` and `stop_async()` in your async tests: ```rust,no_run use pg_embedded_setup_unpriv::{TestCluster, error::BootstrapResult}; @@ -321,7 +329,7 @@ assert!(std::ptr::eq(cluster, cluster2)); # } ``` -#### When to use each fixture +**When to use each fixture:** | Fixture | Use case | | --------------------- | ------------------------------------------------- | @@ -337,8 +345,8 @@ overhead from seconds to milliseconds. `TestCluster::connection()` exposes `TestClusterConnection`, a lightweight view over the running cluster's connection metadata. Use it to read the host, port, superuser name, generated password, or the `.pgpass` path without cloning the -entire bootstrap struct. When persistence of those values beyond the guard is -required, call `metadata()` to obtain an owned `ConnectionMetadata`. +entire bootstrap struct. When you need to persist those values beyond the guard +you can call `metadata()` to obtain an owned `ConnectionMetadata`. Enable the `diesel-support` feature to call `diesel_connection()` and obtain a ready-to-use `diesel::PgConnection`. The default feature set keeps Diesel @@ -490,7 +498,7 @@ let template_name = format!("template_{}", &hash[..8]); # } ``` -If a migration version is already tracked, include it in the template name +If you already track a migration version, include it in the template name instead (for example, `format!("template_v{SCHEMA_VERSION}")`). This keeps template invalidation explicit without hashing the migration directory. @@ -504,7 +512,7 @@ The following table compares test isolation approaches: | Shared cluster, fresh database | Once | 1–5 seconds | Database | | Shared cluster, template clone | Once | 10–50 ms | Database | -#### When to use each approach +**When to use each approach:** - **Per-test cluster (`test_cluster` fixture):** Use when tests modify cluster-level settings, require specific PostgreSQL versions, or need @@ -605,7 +613,7 @@ let temp_db = cluster.temporary_database_from_template("test_db", "migrated_temp # } ``` -#### Drop behaviour +**Drop behaviour:** - `drop_database()` — Explicitly drop the database, failing if connections exist. Consumes the guard. @@ -649,19 +657,68 @@ still running as `root`, follow these steps: without interactive prompts. The `bootstrap_for_tests().environment.pgpass_file` helper returns the path if the bootstrap ran inside the test process. -- Provide `TZDIR=/usr/share/zoneinfo` (or the correct path for the target - distribution) when running the CLI. The library helper sets `TZ` +- Provide `TZDIR=/usr/share/zoneinfo` (or the correct path for your + distribution) if you are running the CLI. The library helper sets `TZ` automatically and, on Unix-like hosts, also seeds `TZDIR` when it discovers a valid timezone database. +## Continuous Integration cache hardening + +CI failures that surface as `postgresql_embedded::setup()` timeouts, or as +`error decoding response body`, are often binary-download failures rather than +PostgreSQL startup failures. The latter message can be emitted when the HTTP +client reports a stalled release-asset download through a body-decoding error. + +Harden CI in three places: + +1. Cache PostgreSQL binary archives independently from Cargo dependencies. + `pg-embed-setup-unpriv` stores release archives under + `PG_BINARY_CACHE_DIR`, then `$XDG_CACHE_HOME/pg-embedded/binaries`, then + `$HOME/.cache/pg-embedded/binaries`, and finally + `/tmp/pg-embedded/binaries`. `postgresql_embedded` also uses + `$HOME/.theseus/postgresql` for runtime installations. Cache both persistent + locations when your workflow can use both crates, but keep those paths out + of a Cargo registry or `target` cache whose key changes on every + `Cargo.lock` update. +2. Pin the release source in test bootstrap: + + ```bash + export POSTGRESQL_RELEASES_URL="https://github.com/theseus-rs/postgresql-binaries/releases" + ``` + + If a test harness sets environment variables itself, set the value only when + it is currently absent so callers can override it intentionally. +3. Pin the PostgreSQL version used by CI, preferably with an exact requirement: + + ```bash + export POSTGRESQL_VERSION="=16.10.0" + ``` + + Exact versions avoid release-list discovery during every CI run and keep the + binary cache key tied to the tested PostgreSQL version. + +Supply `GITHUB_TOKEN` in CI so GitHub release requests avoid anonymous rate +limits. GitHub Actions exposes this token by default as +`${{ secrets.GITHUB_TOKEN }}`. + +If your test runner starts several PostgreSQL-backed test binaries, serialize +the first-use bootstrap or warm the cache before running tests. `cargo-nextest` +users can assign those binaries to a test group with `max-threads = 1`, or run +the job with `NEXTEST_TEST_THREADS=1` when the suite cannot safely share a +cluster bootstrap concurrently. + ## Known issues and mitigations - **TimeZone errors**: The embedded cluster loads timezone data from the host - `tzdata` package. Install it inside the execution environment if the error - `invalid value for parameter "TimeZone": "UTC"` appears. + `tzdata` package. Install it inside the execution environment if you see + `invalid value for parameter "TimeZone": "UTC"`. - **Download rate limits**: `postgresql_embedded` fetches binaries from the - Theseus GitHub releases. Supply a `GITHUB_TOKEN` environment variable if rate - limits are encountered in CI. + Theseus GitHub releases. Supply a `GITHUB_TOKEN` environment variable if you + hit rate limits in CI. +- **Download stalls misreported as body decoding errors**: Warm and cache + PostgreSQL binaries before tests, pin `POSTGRESQL_RELEASES_URL`, and keep the + PostgreSQL binary cache independent from Cargo caches. This prevents unrelated + dependency updates from forcing cold release downloads during test execution. - **CLI arguments in tests**: `PgEnvCfg::load()` ignores `std::env::args` during library use so Cargo test filters (for example, `bootstrap_privileges::bootstrap_as_root`) do not trip the underlying Clap