Add runtime E2E fixture system (endpoint + Playwright client + FrankenPHP harness)#7
Merged
Conversation
SapphireTest creates and drops a throwaway ss_tmpdb_* database per test class, but the mysql image only grants the app user privileges on the single MYSQL_DATABASE. Add a first-init grant script so database-backed tests can create their temp schema instead of failing with access denied.
Keep development-only tooling out of the Packagist tarball via export-ignore: Docker setup, CI workflows, tests, coverage, the Taskfile, and the git dotfiles. Runtime code (src, _config), composer.json, docs, README and LICENSE remain in the release.
Add integration tests covering FixtureLoader's uncovered error paths (no SiteTree created, missing path config, missing fixture file, unknown post-action fixture) and its findPageInFactory fallback to a non-allowlisted SiteTree subclass, plus FixturePostAction's missing image source. Lifts total coverage from 82.33% to 92.11%. The remaining uncovered lines are defensive guards unreachable without mocking (record vanishing mid-write, ModuleResourceLoader returning null for an already-guarded empty resource); no production code was changed.
FixtureController::load() only caught InvalidArgumentException from FixtureLoader::load(), so structural fixture faults (RuntimeException, e.g. "did not create any SiteTree records") escaped uncaught and returned an HTML dev-error page instead of the JSON envelope the Playwright client always expects. Catch RuntimeException too and return a 500 JSON error envelope, preserving the existing 400 behavior for InvalidArgumentException.
SilverStripe 6's DevelopmentAdmin::getRegisteredRoutes() reads each controllers entry via $info['class']; the SS4/5-era 'controller' key left it undefined and threw before any /dev/ route could resolve. This only surfaced once the app was served over real HTTP (Task 7).
… and php-qa
Task 7 added ${WEB_PORT}/${DB_PORT} interpolation to compose.yml, but these two
pre-existing jobs ran $COMPOSE up without task docker-env, emitting 'variable is
not set' warnings (ports degraded to ephemeral bindings). Run task docker-env first,
matching the e2e job.
Thin BuildTask adapter over FixtureLoader exposing both load modes: --fixture=<name> seeds a single fixture, omitting it seeds every configured fixture (loadAll). Guarded to Director::isDev() because both modes reset() first (archives pages), mirroring FixtureController's route gating. InvalidArgumentException maps to INVALID, RuntimeException to FAILURE.
The healthcheck touched a sentinel file before exec'ing frankenphp, so it reported healthy on a dead server and could pass `up -d --wait` before :443 accepted connections — racing the first Playwright request in CI. Probe a real TLS handshake to :443 instead, which only succeeds once FrankenPHP has bound the socket and provisioned its internal cert. Sentinel removed from the entrypoint. Published ports bound 0.0.0.0, exposing the unauthenticated /dev/e2e-fixtures data-wipe endpoint and default-credential MySQL to the LAN on a dev host. Bind both to 127.0.0.1, enforcing the controller's documented 'never reachable except on a dev host' invariant at the network layer. Playwright and host clients still reach the app via localhost.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Adds a runtime YAML fixture system to this shared E2E support module so any SilverStripe module we E2E-test can seed and reset test data over HTTP — fixture loading/parsing is not a per-module concern. Clean-sheet implementation; no consuming module is referenced anywhere.
A dev-gated endpoint (
/dev/e2e-fixtures) loads a named YAML fixture into the DB, applies versioned post-actions, returns the created record IDs as JSON for Playwright, and resets between tests. The loader is config-driven and domain-agnostic; consumers inject behaviour via extension hooks. Both HTTP and CLI entry points are provided.Phase A — module code
WeDevelop\E2e\Fixtures\FixtureResult— immutable JSON DTO returned to Playwright.FixturePostAction—publish_recursive/unpublish/modify/attach_imageagainst anyDataObject.FixtureLoader—Extensible, config-driven (url_segment_prefix,fixture_page_classes,fixtures). FiresonBeforeLoad(string $name, FixtureFactory $factory)/onAfterLoad(FixtureResult $result)so consumers inject domain behaviour (e.g. suppressing model auto-scaffolding) with zero coupling in the engine.FixtureController— dev-gatedPOST load/POST load-all/POST reset(JSON). Intentionally unauthenticated/CSRF-free, safe only because locked to dev by two independent layers (canInit()+init()404) with an additionalconfirm=1guard on the destructivereset._config/fixtures.yml— dev-only: registers the endpoint and relaxesstrict_user_agent_check.client/playwright/index.ts— reusable Playwright client (createFixtureClientwithload/loadAll+authenticateAdmin), consumed via a tsconfig alias@wedevelop/e2e→vendor/wedevelopnl/silverstripe-e2e/client/playwright/index.ts.Phase B — E2E harness + proof
tests/E2E/withglobal.setup.ts(admin login) + a consumer-agnostic demo fixture.tests/E2E/specs/fixture.spec.tsproves the whole chain in a real browser: admin login → load via the endpoint using the TS client → CMS navigation to the created page → title assertion.e2ejob (matrix chromium + firefox).Serving over real HTTP caught a genuine latent bug the isolated PHPUnit tests structurally couldn't: the route config used the SS4/5
controller:key, but SS6DevelopmentAdminreadsclass:— fixed here.Phase C — load-all + CLI
FixtureLoader::loadAll()+POST load-all: reset once, then seed every configured fixture into its ownFixtureFactory(fail-fast on the first that raises). All fixture paths are resolved before the reset, so a misconfigured path cannot wipe E2E data or leave a partial seed. The Playwright client gains a matchingloadAll.LoadFixturesTask(sake load-e2e-fixtures): a thin CLI adapter over both loader modes —--fixture=<name>seeds one fixture, omitting it seeds every configured fixture. Dev-gated with the same posture as the controller (both modesreset(), which archives pages, so it refuses to run outside dev). Errors map cleanly:InvalidArgumentException→INVALID,RuntimeException→FAILURE. 100% line-covered, and exercised end-to-end through the realsakeCLI.Verification (all green locally)
task test55/55 ·task analyse+analyse-php85clean ·task rector-dryclean ·task coverage-check94.02% (≥90%) ·npm run typecheckpass ·task test-e2e3/3 (real admin login + 2 chromium specs; unchanged by Phase C — re-run before merge).Notes for review (deferred, non-blocking)
reset()scope: archives only fixture SiteTree pages (prefix +fixture_page_classesallowlist, refuses on empty allowlist). Bothloadandload-allreset once up front. Images created byattach_imageand non-SiteTree fixture objects are not cleaned between loads — acceptable for a throwaway E2E DB, but worth knowing..gitattributeswas added to export-ignore dev files (.docker/.github/coverage/tests/Taskfile) from the Packagist dist while keepingsrc//client//_config/.e2eCI job is validated withactionlintbut only truly exercised by this PR's CI run..docker/db-init/01-grant-tmpdb.sql) lets the app user create SapphireTest'sss_tmpdb_*databases (first DB-backed test in the suite).