Three things we hit while getting the Playwright suite running against a vendored copy of
deploy/dev. Each one reads as an app failure when it isn't, so they cost more time than
their size suggests.
1. The suite can silently run against the wrong server.
deploy/dev/tests/playwright.config.js declares:
webServer: {
command: `python3 -m http.server 8080 --directory "${DEPLOY_ROOT}"`,
port: 8080,
reuseExistingServer: true,
}
Port 8080 is heavily contended on developer machines — Colima and Docker Desktop forward
it, and it's the conventional port for an SSH tunnel to a demo box. With
reuseExistingServer: true, Playwright finds the port bound, skips starting its own
server, and runs the entire suite against whatever is already there.
It doesn't fail as a bind error or a wrong-target error, which would be obvious. Every
spec fails with TimeoutError: page.waitForSelector: Timeout 30000ms exceeded on
#canvas, because the foreign server 404s on /dev/index.html. It reads as a broken
viewer.
Observed here: Colima had 8080 forwarded, and 01-viewer-load.spec.js reported 0/9.
Moving to a dedicated port with reuseExistingServer: false and changing nothing else
took the same spec to 6/9. Six of those nine failures were never real.
Any of these would fix it: a less contended default port; reuseExistingServer: false; or
a beforeAll assertion that GET /dev/index.html returns 200, so a wrong target fails
loudly in milliseconds.
2. Test fixtures are symlinks pointing outside the deployable tree.
deploy/dev/tests/fixtures/test.ifc and test_revised.ifc symlink to
../../../../reference/residential/Ifc4_SampleHouse{,_Revised}.ifc — outside deploy/dev/.
Anyone who deploys, packages or vendors deploy/dev/ as the unit GH_DEPLOY.md designates
as the source of truth gets two dangling symlinks and can't run the specs that use them.
Vendoring the two files into fixtures/, or resolving them at test setup, would make
deploy/dev/ self-contained.
3. Several specs target UI controls that no longer exist, so they can never pass.
Because page.click() waits for an actionable element, each of these burns the full 60 s
test timeout and reports as a timeout rather than "selector not found" — which reads as a
slow or broken app rather than a stale test.
| Selector |
Spec |
Result at 55900e2d |
#theme-btn |
01-viewer-load 1.7 |
No such element anywhere in deploy/dev; the only "theme" in index.html is <meta name="theme-color"> |
#fly-btn |
01-viewer-load 1.9 |
Referenced from JS, never rendered. tour.js:9 says so: // §S280: may be null (pill removed button) |
#disc-body |
02-panels |
7 of 7 tests fail |
#find-axis-disc |
40-find-isolate |
4 of 4 tests fail |
#import-drop-zone |
37-s252-revit-colour |
1 of 1 test fails |
That's at least 14 failures across three fully-red spec files plus two more tests. The
§S280 comment suggests the fly control was deliberately replaced by a pill UI and the
spec wasn't updated; the others look like the same pattern.
Two cheap suggestions: assert selector existence up front, so these fail in milliseconds
with a clear message instead of timing out; and treat a fully-red spec file as a signal
that the spec, not the app, may have drifted.
(For completeness — 41-room-volume-lens also references three missing #find-axis-*
selectors yet passes 3/3, so those must sit in a tolerant path. A missing selector doesn't
by itself imply a failure.)
Three things we hit while getting the Playwright suite running against a vendored copy of
deploy/dev. Each one reads as an app failure when it isn't, so they cost more time thantheir size suggests.
1. The suite can silently run against the wrong server.
deploy/dev/tests/playwright.config.jsdeclares:Port 8080 is heavily contended on developer machines — Colima and Docker Desktop forward
it, and it's the conventional port for an SSH tunnel to a demo box. With
reuseExistingServer: true, Playwright finds the port bound, skips starting its ownserver, and runs the entire suite against whatever is already there.
It doesn't fail as a bind error or a wrong-target error, which would be obvious. Every
spec fails with
TimeoutError: page.waitForSelector: Timeout 30000ms exceededon#canvas, because the foreign server 404s on/dev/index.html. It reads as a brokenviewer.
Observed here: Colima had 8080 forwarded, and
01-viewer-load.spec.jsreported 0/9.Moving to a dedicated port with
reuseExistingServer: falseand changing nothing elsetook the same spec to 6/9. Six of those nine failures were never real.
Any of these would fix it: a less contended default port;
reuseExistingServer: false; ora
beforeAllassertion thatGET /dev/index.htmlreturns 200, so a wrong target failsloudly in milliseconds.
2. Test fixtures are symlinks pointing outside the deployable tree.
deploy/dev/tests/fixtures/test.ifcandtest_revised.ifcsymlink to../../../../reference/residential/Ifc4_SampleHouse{,_Revised}.ifc— outsidedeploy/dev/.Anyone who deploys, packages or vendors
deploy/dev/as the unitGH_DEPLOY.mddesignatesas the source of truth gets two dangling symlinks and can't run the specs that use them.
Vendoring the two files into
fixtures/, or resolving them at test setup, would makedeploy/dev/self-contained.3. Several specs target UI controls that no longer exist, so they can never pass.
Because
page.click()waits for an actionable element, each of these burns the full 60 stest timeout and reports as a timeout rather than "selector not found" — which reads as a
slow or broken app rather than a stale test.
55900e2d#theme-btn01-viewer-load1.7deploy/dev; the only "theme" inindex.htmlis<meta name="theme-color">#fly-btn01-viewer-load1.9tour.js:9says so:// §S280: may be null (pill removed button)#disc-body02-panels#find-axis-disc40-find-isolate#import-drop-zone37-s252-revit-colourThat's at least 14 failures across three fully-red spec files plus two more tests. The
§S280comment suggests the fly control was deliberately replaced by a pill UI and thespec wasn't updated; the others look like the same pattern.
Two cheap suggestions: assert selector existence up front, so these fail in milliseconds
with a clear message instead of timing out; and treat a fully-red spec file as a signal
that the spec, not the app, may have drifted.
(For completeness —
41-room-volume-lensalso references three missing#find-axis-*selectors yet passes 3/3, so those must sit in a tolerant path. A missing selector doesn't
by itself imply a failure.)