e2e: fold dogfooding into the real suite, add functional mask-effect tests - #90
Merged
Conversation
…tests dogfood-viewer.mjs duplicated capture logic in a standalone script; the vitest e2e suite already drives the real viewer through the same key screens, so it's now the source of the dogfood images instead. dogfoodCapture(page, name) is a no-op unless PPS_DOGFOOD_SERVER_URL is set, in which case the suite's existing tests also stash a snapshot of each screen they already visit and afterAll batches them into one upload against a real, persistent instance -- never auto-approved. Verified live via Podman against a real backend. Also adds two tests proving masks have a real *effect* on pass/fail (not just that their CRUD UI works): a per-image mask drawn live suppresses a real regression on the same (name, viewport), and a category mask created on one snapshot suppresses a regression on a different name sharing that category. Mutation-testing applicable_masks()'s call site in render.py exposed that the shared static site server's "changed" variant, once flipped by an earlier test, never resets -- so a naive baseline capture this late in the suite was already showing the regressed color, making the tests pass regardless of whether masks did anything. Fixed with captureCleanPage(), which force-sets the .box color back to its original value client-side, mirroring how captureRegressedPage() already forces it to the regressed color. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NrQeh9CXrigN1cvpJuyzy4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
dogfood-viewer.mjsduplicated capture logic in a standalone script; removed it and folded its capture-and-upload flow intoe2e.test.tsitself, since the suite already drives the real viewer through the same key screens.dogfoodCapture(page, name)is a no-op unlessPPS_DOGFOOD_SERVER_URLis set — when it is, the suite's existing tests also stash a snapshot of each screen they already visit (run list, run detail, snapshot detail dual/single, Settings, Branches & Releases), andafterAllbatches them into one upload against a real, persistent instance. Never auto-approves.applicable_masks()'s call site inrender.pyexposed a real bug in the new tests' setup: the shared static site server's "changed" variant, once flipped by an earlier test, never resets — so a baseline captured this late in the suite via the ordinary helper was already showing the regressed color, making the new tests pass regardless of whether masks did anything. Fixed withcaptureCleanPage(), which force-sets.boxback to its original color client-side, mirroring howcaptureRegressedPage()already forces it to the regressed color.Test plan
npm run lint --workspaces— cleannpm test(client + viewer unit suites) — 91 passedbackend/.venv/bin/pytest backend/tests— 142 passednpm run test:e2e -w examples/demo-app— 16/16 passedmasks = []inbackend/app/render.pycorrectly breaks both new mask-effect tests (confirmed before and after thecaptureCleanPage()fix — before the fix it silently didn't, which is what surfaced the bug)PPS_DOGFOOD_SERVER_URLset, confirmed a real 6-snapshot run landed on the backend and the candidate image is a genuine viewer screenshot🤖 Generated with Claude Code
https://claude.ai/code/session_01NrQeh9CXrigN1cvpJuyzy4