From 2c14322d35b33defd50c47a1e0f2b983e9703185 Mon Sep 17 00:00:00 2001 From: jcgueriaud1 Date: Wed, 22 Jul 2026 13:04:06 +0300 Subject: [PATCH 1/3] skills: tell agents to invoke via the Skill tool, not read SKILL.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both skills point agents at bundled reference files — vaadin-playwright-test at api-reference.md / element-mapping.md / TESTING.md, visual-verification at vaadin-playwright-test — but those load only when a skill is *invoked* via the Skill tool, not when its SKILL.md is opened by hand. Instructions that say "use / follow / load the skill" read as "go read the markdown", so an agent can end up with only SKILL.md and none of the references. Observed failure: an agent read vaadin-playwright-test's SKILL.md directly, never got api-reference.md, and unzipped the DramaFinder jar to discover the API — exactly what the skill's own "never unzip the jar" rule forbids. - vaadin-playwright-test: header guard tying the invoke-don't-read rule to why "never unzip the jar" works; description verb "load" -> "invoke (via the Skill tool)". - visual-verification: header guard; its internal pointer to vaadin-playwright-test now says "invoke ... via the Skill tool". Co-Authored-By: Claude Opus 4.8 (1M context) --- skills/vaadin-playwright-test/SKILL.md | 11 ++++++++++- skills/visual-verification/SKILL.md | 7 ++++++- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/skills/vaadin-playwright-test/SKILL.md b/skills/vaadin-playwright-test/SKILL.md index 7d5c425..61ba2a1 100644 --- a/skills/vaadin-playwright-test/SKILL.md +++ b/skills/vaadin-playwright-test/SKILL.md @@ -1,10 +1,19 @@ --- name: vaadin-playwright-test -description: Generate Playwright integration tests for Vaadin 25 views using the DramaFinder library, including element interaction, form validation, grid assertions, and navigation checks. Use whenever you are about to write, edit, or run an integration/IT test for a Vaadin view — including when the requirement comes from a GitHub issue, PR, spec, or ticket rather than the user's direct words (e.g. "implement #85", "fix the failing test", "add coverage for this view"). Also use when the user mentions DramaFinder, Playwright, `SpringPlaywrightIT`, `@PlaywrightElement`, files under `src/test/**/tests/it/`, or asks about Playwright testing in a Vaadin project. If a task in a Vaadin project involves any Playwright/IT test, load this skill before writing the test. +description: Generate Playwright integration tests for Vaadin 25 views using the DramaFinder library, including element interaction, form validation, grid assertions, and navigation checks. Use whenever you are about to write, edit, or run an integration/IT test for a Vaadin view — including when the requirement comes from a GitHub issue, PR, spec, or ticket rather than the user's direct words (e.g. "implement #85", "fix the failing test", "add coverage for this view"). Also use when the user mentions DramaFinder, Playwright, `SpringPlaywrightIT`, `@PlaywrightElement`, files under `src/test/**/tests/it/`, or asks about Playwright testing in a Vaadin project. If a task in a Vaadin project involves any Playwright/IT test, invoke this skill (via the Skill tool) before writing the test. --- # Vaadin Playwright Test Generator (DramaFinder) +> **Invoke this skill via the Skill tool — do not just read this `SKILL.md`.** +> The files this skill depends on — [api-reference.md](api-reference.md) (the +> complete DramaFinder API), [element-mapping.md](element-mapping.md), and +> [TESTING.md](TESTING.md) — are bundled beside this file and only load into +> context when the skill is *invoked*, not when you open `SKILL.md` by hand. If +> you are reading this file directly, stop and invoke the skill instead. This is +> also *why* the rule below ("never unzip the jar to discover the API") holds: +> when invoked, the API reference is already in front of you. + ## Best practices Always follow [@TESTING.md](TESTING.md) when generating tests. Key rules: diff --git a/skills/visual-verification/SKILL.md b/skills/visual-verification/SKILL.md index 8de938d..0c518d7 100644 --- a/skills/visual-verification/SKILL.md +++ b/skills/visual-verification/SKILL.md @@ -5,6 +5,9 @@ description: Visually verify an implemented use case by writing a temporary Dram # Visual Verification +> **Invoke this skill via the Skill tool — do not just read this `SKILL.md`.** +> Invoking is what loads this guidance (and anything it references) into context. + Verify what the user sees. The screenshot is the ground truth; DOM and CSS are helpers. But **do not drive the browser interactively via MCP by default** — write a temporary DramaFinder test that performs the whole flow and captures all @@ -54,7 +57,9 @@ cheap. Before running the verification test: ## Writing the temp test Write the test as a normal DramaFinder test — for how to locate Vaadin -components and assert on them, use the **vaadin-playwright-test** skill. This +components and assert on them, invoke the **vaadin-playwright-test** skill via +the Skill tool (don't just read its `SKILL.md` — that's how its bundled API +reference loads). This skill only adds the visual-capture concerns on top of it: - Extend `org.vaadin.addons.dramafinder.agent.VisualVerificationTest`, which From 33b41264db58593bf6db348fe141f68f0e5acf26 Mon Sep 17 00:00:00 2001 From: jcgueriaud1 Date: Thu, 23 Jul 2026 15:58:08 +0300 Subject: [PATCH 2/3] Split visual-verification into vaadin-playwright-screenshot skill Replace the visual-verification skill with vaadin-playwright-screenshot, focused solely on capturing screenshots via a temporary DramaFinder test (screenshot validation was generic, not DramaFinder-specific, so it's dropped rather than kept as a separate skill). - Take the acceptance criteria to screenshot as the skill argument (inline or a reference to a spec/ticket); one screenshot proves each criterion. - Portability: state the DramaFinder version that ships the agent helpers and fail clearly if absent; let the criteria argument be a reference. - Add a generated agent-api-reference.md (VisualVerificationTest, AgentReporting, AgentReport, AgentReportProvider, ComponentSnapshot) so agents look the API up without unzipping the sources jar, mirroring the element api-reference: generator tool, CI staleness guard, and SKILL/AGENTS pointers. Co-Authored-By: Claude Opus 4.8 (1M context) --- .claude-plugin/marketplace.json | 6 +- .github/workflows/agent-api-reference.yml | 84 +++++++ AGENTS.md | 5 + README.md | 14 +- .../.claude-plugin/plugin.json | 10 + skills/vaadin-playwright-screenshot/SKILL.md | 133 ++++++++++ .../agent-api-reference.md | 71 ++++++ .../.claude-plugin/plugin.json | 10 - skills/visual-verification/SKILL.md | 137 ---------- tools/generate-agent-api-reference.java | 234 ++++++++++++++++++ 10 files changed, 547 insertions(+), 157 deletions(-) create mode 100644 .github/workflows/agent-api-reference.yml create mode 100644 skills/vaadin-playwright-screenshot/.claude-plugin/plugin.json create mode 100644 skills/vaadin-playwright-screenshot/SKILL.md create mode 100644 skills/vaadin-playwright-screenshot/agent-api-reference.md delete mode 100644 skills/visual-verification/.claude-plugin/plugin.json delete mode 100644 skills/visual-verification/SKILL.md create mode 100644 tools/generate-agent-api-reference.java diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index aac1334..65eeb50 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -11,9 +11,9 @@ "description": "Generate Playwright integration tests for Vaadin views using the DramaFinder library." }, { - "name": "visual-verification", - "source": "./skills/visual-verification", - "description": "Visually verify Vaadin UI changes with a temporary DramaFinder test that batch-captures screenshots against the running app." + "name": "vaadin-playwright-screenshot", + "source": "./skills/vaadin-playwright-screenshot", + "description": "Capture screenshots of a running Vaadin app with a temporary DramaFinder test that batch-captures every key visual state in one run." } ] } diff --git a/.github/workflows/agent-api-reference.yml b/.github/workflows/agent-api-reference.yml new file mode 100644 index 0000000..cf32cb9 --- /dev/null +++ b/.github/workflows/agent-api-reference.yml @@ -0,0 +1,84 @@ +name: Agent API Reference + +# Keeps skills/vaadin-playwright-screenshot/agent-api-reference.md in lock-step +# with the org.vaadin.addons.dramafinder.agent helpers. +# +# - On push (master / issue branches): regenerate and commit the result back to +# the branch, so the reference updates itself with no manual step. +# - On pull_request: regenerate and FAIL if the committed file is stale. Push +# events from a fork have a read-only token and cannot auto-commit, so this +# verify-only job is what catches an out-of-date reference in a fork PR. +# +# The push path filter excludes agent-api-reference.md itself, so the bot's own +# commit does not re-trigger the workflow (no loop). + +on: + push: + branches: + - master + - 'issue-**' + paths: + - 'src/main/java/org/vaadin/addons/dramafinder/agent/**' + - 'tools/generate-agent-api-reference.java' + - '.github/workflows/agent-api-reference.yml' + pull_request: + branches: [ master ] + paths: + - 'src/main/java/org/vaadin/addons/dramafinder/agent/**' + - 'tools/generate-agent-api-reference.java' + - '.github/workflows/agent-api-reference.yml' + +permissions: + contents: write + +jobs: + update-agent-api-reference: + # Auto-commit path — only on push (needs write access to the branch). + if: github.event_name == 'push' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + ref: ${{ github.ref_name }} + - name: Set up JDK 21 + uses: actions/setup-java@v4 + with: + java-version: '21' + distribution: 'temurin' + - name: Set up JBang + uses: jbangdev/setup-jbang@main + - name: Regenerate agent API reference + run: jbang tools/generate-agent-api-reference.java + - name: Commit the regenerated reference if it changed + run: | + if git diff --quiet skills/vaadin-playwright-screenshot/agent-api-reference.md; then + echo "agent-api-reference.md is already up to date." + else + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git add skills/vaadin-playwright-screenshot/agent-api-reference.md + git commit -m "Regenerate agent-api-reference.md [skip ci]" + git push origin HEAD:${{ github.ref_name }} + fi + + verify-agent-api-reference: + # Verify-only path — for pull requests (incl. forks, whose token is read-only). + if: github.event_name == 'pull_request' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Set up JDK 21 + uses: actions/setup-java@v4 + with: + java-version: '21' + distribution: 'temurin' + - name: Set up JBang + uses: jbangdev/setup-jbang@main + - name: Regenerate agent API reference + run: jbang tools/generate-agent-api-reference.java + - name: Fail if the committed reference is out of date + run: | + if ! git diff --exit-code skills/vaadin-playwright-screenshot/agent-api-reference.md; then + echo "::error::agent-api-reference.md is stale. Run 'jbang tools/generate-agent-api-reference.java' and commit the result." + exit 1 + fi diff --git a/AGENTS.md b/AGENTS.md index de55816..6ab9df9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -261,5 +261,10 @@ public static ButtonElement getByText(Locator locator, String text) { ... } - `api-reference.md` is generated — never edit it by hand. After changing any element's public API, regenerate it (`jbang tools/generate-api-reference.java`) and commit the result; CI fails if it is stale. +- The `org.vaadin.addons.dramafinder.agent` helpers (`VisualVerificationTest`, + `AgentReporting`, …) have their own generated reference at + `skills/vaadin-playwright-screenshot/agent-api-reference.md` + (`tools/generate-agent-api-reference.java`; same never-edit-by-hand, CI-enforced + rule). After changing that package's public API, regenerate and commit it. - Refer to `docs/specifications/` for the prose docs on components with non-obvious behaviour (Grid, TreeGrid, VirtualList, and extension guidance). diff --git a/README.md b/README.md index aac1c74..020d661 100644 --- a/README.md +++ b/README.md @@ -11,17 +11,17 @@ This repository ships two Claude Code skills. Install them as plugins: ``` /plugin marketplace add parttio/dramafinder /plugin install vaadin-playwright-test@dramafinder -/plugin install visual-verification@dramafinder +/plugin install vaadin-playwright-screenshot@dramafinder ``` - **vaadin-playwright-test** — generates Playwright integration tests for Vaadin views using Drama Finder. -- **visual-verification** — visually verifies UI changes by writing a temporary - Drama Finder test that batch-captures screenshots against the running app, - then reviewing them. Backed by the `org.vaadin.addons.dramafinder.agent` - helpers (`VisualVerificationTest`, `AgentReporting`, `ComponentSnapshot`), which - write a screenshot, semantic component snapshot, and stack trace to - `target/agent-report/` on failure. +- **vaadin-playwright-screenshot** — captures screenshots of UI changes by + writing a temporary Drama Finder test that batch-captures every key visual + state against the running app in one run. Backed by the + `org.vaadin.addons.dramafinder.agent` helpers (`VisualVerificationTest`, + `AgentReporting`, `ComponentSnapshot`), which write a screenshot, semantic + component snapshot, and stack trace to `target/agent-report/` on failure. To get later updates, run `/plugin marketplace update dramafinder`. diff --git a/skills/vaadin-playwright-screenshot/.claude-plugin/plugin.json b/skills/vaadin-playwright-screenshot/.claude-plugin/plugin.json new file mode 100644 index 0000000..38c310d --- /dev/null +++ b/skills/vaadin-playwright-screenshot/.claude-plugin/plugin.json @@ -0,0 +1,10 @@ +{ + "name": "vaadin-playwright-screenshot", + "description": "Capture screenshots of a running Vaadin app by writing a temporary DramaFinder test that batch-captures every key visual state in one run — a cheap alternative to interactive Playwright MCP.", + "version": "0.1.0", + "author": { + "name": "jcgueriaud1" + }, + "homepage": "https://github.com/parttio/dramafinder", + "repository": "https://github.com/parttio/dramafinder" +} diff --git a/skills/vaadin-playwright-screenshot/SKILL.md b/skills/vaadin-playwright-screenshot/SKILL.md new file mode 100644 index 0000000..0d5b3d5 --- /dev/null +++ b/skills/vaadin-playwright-screenshot/SKILL.md @@ -0,0 +1,133 @@ +--- +name: vaadin-playwright-screenshot +description: Capture screenshots of a running DramaFinder/Vaadin app by writing a temporary Java/Playwright test that batch-captures every key visual state in one run. Pass the acceptance criteria to screenshot as the argument. Use after implementing UI changes when you need screenshots to review. +--- + +# Vaadin Playwright Screenshot + +> **Invoke this skill via the Skill tool — do not just read this `SKILL.md`.** +> Invoking is what loads this guidance (and anything it references) into +> context. + +To capture what the user sees, **do not drive the browser interactively via MCP +by default** — write a temporary DramaFinder test that performs the whole flow +and captures all screenshots in one batch run. One run replaces dozens of MCP +round-trips and avoids accessibility-tree dumps entirely. Then review the +captured images. + +## Input: the acceptance criteria to screenshot + +This skill takes **acceptance criteria** as its argument — the testable +statements the UI must satisfy. Each criterion tells you *what to screenshot*: +one screenshot that proves it holds. The argument may be either: + +- **Inline criteria**, e.g. "Login page shows email + password fields", + "Invalid login shows an error notification", "After login, the order list is + visible"; or +- **A reference** to where the criteria live — a spec file + (`docs/specs/checkout.md`), a ticket ID, a section heading. Read the reference + and extract the criteria before writing the test. + +If nothing is provided, ask for the criteria (or derive them from the spec / +ticket for the change under test) before writing the test — the criteria are +what the capture run must cover, one screenshot each. Map every criterion to at +least one `shot(...)`, and use the criterion text to name the shot. + +Unless a criterion specifies otherwise, use a **1920x1080** viewport. + +## Prerequisite: the DramaFinder agent helpers must be available + +This skill relies on the `org.vaadin.addons.dramafinder.agent` helpers +(`VisualVerificationTest`, `AgentReporting`), which ship inside DramaFinder +**1.1.6 and later**. They're on the classpath of any project that depends on a +recent enough DramaFinder — no per-project setup. If `VisualVerificationTest` +won't resolve, the project is on an older DramaFinder (or doesn't depend on it): +stop and say so rather than hand-rolling an equivalent base class. + +## Prerequisite: the application must be running + +The temp test connects to an **already running** application — it never boots +the app itself. Keeping the app out of the test run is what makes iterations +cheap. Before running the capture test: + +1. Check whether the app already responds on its URL. +2. If it doesn't, start it the way this project runs its app. Start it in the + background, wait for it to answer, and remember that you started it so you + can stop it afterwards. + +## The capture loop + +1. Ensure the app is running with the required state (see prerequisite above + and "Reaching the state" below). +2. Write the temp test at + `src/test/java//agent/AgentVerifyIT.java` + (fixed name, fixed `agent` sub-package — overwrite the previous one, never + commit it). Extend + `org.vaadin.addons.dramafinder.agent.VisualVerificationTest`, + which already wires in the `AgentReporting` extension. Capture one screenshot + per acceptance criterion. +3. Run only that test with the project's build tool, with quiet output: + - Maven: `mvn -q surefire:test -Dtest=AgentVerifyIT` + - Gradle: `./gradlew test --tests '*.AgentVerifyIT' --console=plain -q` + If a `scripts/agent-verify.*` wrapper exists in the project, prefer it — + it + prints only pass/fail, assertion messages, and the report path. + Point the test at a non-default host/port with + `-Ddramafinder.agent.baseUrl=http://localhost:9000` (or the + `DRAMAFINDER_BASE_URL` env var). Run with `-Dheadless=false` to watch it. +4. Read the report directory — `target/agent-report/` (Maven) or + `build/agent-report/` (Gradle): it holds each numbered screenshot + (`01-…png`, `02-…png`, …). On failure the `AgentReporting` extension also + writes `failure.txt` (assertion message + trimmed stack trace + URL), + `failure.png` (full-page screenshot), and `component-snapshot.txt` (semantic + component snapshot). Review the captured images against the acceptance + criteria. +5. Record, per criterion, whether its screenshot confirms it. Delete or + overwrite the temp test when done. + +## Writing the temp test + +Write the test as a normal DramaFinder test. For how to locate Vaadin components +and assert on them, **invoke the `vaadin-playwright-test` skill via the Skill +tool** (don't just read its `SKILL.md` — invoking is what loads its bundled API +reference). This skill only adds the visual-capture concerns on top of it: + +- Extend `org.vaadin.addons.dramafinder.agent.VisualVerificationTest` (not the + usual base class) — it connects to the already-running app, wires in the + `AgentReporting` extension, and sets a 1920x1080 viewport. Override the + viewport per test, and add 375x812 and 768x1024 passes when a criterion has + responsive requirements. +- For the exact API of `VisualVerificationTest` and the other agent helpers + (`open`, `shot`, `baseUrl`, the report/snapshot types), read the bundled + **`agent-api-reference.md`** next to this file — **do not unzip the DramaFinder + sources jar**. In a consumer project where the skill isn't checked out, fetch + it (one request) from + `https://raw.githubusercontent.com/parttio/dramafinder/master/skills/vaadin-playwright-screenshot/agent-api-reference.md`. +- `shot("name")` for each acceptance criterion and each **unique visual state** + along the way — named after the criterion (`01-login-fields`, + `02-invalid-login-error`, `03-order-list`). Screenshots are auto-numbered into + the report directory. +- Assert behavior only lightly (enough to know the flow progressed). Behaviour + is covered by the behavioral tests (pyramid layer 3) — don't duplicate those + assertions here. Screenshots that prove each criterion are the deliverable. + +## Reaching the state + +- **Deep-link with stable selectors** where possible; drive elements by + DramaFinder locators, button text, `aria-label`, or stable `name` attributes. +- If a criterion needs data or a logged-in user, script the minimal setup steps + inside the same temp test — still one batch run, never set up state + interactively via MCP. Screenshot only the states the criteria call for, not + the setup steps. + +## Steps + +All steps must be done; thoroughness over speed. + +1. Acceptance criteria in hand (provided as the argument, or gathered from the + spec/ticket) +2. App running (started by you if needed) with the required state +3. Temp test reaches the state each criterion describes and captures one + screenshot per criterion (plus any unique intermediate states) +4. Screenshots reviewed against the acceptance criteria +5. Results recorded — note, per criterion, pass/fail and any visual issues diff --git a/skills/vaadin-playwright-screenshot/agent-api-reference.md b/skills/vaadin-playwright-screenshot/agent-api-reference.md new file mode 100644 index 0000000..0114a83 --- /dev/null +++ b/skills/vaadin-playwright-screenshot/agent-api-reference.md @@ -0,0 +1,71 @@ +# DramaFinder Agent Helpers API Reference + +> **Auto-generated from source — do not edit by hand.** Regenerate with `jbang tools/generate-agent-api-reference.java`. +> DramaFinder 1.1.6-SNAPSHOT — 5 agent helper types. + +Public API of the `org.vaadin.addons.dramafinder.agent` helpers used by the **vaadin-playwright-screenshot** skill. Method one-liners come from Javadoc. + +**Do not download or unzip the DramaFinder jar to discover this API — it is all here.** + +A temp verification test extends `VisualVerificationTest`; the other types are its supporting cast (the failure-report writer, the JUnit extension, and the semantic component snapshot). + +## Types + +[VisualVerificationTest](#visualverificationtest) · [AgentReport](#agentreport) · [AgentReportProvider](#agentreportprovider) · [AgentReporting](#agentreporting) · [ComponentSnapshot](#componentsnapshot) + +## VisualVerificationTest + +Base class for temporary, agent-driven visual verification tests that run against an already running application. + +*abstract class* · **implements** AgentReportProvider + +**Methods:** + +- `String baseUrl()` — The base URL the test connects to. +- `void open(String path)` — Navigate to a path relative to the configured base URL and wait for Vaadin to finish loading. +- `void shot(String name)` — Capture a numbered screenshot into the report directory. + +## AgentReport + +Writes agent-oriented verification artifacts for a single test into a report directory (target/agent-report/ for Maven, build/agent-report/ for Gradle). + +**Constructors:** + +- `AgentReport(Page page)` — Create a report writing into the build-tool default directory. +- `AgentReport(Page page, Path directory)` — Create a report writing into the given directory. + +**Static methods:** + +- `Path defaultReportDirectory()` — Resolve the default report directory for the current build tool: build/agent-report when a Gradle build file is present, otherwise target/agent-report (Maven). + +**Methods:** + +- `void captureFailure(Throwable error)` — Write the failure bundle for a failed test: failure.txt (message + trimmed stack trace + page URL), failure.png (full-page screenshot), and component-snapshot.txt (semantic snapshot). +- `Path directory()` — The directory this report writes to. +- `Path shot(String name)` — Capture a numbered screenshot of the current page state. + +## AgentReportProvider + +Implemented by tests that want the AgentReporting extension to write a failure bundle using a specific page and report instance. + +*interface* + +**Methods:** + +- `Page agentPage()` — The page the extension should screenshot and inspect on failure. +- `AgentReport agentReport()` — The report the extension should write the failure bundle into. + +## AgentReporting + +JUnit 5 extension that, on test failure, writes an agent-oriented report to target/agent-report/ (Maven) or build/agent-report/ (Gradle): a failure.txt with the assertion message and a trimmed stack trace, a screenshot of the page at failure time, and a semantic ComponentSnapshot component snapshot. + +**implements** AfterTestExecutionCallback + +## ComponentSnapshot + +Produces a compact, human- and agent-readable summary of the Vaadin components currently present on a page. + +**Static methods:** + +- `String capture(Page page)` — Capture a compact snapshot of all meaningful Vaadin components on the page. + diff --git a/skills/visual-verification/.claude-plugin/plugin.json b/skills/visual-verification/.claude-plugin/plugin.json deleted file mode 100644 index 224e6b2..0000000 --- a/skills/visual-verification/.claude-plugin/plugin.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "name": "visual-verification", - "description": "Visually verify Vaadin UI changes by writing a temporary DramaFinder test that batch-captures screenshots against the running app, then reviewing them — a cheap alternative to interactive Playwright MCP.", - "version": "0.1.0", - "author": { - "name": "jcgueriaud1" - }, - "homepage": "https://github.com/parttio/dramafinder", - "repository": "https://github.com/parttio/dramafinder" -} diff --git a/skills/visual-verification/SKILL.md b/skills/visual-verification/SKILL.md deleted file mode 100644 index 0c518d7..0000000 --- a/skills/visual-verification/SKILL.md +++ /dev/null @@ -1,137 +0,0 @@ ---- -name: visual-verification -description: Visually verify an implemented use case by writing a temporary DramaFinder test (Java/Playwright) against the running app that batch-captures screenshots, then reviewing the screenshots. Use after implementing UI changes. Escalate to Playwright MCP only if the report is insufficient to diagnose a problem. ---- - -# Visual Verification - -> **Invoke this skill via the Skill tool — do not just read this `SKILL.md`.** -> Invoking is what loads this guidance (and anything it references) into context. - -Verify what the user sees. The screenshot is the ground truth; DOM and CSS are -helpers. But **do not drive the browser interactively via MCP by default** — -write a temporary DramaFinder test that performs the whole flow and captures all -screenshots in one batch run, then review the images. One run replaces dozens of -MCP round-trips and avoids accessibility-tree dumps entirely. - -Unless the use case specifies otherwise, use a **1920x1080** viewport. - -## Prerequisite: the application must be running - -The temp test connects to an **already running** application — it never boots -the app itself. Keeping the app out of the test run is what makes iterations -cheap. Before running the verification test: - -1. Check whether the app already responds on its URL. -2. If it doesn't, start it the way this project runs its app — the project - knows how (consult its README / `CLAUDE.md` / `AGENTS.md`). Start it in the - background, wait for it to answer, and remember that you started it so you - can stop it afterwards. - -## The loop - -1. Ensure the app is running with the required state (see prerequisite above - and "Reaching the state" below). -2. Write the temp test at - `src/test/java//agent/AgentVerifyIT.java` - (fixed name, fixed `agent` sub-package — overwrite the previous one, never - commit it). Extend `org.vaadin.addons.dramafinder.agent.VisualVerificationTest`, - which already wires in the `AgentReporting` extension. -3. Run only that test with the project's build tool, with quiet output: - - Maven: `mvn -q surefire:test -Dtest=AgentVerifyIT` - - Gradle: `./gradlew test --tests '*.AgentVerifyIT' --console=plain -q` - If a `scripts/agent-verify.*` wrapper exists in the project, prefer it — it - prints only pass/fail, assertion messages, and the report path. - Point the test at a non-default host/port with - `-Ddramafinder.agent.baseUrl=http://localhost:9000` (or the - `DRAMAFINDER_BASE_URL` env var). Run with `-Dheadless=false` to watch it. -4. Read the report directory — `target/agent-report/` (Maven) or - `build/agent-report/` (Gradle): view each numbered screenshot - (`01-…png`, `02-…png`, …) and apply the visual validation rules below. On - failure the `AgentReporting` extension also writes `failure.txt` (assertion - message + trimmed stack trace + URL), `failure.png` (full-page screenshot), - and `component-snapshot.txt` (semantic component snapshot). -5. Record results in the per-use-case checklist. Delete or overwrite the temp - test when done. - -## Writing the temp test - -Write the test as a normal DramaFinder test — for how to locate Vaadin -components and assert on them, invoke the **vaadin-playwright-test** skill via -the Skill tool (don't just read its `SKILL.md` — that's how its bundled API -reference loads). This -skill only adds the visual-capture concerns on top of it: - -- Extend `org.vaadin.addons.dramafinder.agent.VisualVerificationTest`, which - connects to the already-running app and wires in the `AgentReporting` - extension. It never boots the app or a Spring context. -- Navigate with `open("route")` (relative to the base URL) or plain Playwright - `page.navigate(...)`. The base URL defaults to `http://localhost:8080` and is - overridable via `-Ddramafinder.agent.baseUrl=...` or `DRAMAFINDER_BASE_URL`. -- Use **DramaFinder locators** for all Vaadin components — never hand-rolled - shadow-DOM selectors. -- The base class sets a 1920x1080 viewport for you; override per test, and add - 375x812 and 768x1024 passes when the use case has responsive requirements. -- `shot("name")` at every **key interaction point** and each **unique visual - state** — named descriptively (`01-login`, `02-order-list`, - `03-submit-dialog`). Screenshots are auto-numbered into the report directory. -- Assert behaviour only lightly (enough to know the flow progressed). Behaviour - is covered by the browserless tests (pyramid layer 3) — don't duplicate those - assertions here. Screenshots of unique visual states are the deliverable. - -## Reaching the state - -- **Deep-link with stable selectors** where possible; drive elements by - DramaFinder locators, button text, `aria-label`, or stable `name` attributes. -- If the screen under test needs data or a logged-in user, script the minimal - setup steps inside the same temp test — still one batch run, never set up - state interactively via MCP. Screenshot only the states under test, not the - setup steps. - -## Validating visual appearance - -Review every screenshot in the report directory against: - -1. Layout matches expectations (spacing, alignment, sizing) -2. Spacing & padding are consistent — appropriate breathing room, no cramped or - excessively spaced areas. Nested layouts (AppLayout > VerticalLayout > card) - don't double-up or collapse padding. Similar views (e.g., all admin views) - share the same content padding. -3. Typography is readable and consistent -4. Interactive elements are clearly identifiable -5. Responsive behaviour works at common breakpoints when required by the use - case (mobile, tablet, desktop) -6. Text contrast and readability - - All text clearly readable against its background (titles, labels, values, - badges) - - Colored text (warning/error values, status badges) has sufficient contrast - - Elements inheriting a different color scheme (dark sidebar vs light - content) render correctly — CSS custom properties like - `var(--vaadin-background-color)` may resolve differently per inherited - scheme - - No backgrounds swallow their content text - -## Escalation — Playwright MCP as last resort - -Use the MCP **only** when the batch run cannot answer the question: - -- a failure isn't diagnosable from `failure.txt` + screenshots + the semantic - snapshot, or -- the state is genuinely exploratory (unknown UI, need to poke around - interactively). - -When escalating, start from the evidence the report already produced (open the -failing route directly, don't replay the whole flow), scope any -`browser_snapshot` you take (`depth`, `filename`), and return to the batch loop -as soon as the cause is understood. - -## Steps (per use case) - -All steps must be done; thoroughness over speed. - -1. App running (started by you if needed) with the required state -2. Temp test navigates every route in the use case's UI/Routes section -3. Temp test performs each step of the main flow -4. Screenshots captured at key interaction points and unique visual states -5. Screenshots validated against the rules above -6. Results recorded — note any visual issues in the per-use-case checklist diff --git a/tools/generate-agent-api-reference.java b/tools/generate-agent-api-reference.java new file mode 100644 index 0000000..431b449 --- /dev/null +++ b/tools/generate-agent-api-reference.java @@ -0,0 +1,234 @@ +///usr/bin/env jbang "$0" "$@" ; exit $? +//JAVA 21 +//DEPS com.github.javaparser:javaparser-core:3.26.4 + +// Generates skills/vaadin-playwright-screenshot/agent-api-reference.md — the +// public API of the DramaFinder "agent" helpers used by the +// vaadin-playwright-screenshot skill (VisualVerificationTest, AgentReporting, +// AgentReport, AgentReportProvider, ComponentSnapshot) — straight from source, +// so it can never drift from the released API. +// +// Run locally: jbang tools/generate-agent-api-reference.java +// In CI: see .github/workflows/agent-api-reference.yml +// +// Like the element api-reference generator, it lives in / writes into the SKILL +// folder on purpose: the skill is the self-contained artifact that travels to +// consumer projects, so the reference must ship inside it — that is what lets an +// agent look the API up WITHOUT unzipping the DramaFinder sources jar. + +import com.github.javaparser.StaticJavaParser; +import com.github.javaparser.ParserConfiguration.LanguageLevel; +import com.github.javaparser.ast.CompilationUnit; +import com.github.javaparser.ast.Modifier; +import com.github.javaparser.ast.body.*; +import com.github.javaparser.ast.nodeTypes.NodeWithJavadoc; +import com.github.javaparser.javadoc.Javadoc; + +import java.io.IOException; +import java.nio.file.*; +import java.util.*; +import java.util.stream.*; + +// Package-private on purpose: JBang compiles this under its hyphenated file name, +// and javac only allows a *public* top-level class to differ from the file name. +class GenerateAgentApiReference { + + static final Path REPO = Paths.get(System.getProperty("repo.dir", ".")); + static final Path AGENT_DIR = REPO.resolve("src/main/java/org/vaadin/addons/dramafinder/agent"); + static final Path OUT = REPO.resolve("skills/vaadin-playwright-screenshot/agent-api-reference.md"); + + // VisualVerificationTest is the entry point (what a temp test extends), so it + // leads; the rest follow alphabetically. + static final List ORDER = List.of("VisualVerificationTest"); + + public static void main(String[] args) throws IOException { + StaticJavaParser.getParserConfiguration().setLanguageLevel(LanguageLevel.JAVA_21); + + Map> types = new TreeMap<>(); + List files; + try (Stream s = Files.walk(AGENT_DIR)) { + files = s.filter(p -> p.toString().endsWith(".java")).sorted().collect(Collectors.toList()); + } + for (Path f : files) { + CompilationUnit cu = StaticJavaParser.parse(f); + for (TypeDeclaration t : cu.getTypes()) { + if (t.isPublic()) types.put(t.getNameAsString(), t); + } + } + + // Entry point(s) first, then the remainder alphabetically. + List names = new ArrayList<>(ORDER); + types.keySet().stream().filter(n -> !ORDER.contains(n)).forEach(names::add); + + StringBuilder md = new StringBuilder(); + String version = readVersion(); + + md.append("# DramaFinder Agent Helpers API Reference\n\n"); + md.append("> **Auto-generated from source — do not edit by hand.** "); + md.append("Regenerate with `jbang tools/generate-agent-api-reference.java`.\n"); + md.append("> DramaFinder ").append(version).append(" — ") + .append(types.size()).append(" agent helper types.\n\n"); + md.append("Public API of the `org.vaadin.addons.dramafinder.agent` helpers used by the "); + md.append("**vaadin-playwright-screenshot** skill. Method one-liners come from Javadoc.\n\n"); + md.append("**Do not download or unzip the DramaFinder jar to discover this API — it is all here.**\n\n"); + md.append("A temp verification test extends `VisualVerificationTest`; the other types are "); + md.append("its supporting cast (the failure-report writer, the JUnit extension, and the "); + md.append("semantic component snapshot).\n\n"); + + md.append("## Types\n\n"); + md.append(names.stream() + .map(n -> "[" + n + "](#" + anchor(n) + ")") + .collect(Collectors.joining(" · "))); + md.append("\n\n"); + + for (String n : names) { + renderType(md, types.get(n)); + } + + Files.createDirectories(OUT.getParent()); + Files.writeString(OUT, md.toString()); + System.out.println("Wrote " + OUT + " (" + types.size() + " types)"); + } + + /** Render one type: heading, javadoc, hierarchy, constants, constructors, methods. */ + static void renderType(StringBuilder md, TypeDeclaration t) { + String name = t.getNameAsString(); + md.append("## ").append(name).append("\n\n"); + + firstSentence(t).ifPresent(s -> md.append(s).append("\n\n")); + + List ext = extendedNames(t); + List impl = implementedNames(t); + boolean isIface = t instanceof ClassOrInterfaceDeclaration ci && ci.isInterface(); + // One hierarchy line: kind + extends/implements, only what applies. + List bits = new ArrayList<>(); + if (isIface) bits.add("*interface*"); + else if (t instanceof ClassOrInterfaceDeclaration cid && cid.isAbstract()) bits.add("*abstract class*"); + if (!ext.isEmpty()) bits.add("**extends** " + String.join(", ", ext)); + if (!impl.isEmpty()) bits.add("**implements** " + String.join(", ", impl)); + if (!bits.isEmpty()) md.append(String.join(" · ", bits)).append("\n\n"); + + // Public constants. + List constants = t.getFields().stream() + .filter(f -> f.isPublic() && f.isStatic() && f.isFinal()) + .flatMap(f -> f.getVariables().stream() + .map(v -> "`" + f.getElementType().asString() + " " + v.getNameAsString() + "`")) + .collect(Collectors.toList()); + if (!constants.isEmpty()) { + md.append("**Constants:** ").append(String.join(", ", constants)).append("\n\n"); + } + + // Public constructors. + List ctors = t.getConstructors().stream() + .filter(c -> c.getModifiers().contains(Modifier.publicModifier())) + .collect(Collectors.toList()); + if (!ctors.isEmpty()) { + md.append("**Constructors:**\n\n"); + for (ConstructorDeclaration c : ctors) { + md.append("- `").append(ctorSig(c)).append("`"); + firstSentence(c).ifPresent(s -> md.append(" — ").append(s)); + md.append("\n"); + } + md.append("\n"); + } + + // Public AND protected methods: an abstract base class exposes its + // interaction API (open/shot/baseUrl on VisualVerificationTest) as + // protected members that subclasses call. Interface members are + // implicitly public. Drop @Override noise (documented on the supertype). + List methods = t.getMethods().stream() + .filter(m -> m.isPublic() || m.isProtected() || (isIface && !m.isPrivate() && !m.isStatic())) + .filter(m -> !isOverride(m)) + .sorted(Comparator.comparing(MethodDeclaration::getNameAsString)) + .collect(Collectors.toList()); + List statics = methods.stream().filter(MethodDeclaration::isStatic).toList(); + List instance = methods.stream().filter(m -> !m.isStatic()).toList(); + + if (!statics.isEmpty()) { + md.append("**Static methods:**\n\n"); + statics.forEach(m -> appendMethod(md, m)); + md.append("\n"); + } + if (!instance.isEmpty()) { + md.append("**Methods:**\n\n"); + instance.forEach(m -> appendMethod(md, m)); + md.append("\n"); + } + } + + static void appendMethod(StringBuilder md, MethodDeclaration m) { + md.append("- `").append(methodSig(m)).append("`"); + firstSentence(m).ifPresent(s -> md.append(" — ").append(s)); + md.append("\n"); + } + + // ---- signature helpers ------------------------------------------------- + + static String methodSig(MethodDeclaration m) { + return m.getType().asString() + " " + m.getNameAsString() + "(" + params(m.getParameters()) + ")"; + } + + static String ctorSig(ConstructorDeclaration c) { + return c.getNameAsString() + "(" + params(c.getParameters()) + ")"; + } + + static String params(List ps) { + return ps.stream() + .map(p -> p.getType().asString() + (p.isVarArgs() ? "..." : "") + " " + p.getNameAsString()) + .collect(Collectors.joining(", ")); + } + + // ---- hierarchy helpers ------------------------------------------------- + + static List extendedNames(TypeDeclaration t) { + if (t instanceof ClassOrInterfaceDeclaration cid) { + return cid.getExtendedTypes().stream().map(x -> x.getNameAsString()).collect(Collectors.toList()); + } + return List.of(); + } + + static List implementedNames(TypeDeclaration t) { + if (t instanceof ClassOrInterfaceDeclaration cid && !cid.isInterface()) { + return cid.getImplementedTypes().stream().map(x -> x.getNameAsString()).collect(Collectors.toList()); + } + return List.of(); + } + + static boolean isOverride(MethodDeclaration m) { + return m.getAnnotationByName("Override").isPresent(); + } + + // ---- javadoc ----------------------------------------------------------- + + static Optional firstSentence(NodeWithJavadoc n) { + Optional jd = n.getJavadoc(); + if (jd.isEmpty()) return Optional.empty(); + String text = jd.get().getDescription().toText(); + if (text == null || text.isBlank()) return Optional.empty(); + text = text.replaceAll("\\s+", " ").trim(); + text = text.replaceAll("\\{@\\w+\\s+([^}]+)}", "$1"); + int from = 0; + while (true) { + int dot = text.indexOf(". ", from); + if (dot < 0) break; + String before = text.substring(0, dot); + if (before.endsWith("e.g") || before.endsWith("i.e")) { from = dot + 2; continue; } + text = text.substring(0, dot + 1); + break; + } + return Optional.of(text.trim()); + } + + static String anchor(String name) { + return name.toLowerCase(Locale.ROOT).replaceAll("[^a-z0-9]+", ""); + } + + static String readVersion() { + try { + String pom = Files.readString(REPO.resolve("pom.xml")); + var m = java.util.regex.Pattern.compile("([^<]+)").matcher(pom); + if (m.find()) return m.group(1); + } catch (IOException ignored) {} + return ""; + } +} From 1af10530b2729d8ef5481d8c19a0d667f35eefe4 Mon Sep 17 00:00:00 2001 From: jcgueriaud1 Date: Thu, 23 Jul 2026 16:08:15 +0300 Subject: [PATCH 3/3] Remove tessl github actions --- .github/workflows/tessl-publish.yml | 14 -------------- .github/workflows/tessl-review.yml | 15 --------------- 2 files changed, 29 deletions(-) delete mode 100644 .github/workflows/tessl-publish.yml delete mode 100644 .github/workflows/tessl-review.yml diff --git a/.github/workflows/tessl-publish.yml b/.github/workflows/tessl-publish.yml deleted file mode 100644 index 5221375..0000000 --- a/.github/workflows/tessl-publish.yml +++ /dev/null @@ -1,14 +0,0 @@ -name: Tessl Publish -on: - push: - branches: [ master ] - -jobs: - publish: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v6 - - uses: tesslio/setup-tessl@v2 - with: - token: ${{ secrets.TESSL_TOKEN }} - - run: tessl tile publish \ No newline at end of file diff --git a/.github/workflows/tessl-review.yml b/.github/workflows/tessl-review.yml deleted file mode 100644 index 70c66e8..0000000 --- a/.github/workflows/tessl-review.yml +++ /dev/null @@ -1,15 +0,0 @@ -name: Tessl Review - -on: - pull_request: - branches: [ master ] - -jobs: - review: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v6 - - uses: tesslio/setup-tessl@v2 - with: - token: ${{ secrets.TESSL_TOKEN }} - - run: tessl skill review --threshold 80 skills/vaadin-playwright-test/SKILL.md \ No newline at end of file