diff --git a/CLAUDE.md b/CLAUDE.md index 7eee9137..d094aee3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -106,10 +106,10 @@ git add . && git commit -m "Update version to v0.3.2-alpha" ## Testing Strategy **๐Ÿ”‘ Authentication System for E2E Tests:** -GraphDone includes a comprehensive, battle-tested authentication system for E2E tests in `tests/helpers/auth.ts`. **This is the foundation for all E2E testing** and should be used by every test that requires user authentication. +GraphDone includes a comprehensive, battle-tested authentication system for E2E tests in `tests/lib/auth.ts`. **This is the foundation for all E2E testing** and should be used by every test that requires user authentication. ```typescript -import { login, navigateToWorkspace, TEST_USERS } from '../helpers/auth'; +import { login, navigateToWorkspace, TEST_USERS } from '../../lib/auth'; test('my feature test', async ({ page }) => { // Robust cross-browser authentication @@ -167,15 +167,24 @@ The automated testing has revealed important UI inflexibility issues that need a **๐Ÿš€ Usage:** ```bash -# Run comprehensive tests with beautiful HTML report -./start test - -# View interactive report -make test-report +# Run the unified harness โ€” dual output: HTML (with screenshots + .webm video +# clips) AND machine-parsable JSON, from the reusable tests/lib modules. +npm run test:unified # full battery (unit + e2e + report captures + cloud audit) +npm run test:unified:smoke # fast blocking subset (unit + THE GATE + a focused e2e) +npm run test:unified:open # full battery, then open the HTML report + +# Outputs land in test-artifacts/unified/: report.html + report.json (+ sequences/*.json) +make test-report # open the latest unified HTML report # or -open test-results/reports/index.html +open test-artifacts/unified/report.html ``` +The unified harness (`tests/run-unified.mjs` + `tests/sequences/unified.config.mjs`) +is the single entry; profiles are `smoke|pr|full|report`. The test tree is organised +as `tests/e2e//`, `tests/diagnostics//`, `tests/integration/`, with +all reusable helpers + harness modules under `tests/lib/`. Retired/legacy specs are +parked (not deleted) under `archive/2026-06-test-cleanup/` โ€” see its MANIFEST.md. + **๐Ÿ” HTTPS/TLS Testing Setup (for next developer):** ```bash # 1. Generate development certificates (required for TLS tests) @@ -357,7 +366,7 @@ npm run dev TEST_URL=http://localhost:3127 npm run test:smoke ``` -`tests/e2e/user-smoke.spec.ts` sees the app exactly as a user does: login โ†’ +`tests/e2e/smoke/user-smoke.spec.ts` sees the app exactly as a user does: login โ†’ nodes AND edges render โ†’ no error chrome โ†’ no GraphQL errors reach the client โ†’ no uncaught JS errors โ†’ the grow flow works โ†’ no orphan edges in the DB. **Green unit tests do not mean the app works.** This gate exists because of a @@ -422,16 +431,21 @@ artifacts/ โ””โ”€โ”€ certificates/ # Test certificates tests/ -โ”œโ”€โ”€ e2e/ # All E2E test specs -โ”œโ”€โ”€ helpers/ # Authentication system -โ””โ”€โ”€ *.js # Moved test files from root level +โ”œโ”€โ”€ run-unified.mjs # single entry โ€” npm run test:unified +โ”œโ”€โ”€ sequences/ # unified.config.mjs โ€” declarative sequences + profiles +โ”œโ”€โ”€ e2e// # smoke, auth, graph, ui, a11y, mobile, responsive, api, admin, reports +โ”œโ”€โ”€ diagnostics// # layout, interactions, hierarchy, inspector, physics, perf, ui +โ”œโ”€โ”€ integration/ # infra specs (TLS, installation) โ€” out of the default project +โ”œโ”€โ”€ fixtures/ # shared test data +โ””โ”€โ”€ lib/ # reusable: auth + helpers, metrics/, audit/, reporting/, adapters/, runner/ ``` **Clean Patterns:** -- Test files belong in `tests/` directory -- Screenshots go in `artifacts/screenshots/` -- No loose files at repository root -- Certificate management consolidated +- One canonical `playwright.config.ts` at the repo root; the unified harness is the single test entry. +- All reusable helpers + harness modules live under `tests/lib/`; specs import `../../lib/` (depth-2 domain folders). +- Capture-heavy report-only specs live in `tests/e2e/reports/` (own Playwright projects), excluded from the fast default project. +- Retired/legacy specs are archived (not deleted) in `archive/2026-06-test-cleanup/` with a MANIFEST โ€” mine before deleting. +- No loose files at repository root. ## Key Implementation Guidelines @@ -454,7 +468,7 @@ export const getTypeGradientBackground = (type: WorkItemType, style: GradientSty ``` ### Authentication for Tests -Use the comprehensive auth system in `tests/helpers/auth.ts` for all E2E tests requiring login. +Use the comprehensive auth system in `tests/lib/auth.ts` for all E2E tests requiring login. ## Common Gotchas diff --git a/Makefile b/Makefile index 50d889cc..71dc0d98 100644 --- a/Makefile +++ b/Makefile @@ -23,10 +23,10 @@ help: # Run all comprehensive tests test-all: check-deploy - @echo "๐Ÿงช Running comprehensive test suite..." + @echo "๐Ÿงช Running comprehensive test suite (unified harness)..." @npm run test:comprehensive @echo "โœ… Tests complete! Opening HTML report..." - @open test-results/reports/index.html + @open test-artifacts/unified/report.html # Run HTTPS/SSL tests only test-https: check-deploy @@ -48,11 +48,11 @@ test-unit: # Open test report test-report: - @if [ -f test-results/reports/index.html ]; then \ - open test-results/reports/index.html; \ - echo "๐Ÿ“Š Opening test report in browser..."; \ + @if [ -f test-artifacts/unified/report.html ]; then \ + open test-artifacts/unified/report.html; \ + echo "๐Ÿ“Š Opening unified test report in browser..."; \ else \ - echo "โŒ No test report found. Run 'make test-all' first."; \ + echo "โŒ No test report found. Run 'make test-all' (or npm run test:unified) first."; \ fi # Start production deployment diff --git a/docs/SYSTEMS.md b/docs/SYSTEMS.md index 129e5774..ea1b6e8d 100644 --- a/docs/SYSTEMS.md +++ b/docs/SYSTEMS.md @@ -10,7 +10,7 @@ | Gate | Command | What it proves | |------|---------|----------------| -| **THE GATE** โœ… | `TEST_URL=http://localhost:3127 npm run test:smoke` | The app works from a user's view: login โ†’ nodes **and** edges render โ†’ no GraphQL errors reach the client โ†’ no uncaught JS โ†’ grow + undo work โ†’ no orphan edges in the DB. See `tests/e2e/user-smoke.spec.ts`. | +| **THE GATE** โœ… | `TEST_URL=http://localhost:3127 npm run test:smoke` | The app works from a user's view: login โ†’ nodes **and** edges render โ†’ no GraphQL errors reach the client โ†’ no uncaught JS โ†’ grow + undo work โ†’ no orphan edges in the DB. See `tests/e2e/smoke/user-smoke.spec.ts`. | | Unit suites | `npm run test` (turbo) or `npm run test --workspace=@graphdone/` | core 41 ยท web 103 ยท server 23 ยท mcp-server ~4740 | | Living-graph effects | `TEST_URL=http://localhost:3127 npx playwright test tests/e2e/living-graph.spec.ts` | The differentiator actually renders: breathing, glow, energy-flow, blocked-ache, hover-illumination, reduced-motion suppression. Self-seeds a fixture; gated in the smoke job. | | Performance budgets (ADAPT-8) | `npm run perf:bundle` + `TEST_URL=โ€ฆ npm run test:perf` | Bundle gzip โ‰ค 450kB; graph settles to rest, avg tick โ‰ค 8ms, layout drift โ‰ค 25px, query p95 โ‰ค 800ms. Gated in CI. | diff --git a/docs/TESTING_AND_REFINEMENT_PLAN.md b/docs/TESTING_AND_REFINEMENT_PLAN.md index 9a29f55f..df4b2bc7 100644 --- a/docs/TESTING_AND_REFINEMENT_PLAN.md +++ b/docs/TESTING_AND_REFINEMENT_PLAN.md @@ -10,7 +10,7 @@ | Item | What to do | Exit criteria | |------|-----------|---------------| | **CI now runs the real suites** โœ… | Fixed 2026-06-13: ci.yml was disabling every Vitest suite behind a phantom "Rollup issue" and the comprehensive workflow fabricated results. Now the real core/web/server/mcp suites, the build, and THE GATE (full-stack smoke) all run and gate on every push/PR. Root cause was the `npm ci` optional-deps bug โ€” fixed with `npm install`. | done โ€” green on PR #37 | -| **PR-critical E2E burn-down** | The focused **smoke gate** (`test:smoke`, 3 checks) is the canonical blocking E2E and is green. The broader `test:pr` suite (~64 checks) runs as **advisory** in CI and still has known failures; migrate its specs to the `tests/helpers/api.ts` authenticated pattern (5 already done) until it can gate. | `test:pr` green, then promote to blocking | +| **PR-critical E2E burn-down** | The focused **smoke gate** (`test:smoke`, 3 checks) is the canonical blocking E2E and is green. The broader `test:pr` suite (~64 checks) runs as **advisory** in CI and still has known failures; migrate its specs to the `tests/lib/api.ts` authenticated pattern (5 already done) until it can gate. | `test:pr` green, then promote to blocking | | ~~Chaos suite: assertion failures~~ | โœ… Fixed 2026-06-10. Full suite 4,736 pass / 1 skipped / 0 assertion failures. | done | | Chaos worker-crash on local full run | Running the mcp chaos suites locally (CI skips them via `describe.skipIf(process.env.CI)`) ends with one tinypool "Worker exited unexpectedly" from `multi-perspective-chaos.test.ts` โ€” a resource-exhaustion worker crashing, not an assertion failure (exit 1 only locally). Harden the scenario to not crash the worker, or cap its concurrency. | local `npm run test -w @graphdone/mcp-server` exits 0 | | Quality tiers on real devices | The governor is unit-tested; it has never been observed on a real phone. Throttle CPU 6x + 3G in devtools, confirm tier drops and effects strip. | Manual checklist + screen recording in PR | diff --git a/docs/guides/web-ui-architecture.md b/docs/guides/web-ui-architecture.md index fb62a455..b8fc7c2a 100644 --- a/docs/guides/web-ui-architecture.md +++ b/docs/guides/web-ui-architecture.md @@ -49,7 +49,7 @@ graph TD ## Quality gate `tests/e2e/mobile-audit.spec.ts` + `mobile-dialogs.spec.ts` run the -`tests/helpers/mobileAudit.ts` auditors (sideways-scroll, squeezed labels, +`tests/lib/mobileAudit.ts` auditors (sideways-scroll, squeezed labels, low-contrast/invisible text, modals clipped under the nav) across **every** screen at phone width, in CI right after the smoke gate. diff --git a/docs/security/oauth-testing-guide.md b/docs/security/oauth-testing-guide.md index dcee9f8a..fa55d235 100644 --- a/docs/security/oauth-testing-guide.md +++ b/docs/security/oauth-testing-guide.md @@ -46,7 +46,7 @@ npx playwright test tests/e2e/oauth-linkedin.spec.ts --ui - Security best practices - Known issues prioritization -### 2. Mock OAuth Server (`tests/helpers/mock-oauth-server.ts`) +### 2. Mock OAuth Server (`tests/lib/mock-oauth-server.ts`) **Full-featured mock server:** - โœ… Simulates Google OAuth 2.0 @@ -312,7 +312,7 @@ LINKEDIN_CALLBACK_URL=https://localhost:4128/auth/linkedin/callback **Code:** - OAuth strategies: `packages/server/src/auth/oauth-strategies.ts` - OAuth routes: `packages/server/src/routes/auth.ts` -- Mock server: `tests/helpers/mock-oauth-server.ts` +- Mock server: `tests/lib/mock-oauth-server.ts` - Test fixtures: `tests/fixtures/oauth-profiles.ts` - LinkedIn tests: `tests/e2e/oauth-linkedin.spec.ts` diff --git a/docs/testing/e2e-admin-graph-creation.md b/docs/testing/e2e-admin-graph-creation.md index be05df46..7a4e5c61 100644 --- a/docs/testing/e2e-admin-graph-creation.md +++ b/docs/testing/e2e-admin-graph-creation.md @@ -443,7 +443,7 @@ test.describe('Graph View Modes', () => { ## Test Helper Functions ```typescript -// tests/helpers/graph-creation.ts +// tests/lib/graph-creation.ts import { Page, expect } from '@playwright/test'; export async function navigateToTestGraph(page: Page, graphName: string) { diff --git a/docs/testing/local-vlm-and-scale.md b/docs/testing/local-vlm-and-scale.md index 320a5c04..7df1cc73 100644 --- a/docs/testing/local-vlm-and-scale.md +++ b/docs/testing/local-vlm-and-scale.md @@ -29,7 +29,7 @@ developers are never blocked by hardware they don't have. - `.env.test.example` is committed and documents the variable **names** with placeholder hosts (`http://:`). Copy it to `.env.test.local` and fill in the rest. -- The harness auto-loads `.env.test.local` via `tests/helpers/testEnv.ts`. +- The harness auto-loads `.env.test.local` via `tests/lib/testEnv.ts`. ```bash # .env.test.local (NOT committed) @@ -44,7 +44,7 @@ every GPU you list. ## VLM protocol support -`tests/helpers/vlm.ts` is protocol-agnostic and auto-detects per endpoint: +`tests/lib/vlm.ts` is protocol-agnostic and auto-detects per endpoint: | Protocol | Detected via | Request | |----------|--------------|---------| @@ -57,7 +57,7 @@ strict JSON verdict `{pass, score, issues[], summary}`, parsed leniently. ### Personas Each captured screenshot is judged from several perspectives (see `PERSONAS` -in `tests/helpers/vlm.ts`): +in `tests/lib/vlm.ts`): - **Visual defects** โ€” overlapping/cut-off nodes, unreadable labels, broken layout, missing edges, error chrome. diff --git a/docs/testing/pr-testing-guide.md b/docs/testing/pr-testing-guide.md index adf8f194..7accede4 100644 --- a/docs/testing/pr-testing-guide.md +++ b/docs/testing/pr-testing-guide.md @@ -332,7 +332,7 @@ See [docs/oauth-testing-guide.md](../security/oauth-testing-guide.md) for OAuth- ### Writing Reliable Tests -- โœ… **Use auth helpers** - Leverage `tests/helpers/auth.ts` for consistent login +- โœ… **Use auth helpers** - Leverage `tests/lib/auth.ts` for consistent login - โœ… **Wait for elements** - Use `await expect().toBeVisible()` not `page.waitForTimeout()` - โœ… **Test data isolation** - Create unique test data, don't rely on shared state - โœ… **Handle flakiness** - Add retries for network-dependent tests @@ -377,8 +377,8 @@ open test-results/reports/pr-report.html **Test Code**: - Test runner: `tests/run-pr-tests.js` (PR), `tests/run-all-tests.js` (comprehensive) -- Auth helpers: `tests/helpers/auth.ts` -- OAuth mock server: `tests/helpers/mock-oauth-server.ts` +- Auth helpers: `tests/lib/auth.ts` +- OAuth mock server: `tests/lib/mock-oauth-server.ts` - Test fixtures: `tests/fixtures/oauth-profiles.ts` - E2E tests: `tests/e2e/*.spec.ts` diff --git a/package.json b/package.json index 76f7eb91..bf34e42f 100644 --- a/package.json +++ b/package.json @@ -29,7 +29,7 @@ "test:unified:lib": "node --test tests/lib/", "test:installation": "./scripts/test-installation-simple.sh", "test:https": "node tests/integration/ssl-certificate-analysis.js && node tests/integration/mobile-https-compatibility-test.js", - "test:report": "open test-results/reports/index.html", + "test:report": "open test-artifacts/unified/report.html", "lint": "turbo run lint", "typecheck": "turbo run typecheck", "clean": "turbo run clean && rm -rf node_modules", diff --git a/tests/README.md b/tests/README.md index c49dbe1f..58f11b55 100644 --- a/tests/README.md +++ b/tests/README.md @@ -17,7 +17,7 @@ tests/ ## ๐Ÿ”‘ Authentication System for E2E Tests -**Location**: `tests/helpers/auth.ts` +**Location**: `tests/lib/auth.ts` This is the **foundational authentication system** that ALL E2E tests should use. It provides robust, cross-browser authentication with intelligent state management. diff --git a/tests/lib/README.md b/tests/lib/README.md index 8021c5d2..1bbabd23 100644 --- a/tests/lib/README.md +++ b/tests/lib/README.md @@ -1,64 +1,36 @@ -# GraphDone Testing Helpers - -This directory contains testing utilities and helpers for GraphDone E2E tests. - -## ๐Ÿ”‘ Authentication System (`auth.ts`) - -**THE FOUNDATION FOR ALL E2E TESTS** - This is the robust, cross-browser authentication system that every E2E test should use. - -### Quick Start - -```typescript -import { login, navigateToWorkspace, TEST_USERS } from './auth'; - -test('my test', async ({ page }) => { - await login(page, TEST_USERS.ADMIN); - await navigateToWorkspace(page); - // Your test code here - fully authenticated! -}); -``` - -### Why Use This System? - -โœ… **Cross-browser tested** (Chromium, Firefox, WebKit) -โœ… **Handles all edge cases** (connection failures, timeouts, UI changes) -โœ… **Smart retry logic** with exponential backoff -โœ… **Skips redundant logins** for better performance -โœ… **Comprehensive logging** for easy debugging -โœ… **Session cleanup** for test isolation - -### Available Functions - -- `login(page, credentials?, options?)` - Main authentication function -- `navigateToWorkspace(page)` - Navigate to workspace with readiness verification -- `getAuthState(page)` - Get current authentication status -- `createTestGraph(page, options?)` - Create test graphs -- `logout(page)` - Clean logout with multiple strategies -- `cleanupAuth(page)` - Complete session cleanup for teardown - -### Test Credentials - -```typescript -TEST_USERS = { - ADMIN: { username: 'admin', password: 'graphdone', role: 'admin' }, - MEMBER: { username: 'member', password: 'graphdone', role: 'member' }, - VIEWER: { username: 'viewer', password: 'graphdone', role: 'viewer' }, - GUEST: { username: '', password: '', role: 'guest' } -} -``` - -## ๐Ÿ“š Documentation - -For complete documentation and examples, see: **[../README.md](../README.md)** - -## ๐Ÿšจ Important - -**DO NOT create custom authentication logic in your tests.** Use this system instead to ensure: -- Consistency across all tests -- Proper error handling -- Cross-browser compatibility -- Easier maintenance - ---- - -*This authentication system was battle-tested across all browsers and handles every edge case we've encountered. Trust it and use it!* ๐Ÿ”‘ \ No newline at end of file +# `tests/lib/` โ€” reusable test layer + unified harness + +Everything reusable across the test suite lives here. Specs import from it via +`../../lib/` (specs sit in depth-2 domain folders like `tests/e2e/auth/`). + +## Helpers (consumed by specs) +- `auth.ts` โ€” battle-tested `login` / `navigateToWorkspace` / `TEST_USERS` (the + foundation for every authenticated E2E spec) +- `api.ts` โ€” GraphQL/REST helpers that authenticate the way the UI does +- `seedGraph.ts` โ€” seed graphs/work items for a test +- `dbHealing.ts` โ€” DB isolation + cleanup for heavy suites +- `testEnv.ts` โ€” `.env.test.local` loader (VLM endpoints, etc.) +- `mobileAudit.ts`, `zorder.ts` โ€” DOM auditors used by the mobile/z-order specs +- `vlm.ts` โ€” local-VLM client for the visual-evaluation sequence +- `mock-oauth-server.ts` โ€” OAuth mock for the admin/oauth-provider specs +- `metrics/balance_metrics.py` โ€” OpenCV graph-balance metrics (subprocess; also + referenced by the GraphDone-Cloud live-audit via the `core/` submodule) + +## Unified harness (`npm run test:unified`) +The single, reproducible entry (`tests/run-unified.mjs`) runs a profile of +sequences from `tests/sequences/unified.config.mjs` and emits DUAL output to +`test-artifacts/unified/`: `report.html` (per-sequence pass/warn/fail with embedded +`` screenshots + `