Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 31 additions & 17 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,10 +106,10 @@ git add . && git commit -m "Update version to v0.3.2-alpha"
## Testing Strategy

**🔑 Authentication System for E2E Tests:**
GraphDone includes a comprehensive, battle-tested authentication system for E2E tests in `tests/helpers/auth.ts`. **This is the foundation for all E2E testing** and should be used by every test that requires user authentication.
GraphDone includes a comprehensive, battle-tested authentication system for E2E tests in `tests/lib/auth.ts`. **This is the foundation for all E2E testing** and should be used by every test that requires user authentication.

```typescript
import { login, navigateToWorkspace, TEST_USERS } from '../helpers/auth';
import { login, navigateToWorkspace, TEST_USERS } from '../../lib/auth';

test('my feature test', async ({ page }) => {
// Robust cross-browser authentication
Expand Down Expand Up @@ -167,15 +167,24 @@ The automated testing has revealed important UI inflexibility issues that need a

**🚀 Usage:**
```bash
# Run comprehensive tests with beautiful HTML report
./start test

# View interactive report
make test-report
# Run the unified harness — dual output: HTML (with screenshots + .webm video
# clips) AND machine-parsable JSON, from the reusable tests/lib modules.
npm run test:unified # full battery (unit + e2e + report captures + cloud audit)
npm run test:unified:smoke # fast blocking subset (unit + THE GATE + a focused e2e)
npm run test:unified:open # full battery, then open the HTML report

# Outputs land in test-artifacts/unified/: report.html + report.json (+ sequences/*.json)
make test-report # open the latest unified HTML report
# or
open test-results/reports/index.html
open test-artifacts/unified/report.html
```

The unified harness (`tests/run-unified.mjs` + `tests/sequences/unified.config.mjs`)
is the single entry; profiles are `smoke|pr|full|report`. The test tree is organised
as `tests/e2e/<domain>/`, `tests/diagnostics/<concern>/`, `tests/integration/`, with
all reusable helpers + harness modules under `tests/lib/`. Retired/legacy specs are
parked (not deleted) under `archive/2026-06-test-cleanup/` — see its MANIFEST.md.

**🔐 HTTPS/TLS Testing Setup (for next developer):**
```bash
# 1. Generate development certificates (required for TLS tests)
Expand Down Expand Up @@ -357,7 +366,7 @@ npm run dev
TEST_URL=http://localhost:3127 npm run test:smoke
```

`tests/e2e/user-smoke.spec.ts` sees the app exactly as a user does: login →
`tests/e2e/smoke/user-smoke.spec.ts` sees the app exactly as a user does: login →
nodes AND edges render → no error chrome → no GraphQL errors reach the client
→ no uncaught JS errors → the grow flow works → no orphan edges in the DB.
**Green unit tests do not mean the app works.** This gate exists because of a
Expand Down Expand Up @@ -422,16 +431,21 @@ artifacts/
└── certificates/ # Test certificates

tests/
├── e2e/ # All E2E test specs
├── helpers/ # Authentication system
└── *.js # Moved test files from root level
├── run-unified.mjs # single entry — npm run test:unified
├── sequences/ # unified.config.mjs — declarative sequences + profiles
├── e2e/<domain>/ # smoke, auth, graph, ui, a11y, mobile, responsive, api, admin, reports
├── diagnostics/<concern>/ # layout, interactions, hierarchy, inspector, physics, perf, ui
├── integration/ # infra specs (TLS, installation) — out of the default project
├── fixtures/ # shared test data
└── lib/ # reusable: auth + helpers, metrics/, audit/, reporting/, adapters/, runner/
```

**Clean Patterns:**
- Test files belong in `tests/` directory
- Screenshots go in `artifacts/screenshots/`
- No loose files at repository root
- Certificate management consolidated
- One canonical `playwright.config.ts` at the repo root; the unified harness is the single test entry.
- All reusable helpers + harness modules live under `tests/lib/`; specs import `../../lib/<x>` (depth-2 domain folders).
- Capture-heavy report-only specs live in `tests/e2e/reports/` (own Playwright projects), excluded from the fast default project.
- Retired/legacy specs are archived (not deleted) in `archive/2026-06-test-cleanup/` with a MANIFEST — mine before deleting.
- No loose files at repository root.

## Key Implementation Guidelines

Expand All @@ -454,7 +468,7 @@ export const getTypeGradientBackground = (type: WorkItemType, style: GradientSty
```

### Authentication for Tests
Use the comprehensive auth system in `tests/helpers/auth.ts` for all E2E tests requiring login.
Use the comprehensive auth system in `tests/lib/auth.ts` for all E2E tests requiring login.

## Common Gotchas

Expand Down
12 changes: 6 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,10 @@ help:

# Run all comprehensive tests
test-all: check-deploy
@echo "🧪 Running comprehensive test suite..."
@echo "🧪 Running comprehensive test suite (unified harness)..."
@npm run test:comprehensive
@echo "✅ Tests complete! Opening HTML report..."
@open test-results/reports/index.html
@open test-artifacts/unified/report.html

# Run HTTPS/SSL tests only
test-https: check-deploy
Expand All @@ -48,11 +48,11 @@ test-unit:

# Open test report
test-report:
@if [ -f test-results/reports/index.html ]; then \
open test-results/reports/index.html; \
echo "📊 Opening test report in browser..."; \
@if [ -f test-artifacts/unified/report.html ]; then \
open test-artifacts/unified/report.html; \
echo "📊 Opening unified test report in browser..."; \
else \
echo "❌ No test report found. Run 'make test-all' first."; \
echo "❌ No test report found. Run 'make test-all' (or npm run test:unified) first."; \
fi

# Start production deployment
Expand Down
2 changes: 1 addition & 1 deletion docs/SYSTEMS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

| Gate | Command | What it proves |
|------|---------|----------------|
| **THE GATE** ✅ | `TEST_URL=http://localhost:3127 npm run test:smoke` | The app works from a user's view: login → nodes **and** edges render → no GraphQL errors reach the client → no uncaught JS → grow + undo work → no orphan edges in the DB. See `tests/e2e/user-smoke.spec.ts`. |
| **THE GATE** ✅ | `TEST_URL=http://localhost:3127 npm run test:smoke` | The app works from a user's view: login → nodes **and** edges render → no GraphQL errors reach the client → no uncaught JS → grow + undo work → no orphan edges in the DB. See `tests/e2e/smoke/user-smoke.spec.ts`. |
| Unit suites | `npm run test` (turbo) or `npm run test --workspace=@graphdone/<pkg>` | core 41 · web 103 · server 23 · mcp-server ~4740 |
| Living-graph effects | `TEST_URL=http://localhost:3127 npx playwright test tests/e2e/living-graph.spec.ts` | The differentiator actually renders: breathing, glow, energy-flow, blocked-ache, hover-illumination, reduced-motion suppression. Self-seeds a fixture; gated in the smoke job. |
| Performance budgets (ADAPT-8) | `npm run perf:bundle` + `TEST_URL=… npm run test:perf` | Bundle gzip ≤ 450kB; graph settles to rest, avg tick ≤ 8ms, layout drift ≤ 25px, query p95 ≤ 800ms. Gated in CI. |
Expand Down
2 changes: 1 addition & 1 deletion docs/TESTING_AND_REFINEMENT_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
| Item | What to do | Exit criteria |
|------|-----------|---------------|
| **CI now runs the real suites** ✅ | Fixed 2026-06-13: ci.yml was disabling every Vitest suite behind a phantom "Rollup issue" and the comprehensive workflow fabricated results. Now the real core/web/server/mcp suites, the build, and THE GATE (full-stack smoke) all run and gate on every push/PR. Root cause was the `npm ci` optional-deps bug — fixed with `npm install`. | done — green on PR #37 |
| **PR-critical E2E burn-down** | The focused **smoke gate** (`test:smoke`, 3 checks) is the canonical blocking E2E and is green. The broader `test:pr` suite (~64 checks) runs as **advisory** in CI and still has known failures; migrate its specs to the `tests/helpers/api.ts` authenticated pattern (5 already done) until it can gate. | `test:pr` green, then promote to blocking |
| **PR-critical E2E burn-down** | The focused **smoke gate** (`test:smoke`, 3 checks) is the canonical blocking E2E and is green. The broader `test:pr` suite (~64 checks) runs as **advisory** in CI and still has known failures; migrate its specs to the `tests/lib/api.ts` authenticated pattern (5 already done) until it can gate. | `test:pr` green, then promote to blocking |
| ~~Chaos suite: assertion failures~~ | ✅ Fixed 2026-06-10. Full suite 4,736 pass / 1 skipped / 0 assertion failures. | done |
| Chaos worker-crash on local full run | Running the mcp chaos suites locally (CI skips them via `describe.skipIf(process.env.CI)`) ends with one tinypool "Worker exited unexpectedly" from `multi-perspective-chaos.test.ts` — a resource-exhaustion worker crashing, not an assertion failure (exit 1 only locally). Harden the scenario to not crash the worker, or cap its concurrency. | local `npm run test -w @graphdone/mcp-server` exits 0 |
| Quality tiers on real devices | The governor is unit-tested; it has never been observed on a real phone. Throttle CPU 6x + 3G in devtools, confirm tier drops and effects strip. | Manual checklist + screen recording in PR |
Expand Down
2 changes: 1 addition & 1 deletion docs/guides/web-ui-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ graph TD
## Quality gate

`tests/e2e/mobile-audit.spec.ts` + `mobile-dialogs.spec.ts` run the
`tests/helpers/mobileAudit.ts` auditors (sideways-scroll, squeezed labels,
`tests/lib/mobileAudit.ts` auditors (sideways-scroll, squeezed labels,
low-contrast/invisible text, modals clipped under the nav) across **every** screen
at phone width, in CI right after the smoke gate.

Expand Down
4 changes: 2 additions & 2 deletions docs/security/oauth-testing-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ npx playwright test tests/e2e/oauth-linkedin.spec.ts --ui
- Security best practices
- Known issues prioritization

### 2. Mock OAuth Server (`tests/helpers/mock-oauth-server.ts`)
### 2. Mock OAuth Server (`tests/lib/mock-oauth-server.ts`)

**Full-featured mock server:**
- ✅ Simulates Google OAuth 2.0
Expand Down Expand Up @@ -312,7 +312,7 @@ LINKEDIN_CALLBACK_URL=https://localhost:4128/auth/linkedin/callback
**Code:**
- OAuth strategies: `packages/server/src/auth/oauth-strategies.ts`
- OAuth routes: `packages/server/src/routes/auth.ts`
- Mock server: `tests/helpers/mock-oauth-server.ts`
- Mock server: `tests/lib/mock-oauth-server.ts`
- Test fixtures: `tests/fixtures/oauth-profiles.ts`
- LinkedIn tests: `tests/e2e/oauth-linkedin.spec.ts`

Expand Down
2 changes: 1 addition & 1 deletion docs/testing/e2e-admin-graph-creation.md
Original file line number Diff line number Diff line change
Expand Up @@ -443,7 +443,7 @@ test.describe('Graph View Modes', () => {
## Test Helper Functions

```typescript
// tests/helpers/graph-creation.ts
// tests/lib/graph-creation.ts
import { Page, expect } from '@playwright/test';

export async function navigateToTestGraph(page: Page, graphName: string) {
Expand Down
6 changes: 3 additions & 3 deletions docs/testing/local-vlm-and-scale.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ developers are never blocked by hardware they don't have.
- `.env.test.example` is committed and documents the variable **names** with
placeholder hosts (`http://<gpu-host>:<port>`). Copy it to `.env.test.local`
and fill in the rest.
- The harness auto-loads `.env.test.local` via `tests/helpers/testEnv.ts`.
- The harness auto-loads `.env.test.local` via `tests/lib/testEnv.ts`.

```bash
# .env.test.local (NOT committed)
Expand All @@ -44,7 +44,7 @@ every GPU you list.

## VLM protocol support

`tests/helpers/vlm.ts` is protocol-agnostic and auto-detects per endpoint:
`tests/lib/vlm.ts` is protocol-agnostic and auto-detects per endpoint:

| Protocol | Detected via | Request |
|----------|--------------|---------|
Expand All @@ -57,7 +57,7 @@ strict JSON verdict `{pass, score, issues[], summary}`, parsed leniently.
### Personas

Each captured screenshot is judged from several perspectives (see `PERSONAS`
in `tests/helpers/vlm.ts`):
in `tests/lib/vlm.ts`):

- **Visual defects** — overlapping/cut-off nodes, unreadable labels, broken
layout, missing edges, error chrome.
Expand Down
6 changes: 3 additions & 3 deletions docs/testing/pr-testing-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -332,7 +332,7 @@ See [docs/oauth-testing-guide.md](../security/oauth-testing-guide.md) for OAuth-

### Writing Reliable Tests

- ✅ **Use auth helpers** - Leverage `tests/helpers/auth.ts` for consistent login
- ✅ **Use auth helpers** - Leverage `tests/lib/auth.ts` for consistent login
- ✅ **Wait for elements** - Use `await expect().toBeVisible()` not `page.waitForTimeout()`
- ✅ **Test data isolation** - Create unique test data, don't rely on shared state
- ✅ **Handle flakiness** - Add retries for network-dependent tests
Expand Down Expand Up @@ -377,8 +377,8 @@ open test-results/reports/pr-report.html

**Test Code**:
- Test runner: `tests/run-pr-tests.js` (PR), `tests/run-all-tests.js` (comprehensive)
- Auth helpers: `tests/helpers/auth.ts`
- OAuth mock server: `tests/helpers/mock-oauth-server.ts`
- Auth helpers: `tests/lib/auth.ts`
- OAuth mock server: `tests/lib/mock-oauth-server.ts`
- Test fixtures: `tests/fixtures/oauth-profiles.ts`
- E2E tests: `tests/e2e/*.spec.ts`

Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
"test:unified:lib": "node --test tests/lib/",
"test:installation": "./scripts/test-installation-simple.sh",
"test:https": "node tests/integration/ssl-certificate-analysis.js && node tests/integration/mobile-https-compatibility-test.js",
"test:report": "open test-results/reports/index.html",
"test:report": "open test-artifacts/unified/report.html",
"lint": "turbo run lint",
"typecheck": "turbo run typecheck",
"clean": "turbo run clean && rm -rf node_modules",
Expand Down
2 changes: 1 addition & 1 deletion tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ tests/

## 🔑 Authentication System for E2E Tests

**Location**: `tests/helpers/auth.ts`
**Location**: `tests/lib/auth.ts`

This is the **foundational authentication system** that ALL E2E tests should use. It provides robust, cross-browser authentication with intelligent state management.

Expand Down
100 changes: 36 additions & 64 deletions tests/lib/README.md
Original file line number Diff line number Diff line change
@@ -1,64 +1,36 @@
# GraphDone Testing Helpers

This directory contains testing utilities and helpers for GraphDone E2E tests.

## 🔑 Authentication System (`auth.ts`)

**THE FOUNDATION FOR ALL E2E TESTS** - This is the robust, cross-browser authentication system that every E2E test should use.

### Quick Start

```typescript
import { login, navigateToWorkspace, TEST_USERS } from './auth';

test('my test', async ({ page }) => {
await login(page, TEST_USERS.ADMIN);
await navigateToWorkspace(page);
// Your test code here - fully authenticated!
});
```

### Why Use This System?

✅ **Cross-browser tested** (Chromium, Firefox, WebKit)
✅ **Handles all edge cases** (connection failures, timeouts, UI changes)
✅ **Smart retry logic** with exponential backoff
✅ **Skips redundant logins** for better performance
✅ **Comprehensive logging** for easy debugging
✅ **Session cleanup** for test isolation

### Available Functions

- `login(page, credentials?, options?)` - Main authentication function
- `navigateToWorkspace(page)` - Navigate to workspace with readiness verification
- `getAuthState(page)` - Get current authentication status
- `createTestGraph(page, options?)` - Create test graphs
- `logout(page)` - Clean logout with multiple strategies
- `cleanupAuth(page)` - Complete session cleanup for teardown

### Test Credentials

```typescript
TEST_USERS = {
ADMIN: { username: 'admin', password: 'graphdone', role: 'admin' },
MEMBER: { username: 'member', password: 'graphdone', role: 'member' },
VIEWER: { username: 'viewer', password: 'graphdone', role: 'viewer' },
GUEST: { username: '', password: '', role: 'guest' }
}
```

## 📚 Documentation

For complete documentation and examples, see: **[../README.md](../README.md)**

## 🚨 Important

**DO NOT create custom authentication logic in your tests.** Use this system instead to ensure:
- Consistency across all tests
- Proper error handling
- Cross-browser compatibility
- Easier maintenance

---

*This authentication system was battle-tested across all browsers and handles every edge case we've encountered. Trust it and use it!* 🔑
# `tests/lib/` — reusable test layer + unified harness

Everything reusable across the test suite lives here. Specs import from it via
`../../lib/<module>` (specs sit in depth-2 domain folders like `tests/e2e/auth/`).

## Helpers (consumed by specs)
- `auth.ts` — battle-tested `login` / `navigateToWorkspace` / `TEST_USERS` (the
foundation for every authenticated E2E spec)
- `api.ts` — GraphQL/REST helpers that authenticate the way the UI does
- `seedGraph.ts` — seed graphs/work items for a test
- `dbHealing.ts` — DB isolation + cleanup for heavy suites
- `testEnv.ts` — `.env.test.local` loader (VLM endpoints, etc.)
- `mobileAudit.ts`, `zorder.ts` — DOM auditors used by the mobile/z-order specs
- `vlm.ts` — local-VLM client for the visual-evaluation sequence
- `mock-oauth-server.ts` — OAuth mock for the admin/oauth-provider specs
- `metrics/balance_metrics.py` — OpenCV graph-balance metrics (subprocess; also
referenced by the GraphDone-Cloud live-audit via the `core/` submodule)

## Unified harness (`npm run test:unified`)
The single, reproducible entry (`tests/run-unified.mjs`) runs a profile of
sequences from `tests/sequences/unified.config.mjs` and emits DUAL output to
`test-artifacts/unified/`: `report.html` (per-sequence pass/warn/fail with embedded
`<img>` screenshots + `<video>` .webm clips) and `report.json`
(`schema: graphdone.unified-report/1`). Exit code = rollup status (CI-gateable).

- `reporting/` — `aggregate.mjs` (pure rollup, `node --test`), `html.mjs`, `json.mjs`,
and the consolidated `generate-*-report.mjs` domain generators
- `adapters/` — normalise external results into unified sequences:
`playwright.mjs` (parses PW JSON + harvests video/screenshots), `vitest.mjs`,
`cloud-audit.mjs` (ingests the sibling GraphDone-Cloud live-audit findings.json)
- `runner/runSequence.mjs` — argv-array spawn that captures exit/stdout/duration

Profiles: `smoke` (fast blocking) · `pr` (broader blocking) · `full` (everything
incl. capture-heavy report sequences + cloud audit) · `report` (captures only).

Run the lib's own unit tests with `npm run test:unified:lib` (`node --test tests/lib/`).
Loading