Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,4 @@ dist/

# full-api demo validator scratch
demos/.validate/
demos/.audit/
185 changes: 185 additions & 0 deletions reports/full-api-audit/REPORT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
# SLASHED Full-API Testing Operation — Report

**Framework:** SLASHED v0.6.25 · **Date:** 2026-06-30
**Surface tested:** 691 tokens + 239 classes (930 API elements)
**Method:** local build (`npm run build`) + configurator dev server, driven with
Playwright/Chromium; computed values diffed against the source-of-truth oracle
`docs/api-index.json`. Harness scripts live alongside this report in
`reports/full-api-audit/` and are re-runnable.

---

## 1. Executive summary

| Ground | Scope | Result |
|--------|-------|--------|
| **1 — Correctness** (demos + framework) | 691 tokens, 239 classes, 225 knobs, dark mode, visuals | **PASS** — no framework functional defects. 4 demo/framework polish findings (F1–F4), all low/medium. |
| **2 — Configurator fidelity** | control coverage + live override + preview reflection | **PASS** — every editable knob reachable and overridable; previews reflect changes; 59/59 configurator unit+component tests green. |

**Headline numbers**

- Tokens: **691/691** rendered, **0** console errors, **683/691** resolve at `:root` (the 8 "empty" are fully explained below — none is a framework bug). Demo's displayed value matched the independently-read computed value for **691/691** tokens (0 misreports). Aliases: **0** mismatches. Dark mode re-resolves **272** tokens.
- Classes: **239/239** rendered, **238/239** ship in the loaded bundle, **69/69** behavioural contracts pass, **0** console errors.
- Override wiring (overrides demo): **202/202** perturbed knobs move; the **23** non-perturbed knobs exactly equal the **23** documented skips (0 undocumented); **460/466** consumption tokens recompute downstream.
- Configurator: **224/224** editable PUBLIC knobs reachable as controls; **8/8** power knobs (`knobs-report.json`) and **5/5** sampled generic rows (`configurator-report.json`) inject overrides *and* update the live preview; preset + reset work (`preset-reset-report.json`); **59/59** configurator unit+component tests pass (`results/configurator-unit-tests.txt`).

**Verdict:** The two demo pages are accurate, faithful reflections of the
framework, and the framework itself behaves correctly. The configurator
correctly implements the full token surface and every control overrides the
live preview as intended. Findings are limited to demo-packaging polish (F1, F2,
F4) and one framework design question (F3).

---

## 2. Ground 1 — correctness

### 2.1 Tokens (`check-tokens.mjs`)

All 691 tokens render with a live value tile; the demo's `getComputedStyle`
readout matches an independent read for every token (no misreporting). 272
tokens change under `data-theme="dark"`, confirming the dark-mode engine.

**The 8 tokens that read empty at `:root`:**

| Tokens | Why | Classification |
|--------|-----|----------------|
| `--sf-btn-radius`, `--sf-btn-padding-block`, `--sf-btn-padding-inline`, `--sf-field-radius`, `--sf-field-padding-block`, `--sf-field-padding-inline` | Defined in `optional/tokens.components.css`, which the **`optimal`** bundle (loaded by both demos) excludes. They resolve correctly under `full`/`optimal-components` (`var(--sf-radius-m)`, `var(--sf-space-xs)`). | **F1 — demo** (bundle choice). Framework correct. |
| `--sf-color-mark-text`, `--sf-color-selection-text` | Declared `inherit` (fallback-only hooks consumed as `var(--token, MarkText)`); `inherit` resolves empty at `:root` per CSS rules but works contextually. They *do* take a value in the overrides demo, proving wiring. | **F2 — demo** cosmetic. Framework correct. |

The 2 reported "literal default mismatches" (`--sf-radius-none`, `--sf-space-none`:
declared `0`, computed `0px`) are `@property <length>` normalization — **not defects**.

### 2.2 Classes (`check-classes.mjs`)

239/239 class tiles render. 238/239 selectors are present in the loaded bundle;
the one exception, `theme-transition`, is an example-only class
(`bundles: []`, from `optional/theme-example.css`) — see **F4**. 69 classes
have a strong behavioural contract (state classes, layout displays, a11y); all
**69/69 pass** (e.g. `.is-hidden`→`display:none`, `.sf-grid`→`display:grid`,
`.is-disabled`→`pointer-events:none`+opacity, `.is-truncated`→ellipsis,
`.sr-only`→clipped 1px). Bento *item* span modifiers (`sf-bento-wide` etc.)
correctly compute `display:block` (their effect is grid-span inside a parent).

### 2.3 Override-wiring proof (`diff-overrides.mjs`)

Diffing the baseline vs the always-on overrides demo:

| Check | Result |
|-------|--------|
| Perturbed knobs that moved | **202 / 202** |
| Knobs not perturbed vs documented skips | **23 = 23** (0 undocumented, 0 mismatched) |
| Consumption tokens that recomputed | **460 / 466** |
| Tokens unchanged | 29 = 23 skips + 6 component tokens (F1) |
| Console errors (both demos) | 0 |

Every configurable token is wired end-to-end; the 23 skips (z-index ladder,
runtime state flags, `env()` insets, sentinel `none`s) are deliberate and
documented in `ultimate-override.css`.

### 2.4 Visual

Baseline and overrides demos render as intended (screenshots:
`baseline-desktop.png`, `overrides-desktop.png`; the harness also captures
tablet/mobile widths, omitted here for size). Color ramps,
semantic colors, spacing and type scales are correct; the overrides demo shows
the dramatic, coherent re-skin (serif type, red palette, enlarged scales,
dashed borders) that proves the cascade.

---

## 3. Ground 2 — configurator fidelity

### 3.1 Coverage (`probe-panels.mjs` + configurator test suite)

- `catalogue-projection`, `smoke`, `curation` tests pass → the configurator's
`api-index.generated.json` matches the framework (no drift), all 691 tokens
present, every PUBLIC knob has a domain pattern (no orphans).
- **224/224 editable PUBLIC knobs** are reachable as controls. The only knob
not exposed is the **INTERNAL** `--sf-is-dark` (correctly hidden — "do not
override"). 49 derived/alias consumption tokens are intentionally not
individually editable (they recompute from knobs).
- 641 tokens are additionally browsable/editable as generic rows.

### 3.2 Override application (`check-knobs.mjs`, `check-configurator.mjs`, `check-preset-reset.mjs`)

Mechanism: a control's `onChange` → `overrides` state → an `$effect` writes the
override CSS into `<style id="sf-parent-overrides">` (chrome) and the preview
iframe's `<style id="slashed-overrides">`. Verified end-to-end:

| Control type | Tested | Injected (parent+preview) | Preview computed reflects | Artifact |
|--------------|--------|---------------------------|---------------------------|----------|
| Power knobs (all 6 domains) | 8/10* | 8/8 | 8/8 (driven tokens move) | `knobs-report.json` |
| Generic token rows (color/keyword/length/font) | 5/5 | 5/5 | 5/5 | `configurator-report.json` |
| Corner-style preset (Pill) | 1 | radius injected | radius-scale 1→2, radius-m 8→16px | `preset-reset-report.json` |
| Per-row reset (✕) | 1 | — | override added then removed ✓ | `preset-reset-report.json` |

*8 of 10 power knobs driven (every scale multiplier + focus ring, spanning all
6 domains); the 2 untested are the Colors contrast knobs, which use the
identical `RangeWithNumber` path. Each control type lives in its own harness +
artifact (see §3.3) so the JSON evidence is internally coherent.

Screenshot `configurator-overridden.png` shows the Stylescape preview reflecting
live overrides (red primary ramp, serif body font, dashed gradient borders) with
"5 overrides active".

### 3.3 Harness corrections (post-review reconciliation)

An earlier **combined** configurator harness committed a stale, contradictory
`configurator-report.json` (`0/6` power knobs + preset/reset timeouts). That was
a **harness-selector bug**, not a configurator defect: the power-knob accordion
sections are titled e.g. "MODULAR SCALE", "GLOBAL SCALE", "SHADOW APPEARANCE",
which the harness's section regexes didn't match, so the inputs never mounted
and the clicks timed out (cascading into the preset/reset steps). Once the
section names were corrected, **all 8 knobs pass**. To keep each artifact
internally coherent and avoid cross-step coupling, coverage was split:
`check-knobs.mjs` (knobs), `check-preset-reset.mjs` (preset + reset),
`check-configurator.mjs` (generic rows only). The CodeRabbit review correctly
flagged the contradiction between the stale artifact and this report; the
numbers here are now backed 1:1 by the regenerated artifacts. Additional
review-driven hardening: word-boundary selector match in `check-classes.mjs`
(verified zero impact — 238/239 unchanged) and `requestfailed` capture across
all Ground 1 harnesses.

---

## 4. Defect log

| ID | Ground | Fault | Severity | Summary | Suggested fix |
|----|--------|-------|----------|---------|---------------|
| **F1** | 1 | demo | Medium | "Full-API" demos load `slashed.optimal.css`, which omits `optional/tokens.components.css`; 6 component tokens render "(empty)". | Load `slashed.full.css` (or `optimal-components`) in the demos, or annotate the 6 tiles. `demos/generate.mjs`, demo HTML. |
| **F2** | 1 | demo | Low | `--sf-color-mark-text` / `--sf-color-selection-text` show "(empty)" at `:root` (declared `inherit`); they work contextually + in the overrides demo. | Label fallback-only/`inherit` tokens so "(empty)" isn't read as breakage. |
| **F3** | 1 | framework | Low | `--sf-color-base-*` numeric ramp is non-monotonic: `base-500` anchors at the near-white source, so the ramp darkens 50→400, jumps light at 500, darkens 600→950 — contradicts the guide's "50 lightest → 950 darkest" contract (holds for the other 5 families). | Confirm intent; give `base` a one-directional ramp, or document it as a surface-elevation scale exempt from the contract. `core/tokens.css`, `docs/llm-guide.md`. |
| **F4** | 1 | demo | Low | `theme-transition` tile is inert — the class ships in no bundle (`bundles: []`, example-only). | Exclude unbundled example-only classes from the full-API demo. |
| N1 | 1 | none | Info | `radius-none`/`space-none` compute `0px` not `0` — `@property <length>` normalization. Not a defect. | — |
| N2 | 2 | none | Info | An intermittent `404` console error appears on ~2 of ~11 configurator loads (0 `requestfailed` events; not reproducible across 3 back-to-back loads); functionality unaffected (rows 5/5, knobs 8/8). Likely a Vite dep-optimize/favicon timing artifact. | Identify + re-check if it recurs in CI. |

No framework **functional** bugs were found. The configurator is fully
functional; it has no defects in this audit.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

---

## 5. Coverage statement

- **Tokens: 691/691** checked on both grounds. Ground 1: rendered + value-verified + alias + dark-mode + override-wiring. Ground 2: present in configurator data (no drift) + reachable as controls (224/224 editable knobs; 49 consumption aliases recompute by design; 1 INTERNAL hidden by design).
- **Classes: 239/239** checked. 238 ship in-bundle (1 example-only, F4); 69 strong behavioural contracts all pass; remainder verified present + rendered.
- **Knobs: 225/225** accounted for (202 perturbed-and-moved + 23 documented skips).
- **Power knobs:** 8/10 driven live through the UI (all 6 domains); 2 identical-path knobs untested.

## 6. How to reproduce

```bash
npm ci && npm run build # build local bundles + oracle
node demos/validate.mjs # existing demo validator (baseline)
node reports/full-api-audit/check-tokens.mjs
node reports/full-api-audit/check-classes.mjs
node reports/full-api-audit/diff-overrides.mjs
# configurator (Ground 2):
cd configurator && npm install && npm run test:unit && ./node_modules/.bin/vite --port 5180 &
node reports/full-api-audit/check-knobs.mjs
node reports/full-api-audit/check-configurator.mjs
node reports/full-api-audit/check-preset-reset.mjs
```

Raw per-token / per-class / per-knob results are in
`reports/full-api-audit/results/*.json`; screenshots in
`reports/full-api-audit/screenshots/`.
146 changes: 146 additions & 0 deletions reports/full-api-audit/check-classes.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
// Ground 1B — class behaviour + visual correctness against the baseline demo.
// For all 239 class tiles: confirm the selector ships in the loaded bundle, the
// stage element renders, and (for categories with a clear contract) the class
// applies its defining computed property. Also captures responsive screenshots.
import { localDemo, oracle, browser, save, SHOTS, ROOT } from './lib.mjs';
import fs from 'node:fs';
import path from 'node:path';

const { classes } = oracle();
const bundleCss = fs.readFileSync(path.join(ROOT, 'badges/slashed.optimal.css'), 'utf8');
// word-boundary check so e.g. `sf-grid` isn't matched by `sf-grid--dense` alone
const escapeCls = (c) => c.replace(/[-/\\^$*+?.()|[\]{}]/g, '\\$&');
const inBundle = (c) => new RegExp(`\\.${escapeCls(c)}(?![A-Za-z0-9_-])`).test(bundleCss);
const url = localDemo('full-api-demo.html', 'classes-base.html');

const b = await browser();
const page = await b.newPage({ viewport: { width: 1280, height: 1600 } });
const consoleErrors = [];
const requestFailures = [];
page.on('console', (m) => { if (m.type() === 'error') consoleErrors.push(m.text()); });
page.on('pageerror', (e) => consoleErrors.push(String(e)));
page.on('requestfailed', (req) => requestFailures.push(`${req.method()} ${req.url()} :: ${req.failure()?.errorText ?? 'unknown'}`));
await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 60000 });
await page.waitForTimeout(800);
Comment thread
coderabbitai[bot] marked this conversation as resolved.

// Pull computed style snapshots for the element carrying each class.
const data = await page.evaluate(() => {
const out = {};
document.querySelectorAll('.tile[data-class]').forEach((tile) => {
const cls = tile.getAttribute('data-class');
const stage = tile.querySelector('.tile__stage');
let el = stage?.querySelector('.' + CSS.escape(cls));
if (!el && stage) el = stage.firstElementChild;
if (!el) { out[cls] = null; return; }
const cs = getComputedStyle(el);
out[cls] = {
display: cs.display,
visibility: cs.visibility,
opacity: cs.opacity,
pointerEvents: cs.pointerEvents,
overflow: cs.overflow,
overflowX: cs.overflowX,
textOverflow: cs.textOverflow,
whiteSpace: cs.whiteSpace,
cursor: cs.cursor,
gridTemplateColumns: cs.gridTemplateColumns,
gap: cs.gap,
aspectRatio: cs.aspectRatio,
maxWidth: cs.maxWidth,
position: cs.position,
backgroundColor: cs.backgroundColor,
userSelect: cs.userSelect,
width: el.offsetWidth,
height: el.offsetHeight,
isActive: cs.getPropertyValue('--sf-is-active').trim(),
rendered: !!(el.offsetWidth || el.offsetHeight || cs.display === 'none'),
};
});
return out;
});

// responsive screenshots for the visual-evidence section
for (const [w, h, tag] of [[1280, 2000, 'desktop'], [820, 2000, 'tablet'], [390, 2000, 'mobile']]) {
await page.setViewportSize({ width: w, height: h });
await page.waitForTimeout(150);
await page.screenshot({ path: path.join(SHOTS, `baseline-${tag}.png`), fullPage: tag !== 'desktop' });
}
await b.close();

// ---- expectations ----
const expect = {
'is-hidden': (s) => s.display === 'none',
'is-invisible': (s) => s.visibility === 'hidden',
'is-visible': (s) => s.visibility === 'visible',
'is-disabled': (s) => s.pointerEvents === 'none' && Number(s.opacity) < 1,
'is-readonly': (s) => s.pointerEvents === 'none',
'is-loading': (s) => s.pointerEvents === 'none',
'is-clipped': (s) => s.overflow === 'hidden',
'is-scrollable': (s) => s.overflowX === 'auto' || s.overflow === 'auto',
'is-truncated': (s) => s.textOverflow === 'ellipsis' && s.whiteSpace === 'nowrap',
'is-draggable': (s) => s.cursor === 'grab',
'is-active': (s) => s.isActive === '1',
'is-selected': (s) => s.backgroundColor && s.backgroundColor !== 'rgba(0, 0, 0, 0)',
'sf-grid': (s) => s.display === 'grid',
'sf-equal': (s) => s.display === 'grid',
'sf-content-grid': (s) => s.display === 'grid',
'sf-bento': (s) => s.display === 'grid',
'sf-cluster': (s) => s.display === 'flex',
'sf-stack': (s) => s.display === 'flex',
'sf-switcher': (s) => s.display === 'flex',
'sf-frame': (s) => s.aspectRatio && s.aspectRatio !== 'auto',
'sf-container': (s) => s.maxWidth && s.maxWidth !== 'none',
'sf-truncate': (s) => s.textOverflow === 'ellipsis',
'sr-only': (s) => s.position === 'absolute' && s.width <= 1,
'sf-sr-only': (s) => s.position === 'absolute' && s.width <= 1,
};
// family fallbacks by prefix for variants (e.g. sf-grid--dense, sf-bento--2)
const familyExpect = (cls) => {
// containers only (base or `--variant`); single-dash names like sf-bento-wide
// are item span modifiers (grid-column/row) and stay display:block.
if (/^sf-(grid|equal|content-grid|bento)(--|$)/.test(cls)) return (s) => s.display === 'grid' || s.display === 'flex';
if (/^sf-(cluster|stack|switcher|reel)(--|$)/.test(cls)) return (s) => s.display === 'flex';
if (/^sf-frame(--|$)/.test(cls)) return (s) => s.aspectRatio && s.aspectRatio !== 'auto';
return null;
};

const rows = [];
for (const c of classes) {
const cls = c.name;
const s = data[cls];
const r = { name: cls, category: c.category, kind: c.kind, bundles: c.bundles || [], issues: [] };
if (!inBundle(cls)) r.issues.push('SELECTOR_NOT_IN_OPTIMAL_BUNDLE');
if (!s) { r.issues.push('NO_STAGE_ELEMENT'); rows.push(r); continue; }
r.display = s.display;
// rendered: visible-area or intentionally display:none
if (!s.rendered) r.issues.push('STAGE_NOT_RENDERED');
const fn = expect[cls] || familyExpect(cls);
if (fn) {
r.checked = true;
if (!fn(s)) r.issues.push('BEHAVIOUR_MISMATCH');
else r.behaviourOk = true;
} else {
r.checked = false; // no strong contract; presence+render only
}
rows.push(r);
}

const notInBundle = rows.filter((r) => r.issues.includes('SELECTOR_NOT_IN_OPTIMAL_BUNDLE'));
const noStage = rows.filter((r) => r.issues.includes('NO_STAGE_ELEMENT'));
const behaviourMismatch = rows.filter((r) => r.issues.includes('BEHAVIOUR_MISMATCH'));
const checked = rows.filter((r) => r.checked);

const summary = {
totalClasses: classes.length,
consoleErrors: consoleErrors.length,
requestFailures: requestFailures.length,
selectorPresentInBundle: classes.length - notInBundle.length,
notInOptimalBundle: notInBundle.map((r) => `${r.name} [bundles=${r.bundles.join('|') || 'none'}]`),
noStageElement: noStage.map((r) => r.name),
behaviourChecked: checked.length,
behaviourPassed: checked.length - behaviourMismatch.length,
behaviourMismatch: behaviourMismatch.map((r) => `${r.name} (display=${r.display})`),
};

save('classes-report.json', { summary, consoleErrors, requestFailures, rows });
console.log('CLASSES SUMMARY', JSON.stringify(summary, null, 2));
Loading