Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions bench/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
results/
56 changes: 56 additions & 0 deletions bench/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Benchmarks

Reproducible performance scenarios for the editor core. They exist to answer
one question per scenario with numbers that are comparable between runs on the
same machine — not to produce absolute rankings across machines.

## Running

```bash
pnpm build # benchmarks import packages/core/dist, so build first
pnpm bench
```

Output: a table on stdout plus `bench/results/latest.json` (git-ignored) for
diffing two runs.

## Scenarios

| Scenario | Question it answers |
|---|---|
| `live-preview/cursor-move` | How expensive is one cursor move (arrow key, click) as the document grows? This is the per-keystroke path: it must not grow linearly with document size. |
| `live-preview/set-document` | What does a full document replace cost on a large document (cache-cold rebuild)? |
| `live-preview/create-editor` | What does mounting a fresh editor on a ~23KB document cost once modules are warm? |
| `api/export-html` | What does one `exportHTML()` call cost? |
| `api/get-ast` / `api/get-document-stats` | Cost of the read-only public APIs. |

Documents are generated deterministically (`bench/scenarios/documents.mjs`):
repeating sections with headings, inline formatting, links, lists, task lists
and a fenced code block every fifth section — the constructs the live preview
decorates.

## Method

- Each scenario runs 5 untimed warm-up iterations, then 30 timed ones.
- Reported: median and p95. Medians are the comparison metric; p95 catches tails.
- Everything runs in-process in jsdom, so absolute numbers are not comparable
with a real browser. Ratios between scenarios on the same run are.
- No new dependencies: the harness is plain Node + the already-present jsdom.

## Regression budgets

`packages/core/test/live-preview-cursor-budget.test.ts` encodes the invariant
that mattered historically: **cursor-move latency must not scale linearly with
document size**. It asserts the ratio between cursor-move latency on a ~96KB
document and a ~4KB document stays below a fixed threshold. Ratios are used
instead of absolute milliseconds so the gate survives slower CI runners; the
threshold sits between the pre-optimization ratio (~5.2x) and the current one
(~2.5x).

## Adding a scenario

1. Create `bench/scenarios/<name>.mjs` exporting `run(measure): Row[]`.
2. Register the file in `bench/run.mjs` (`scenarioFiles`).
3. Keep scenarios black-box: drive the public API (`createEditor` and the
`EditorAPI` methods), never internals — otherwise the benchmark stops
measuring what consumers pay.
99 changes: 99 additions & 0 deletions bench/harness.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
/**
* Minimal zero-dependency benchmark harness for Nexus-Editor.
*
* Runs against the built packages (`pnpm build` first), drives a real editor
* in jsdom, and reports median / p95 over a fixed sample count so numbers are
* comparable between runs on the same machine.
*
* @typedef {Object} Sample
* @property {number} median
* @property {number} p95
* @property {number} min
* @property {number} max
* @property {number} samples
*
* @typedef {Object} Row
* @property {string} scenario
* @property {string} subject
* @property {string} metric
* @property {number} value
* @property {string} unit
*/

import { JSDOM } from "jsdom";

/**
* Install the DOM globals CodeMirror needs, once per process. The globals
* live for the whole run because the editor holds them.
*/
export function installDom() {
const dom = new JSDOM("<!doctype html><html><body></body></html>", {
pretendToBeVisual: true,
});
globalThis.window = dom.window;
globalThis.document = dom.window.document;
Object.defineProperty(globalThis, "navigator", {
value: dom.window.navigator,
configurable: true,
});
globalThis.Range = dom.window.Range;
globalThis.MutationObserver = dom.window.MutationObserver;
globalThis.Element = dom.window.Element;
globalThis.HTMLElement = dom.window.HTMLElement;
globalThis.SVGElement = dom.window.SVGElement;
globalThis.CustomEvent = dom.window.CustomEvent;
globalThis.requestAnimationFrame = (cb) => setTimeout(() => cb(performance.now()), 0);
}

function percentile(sorted, p) {
return sorted[Math.min(sorted.length - 1, Math.floor(sorted.length * p))];
}

/**
* Run `fn` `warmup` times untouched, then `samples` timed runs. Returns
* median / p95 / min / max in milliseconds.
*
* @param {() => void} fn
* @param {{ warmup?: number, samples?: number }} [options]
* @returns {Sample}
*/
export function measure(fn, options) {
const warmup = options?.warmup ?? 5;
const samples = options?.samples ?? 30;
for (let i = 0; i < warmup; i++) fn();
const times = [];
for (let i = 0; i < samples; i++) {
const start = performance.now();
fn();
times.push(performance.now() - start);
}
times.sort((a, b) => a - b);
return {
median: percentile(times, 0.5),
p95: percentile(times, 0.95),
min: times[0],
max: times[times.length - 1],
samples,
};
}

/**
* @param {Row[]} rows
*/
export function printTable(rows) {
const header = ["scenario", "subject", "metric", "median", "unit"];
const body = rows.map((row) => [
row.scenario,
row.subject,
row.metric,
row.value.toFixed(2),
row.unit,
]);
const widths = header.map((column, i) =>
Math.max(column.length, ...body.map((cells) => cells[i].length))
);
const line = (cells) => cells.map((cell, i) => cell.padEnd(widths[i])).join(" | ");
console.log(line(header));
console.log(widths.map((width) => "-".repeat(width)).join("-|-"));
for (const cells of body) console.log(line(cells));
}
44 changes: 44 additions & 0 deletions bench/run.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
#!/usr/bin/env node
/**
* Benchmark entry point. Usage:
*
* pnpm build # benchmarks run against the built packages
* pnpm bench # runs every scenario, prints a table, writes JSON
*
* Results are written to bench/results/latest.json so a run can be diffed
* against a previous one on the same machine.
*/
import fs from "node:fs";
import path from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";

import { installDom, measure, printTable } from "./harness.mjs";

const here = path.dirname(fileURLToPath(import.meta.url));
const coreDist = path.join(here, "..", "packages", "core", "dist", "index.js");

if (!fs.existsSync(coreDist)) {
console.error("packages/core/dist/index.js not found — run `pnpm build` before `pnpm bench`.");
process.exit(1);
}

installDom();

const scenarioFiles = ["core-editor.mjs", "public-api.mjs"];
const rows = [];
for (const file of scenarioFiles) {
const { run } = await import(pathToFileURL(path.join(here, "scenarios", file)).href);
rows.push(...run(measure));
}

printTable(rows);

const outDir = path.join(here, "results");
fs.mkdirSync(outDir, { recursive: true });
const payload = {
generatedAt: new Date().toISOString(),
node: process.version,
rows,
};
fs.writeFileSync(path.join(outDir, "latest.json"), `${JSON.stringify(payload, null, 2)}\n`);
console.log(`\nwritten to bench/results/latest.json`);
88 changes: 88 additions & 0 deletions bench/scenarios/core-editor.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
/**
* Core-editor scenarios: how expensive is one cursor move / one document
* replace as the document grows, and what does the first render cost.
*/
import { createEditor } from "../../packages/core/dist/index.js";
import { buildDocument } from "./documents.mjs";

function mount(initialValue) {
const container = document.createElement("div");
document.body.appendChild(container);
const editor = createEditor({ container, initialValue, livePreview: true });
return { editor, container };
}

export function run(measure) {
const rows = [];
const sizes = [
{ blocks: 20, label: "~4KB" },
{ blocks: 100, label: "~23KB" },
{ blocks: 400, label: "~96KB" },
];

for (const size of sizes) {
const doc = buildDocument(size.blocks);
const { editor, container } = mount(doc);
const step = Math.max(1, Math.floor(doc.length / 30));
const targets = [];
for (let pos = 0; pos < doc.length; pos += step) targets.push(pos);
let index = 0;
const sample = measure(() => {
editor.setSelection(targets[index % targets.length]);
index += 1;
});
rows.push({
scenario: "live-preview/cursor-move",
subject: `doc ${size.label} (${size.blocks} sections)`,
metric: "median",
value: sample.median,
unit: "ms",
});
rows.push({
scenario: "live-preview/cursor-move",
subject: `doc ${size.label} (${size.blocks} sections)`,
metric: "p95",
value: sample.p95,
unit: "ms",
});
editor.destroy();
container.remove();
}

// Full document replace on the largest document (cache-cold rebuild path).
{
const doc = buildDocument(400);
const { editor, container } = mount(doc);
const sample = measure(() => {
editor.setDocument(doc);
});
rows.push({
scenario: "live-preview/set-document",
subject: "doc ~96KB",
metric: "median",
value: sample.median,
unit: "ms",
});
editor.destroy();
container.remove();
}

// First render of a fresh editor (module warm-up already done by the runs above).
{
const doc = buildDocument(100);
const sample = measure(() => {
const { editor, container } = mount(doc);
editor.destroy();
container.remove();
});
rows.push({
scenario: "live-preview/create-editor",
subject: "doc ~23KB",
metric: "median",
value: sample.median,
unit: "ms",
});
}

return rows;
}
32 changes: 32 additions & 0 deletions bench/scenarios/documents.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
/**
* Deterministic document fixtures shared by the benchmark scenarios: a
* realistic mix of headings, inline formatting, links, code blocks, lists and
* task lists, scaled by repeating sections.
*/

export function section(index) {
return [
`## Section ${index}`,
"",
`Some **bold ${index}** text with a [link ${index}](https://example.com/${index}) and \`inline code ${index}\`.`,
"",
`- item ${index}a`,
`- [ ] task ${index}b`,
"",
].join("\n");
}

export function codeBlock(index) {
const lines = Array.from({ length: 20 }, (_, line) => `const value${line}_${index} = ${index} * ${line};`);
return "```js\n" + lines.join("\n") + "\n```\n\n";
}

/** Build a document of roughly `blocks` sections; every 5th section adds a code block. */
export function buildDocument(blocks) {
let doc = "# Document\n\n";
for (let i = 0; i < blocks; i++) {
doc += section(i);
if (i % 5 === 0) doc += codeBlock(i);
}
return doc;
}
40 changes: 40 additions & 0 deletions bench/scenarios/public-api.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
/**
* Public-API scenarios that do not touch the live preview: HTML export and
* read-only accessors.
*/
import { createEditor } from "../../packages/core/dist/index.js";
import { buildDocument } from "./documents.mjs";

function mount(initialValue) {
const container = document.createElement("div");
document.body.appendChild(container);
const editor = createEditor({ container, initialValue, plugins: [] });
return { editor, container };
}

export function run(measure) {
const rows = [];
const doc = buildDocument(100);
const { editor, container } = mount(doc);

const scenarios = [
{ scenario: "api/export-html", call: () => editor.exportHTML() },
{ scenario: "api/get-ast", call: () => editor.getAst() },
{ scenario: "api/get-document-stats", call: () => editor.getDocumentStats() },
];

for (const { scenario, call } of scenarios) {
const sample = measure(call);
rows.push({
scenario,
subject: "doc ~23KB",
metric: "median",
value: sample.median,
unit: "ms",
});
}

editor.destroy();
container.remove();
return rows;
}
1 change: 1 addition & 0 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ Plugin-platform documentation (Chinese): [native API](./plugins/native-plugin-ap
| 25 | End-to-end testing | repo infra | P1 | planned | No | Candidate: Playwright against electron-demo |
| 26 | CI/CD pipeline polish | `.github/workflows` | P1 | in-progress | No | PR check / CI and publish workflow exist; missing e2e gate |
| 28 | Markdown-aware word / reading-time stats | new `plugin-wordcount` | P1 | done | Yes | Walks editor AST (no double parse) + CJK-first + ARIA status bar — see `openspec/changes/add-plugin-wordcount` |
| 30 | Performance benchmark harness + cursor-latency budget | repo infra (`bench/`, `packages/core`) | P1 | done | No | `pnpm bench` (zero-dependency, black-box, `bench/README.md`); scaling budget in `packages/core/test/live-preview-cursor-budget.test.ts`; enabled per-range decoration reuse in `live-preview` |

---

Expand Down
2 changes: 1 addition & 1 deletion openspec/changes/add-missing-features/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,6 @@
## Phase 4: P3 — Quality

- [x] 4.1 Accessibility (a11y) — ARIA attributes on headings, code blocks, tables
- [ ] 4.2 Performance benchmarks
- [x] 4.2 Performance benchmarks (`pnpm bench` + `bench/README.md`, cursor-latency budget in `packages/core/test/live-preview-cursor-budget.test.ts`)
- [ ] 4.3 API documentation
- [ ] 4.4 E2E tests
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
"check:api": "pnpm --filter @floatboat/nexus-plugin-api check:api && pnpm --filter @floatboat/nexus-plugin-runtime check:api && pnpm --filter @floatboat/nexus-reference-plugins check:api",
"typecheck": "pnpm -r exec tsc --noEmit",
"test": "vitest run",
"bench": "node bench/run.mjs",
"dev:electron-demo": "pnpm --filter @floatboat/nexus-electron-demo dev",
"build:electron-demo": "pnpm --filter @floatboat/nexus-electron-demo build",
"publish:packages": "pnpm -r --filter \"./packages/*\" publish --access public --no-git-checks"
Expand Down
Loading