One app, six times: where Markout wins, and where it doesn't

[fcapolini]

We keep a benchmark in the repo that writes the same product catalog in
Markout, Alpine 3, React 18, Svelte 5, Vue 3.6 Vapor and Next 15, drives all
of them through the same four interactions in a real Chromium, and weighs them
as well as timing them. Same data, same CSS, same class names, same DOM — and
the DOM equality is checked, not assumed, because two ports can match on
every tag and still render different numbers.

Markout appears twice, because served and built are different deliveries.

The comparison we didn't have until last week

Every other port is a Vite SPA, so markout build had peers and served
mode had none — the delivery the whole project is arranged around was being
compared against itself. Next 15 is the one other entrant that renders on the
server and ships the result, so it's there for delivery, not as a seventh
runtime. At 1,020 rows:

	Markout (server)	Next
First card	12.4ms	9.8ms
Interactive	37.2ms	56.5ms
Total gzip	17.6 KB	108.8 KB
Build	161ms	8,125ms

Next is ahead on first card at every size — and the honest version of that
is smaller than the table makes it look. The gap is 2.2–4.2ms at the three
realistic sizes across two runs, against a run-to-run swing of 1.2–1.9ms on
that same row. The direction is consistent in all eight measurements and has a
cause we can point at — Markout's served document is 60 KB against Next's
27.6 KB, because it carries the page's expressions and scope tree in the markup
— so it's a real property and not noise. But the magnitude is a couple of
milliseconds, and nobody perceives a couple of milliseconds. At 10,020 rows it
widens to 8–10ms, comfortably clear of the noise, at a catalog size nobody
ships.

Interactive is a column we added because of this port. A page that arrives
rendered has two instants — content on screen, and content that answers a click
— where a page that builds its own content in the browser has one. That gap is
19.3ms, about ten times Markout's run-to-run variance and six times the
first-card gap, so as measurements go it's the more solid of the two.

Applying the same standard to our own side: 37ms and 56ms are both well under
the threshold where anyone notices a click landing. On localhost, neither
timing column is something a visitor feels. What a visitor does feel is
weight — 91 KB gzipped is roughly 450ms of transfer on throttled 4G, and that
one is not subtle. It's also the column that gets worse off the loopback,
while the millisecond differences stay invisible.

Where Markout is behind, and what we're doing about it

Not buried at the bottom, because these are the numbers we'd want to see first
if we were evaluating it — and because this is what the benchmark is for.
It isn't a scoreboard we publish when it flatters us; it's how we find out
which columns need work next.

First, the context that matters most: at 30 rows the mount gap is gone
entirely. Markout builds the page in 1.7ms against Svelte's 1.4, Next's 1.9,
Vue's 2.0 and React's 2.2. Thirty rows is the size a page of this shape usually
is. Everything below is a scale cost, at sizes past what any of these tools
is really for:

• Heap at scale. 297.7 MB at 10,020 rows against React's 64.5 and Svelte's
  79.6. That's a card building 16 scopes, weighed instead of timed.
• Mount at scale. 2.5–4.8× slower than the framework ports from 300 rows up.
• Repeated small mutations. 20× add-to-cart at 10,020 rows: 249.6ms against
  62–71ms for everyone else. The worst column in the file.

These have one shared cause — a card builds 16 scopes, so a 10,000-row page
builds ~160,000 of them — and two measured levers, both recorded in TODO.md:
building fewer scopes per card returned 46% of mount, and making each
scope allocate less returned 19%, along with improvements to filter and
add-to-cart that making the walk faster never produced. Neither is speculative;
both were measured by doing it. Mount moved 19% within a single release cycle
on the strength of that work, and the rest of the list is still in front of us.

Markout is 0.6.1 and considerably younger than everything in this table. Being
2.5× off Svelte on a stress-test column at a catalog size nobody ships is a
list of things to fix, not a verdict — and we'd rather publish the list than
pick benchmarks that hide it.

Two bugs the benchmark had

Worth saying out loud, because a benchmark's credibility is mostly its bug
history:

• The weight table classified every <link> initiator as CSS, so 3.3 KB of
  preloaded JavaScript sat in the stylesheet column. Next was the first port to
  preload anything, which is why it survived four ports.
• The build table was timing PATH lookups. execFileSync('node', …) searches
  PATH rather than exec'ing, npm and npx each prepend entries, and each one
  costs milliseconds to rule out. It added a near-constant 25–70ms per spawn —
  noise against an 8-second build, half of a 150ms one. It didn't shift the
  table so much as tilt it, and only against the fastest row, which was ours.

What we'd like

Tell us which number you don't believe. The methodology, the parity contract,
and every judgment call that could have rigged it are written down next to the
code that makes them — including the ones that favour the other tools.

And if you'd port it to something we're missing, the recipe is in the README.
Solid, Qwik and Astro are the obvious gaps.