Revised regression test infrastructure

[ericvh]

Revise test infrastructure to do automated regressions against mainline, for-next, fixes-next, and development branch. Also add better dashboard support for tracking tests over time and performance/latency over time. Consider adding automated bisection to the test framework.

[ericvh]

Assessment (2026-08-26) — partially done; keep open

Validated against current main, live Actions, releases, and wiki.

Still a problem?

Partially. Core revised infra exists; several issue goals remain.

What’s already in place

• Clean-slate guest-direct harness (QEMU arm64 + virtio-9p)
• Split pipeline: sync → Image publish → harness (workflow_run) → results.json / wiki Regression-Dashboard
• Tip matrix currently green: smoke, fsx, postmark, dbench, diod-regression, qemu-9p2000.L (run)

Gaps vs this issue

• No automated matrix for for-next / fixes-next / devel (only newest v6+ tag + optional upstream→kernel-main)
• No historical performance/latency dashboard over time
• No automated bisection

Plan

1. Add scheduled/manual publish+harness for maintainer tips (ericvh/for-next, fixes/next, named devel) with distinct release tags.
2. Persist suite timings + benchmark metrics in results.json; extend wiki (or pages) with history.
3. Optional workflow_dispatch bisect: bad/good SHA → rebuild Image → smoke/protocol gate.

[ericvh]

Validation (2026-08-26)

Still a problem as an umbrella, but the core harness exists.

Current main is a working guest-direct pipeline (QEMU arm64 + virtio-9p + initrd/chroot), not the 2024 SSH/cpu setup. Latest tip report for v7.2 / kernel-latest is all green (run 32931311311, wiki dashboard): smoke, fsx, postmark, dbench, diod-regression, qemu-9p2000.L.

What this issue asked for that is not done:

• Automated regressions against mainline, for-next, fixes-next, and a development branch (today we publish/test the newest v6+ tag plus a stale kernel-main Image from 2026-05-24).
• Dashboard over time (wiki table is last-run only; results.json has pass/fail, no history/perf/latency series).
• Automated bisection.

Keep this issue as the umbrella; do the remaining work via #5, #6, #11, and a dedicated bisection follow-up rather than another harness rewrite.

Plan

1. Refs (with Automate nightly runs against mainline, for-next and pending fixes. #5/Add trigger for pushes/merges to v9fs/linux to trigger regression runs #6): scheduled publish+harness for v9fs/linux upstream (mainline), ericvh/for-next, fixes/next, and optionally ericvh/devel. Keep all workflows in this repo (Contract: all regression CI in v9fs/test; v9fs/linux is mirror-only #15).
2. Dashboard: append-only history (wiki page or results.json artifacts on kernel-latest) with suite status and extracted dbench/postmark numbers; sparkline or table by linux_ref/date.
3. Bisection: only after multi-ref CI is stable. git bisect on v9fs/linux between last-pass and first-fail tag/SHA, reusing linux-kernel-publish.yml + a one-suite harness dispatch (smoke or the failing matrix cell). Cap iterations; post the culprit SHA on the wiki/issue.
4. Close this issue when (1)+(2) exist; treat bisection as a stretch checkbox, not a rewrite.