test(aot): prove native-entry lowering executes as generated Go, and fix the frame defect it found

[nnunley]

--entry-frame binaries could silently drop every namespace-level definition, and nothing in the test suite would notice. This adds a gate that proves lowered entries really execute as generated Go, and fixes the frame defect that gate immediately found.

The defect

The generated frame ran BootCore → SetCommandLineArgs → DecodeExecUnit → LoadProgramNamespaces → prog.<Entry>(ec). For a single-file program lg -c emits a bundle with an empty NS table, so LoadProgramNamespaces iterated nothing and no namespace-level var was ever defined.

The binary was therefore correct only when every function reachable from the entry lowered to a direct native call. Any VM-backed callee — a variadic defn, or anything not :direct-callable? — resolved to a Nil var and panicked:

panic: interface conversion: *vm.Nil is not vm.Fn: missing method Arity
  rt.CachedVarFn(...) pkg/rt/golower_runtime.go:83
  tmpmod/vmbacked.Main(...)

Top-level forms in the program namespace were dropped for the same reason.

rt.RunProgramMainChunk already existed for exactly this VM-fallback role (#425 Finding 2); the frame simply never emitted a call to it.

The fix

pkg/rt/core/ir/passes/entry_frame.lg now emits rt.RunProgramMainChunk(unit) after LoadProgramNamespaces, using the existing exit-on-error shape. It is idempotent by construction: it returns early when the main chunk was already replayed as one of the NS chunks, so a bundle that does carry NS entries still executes top-level forms exactly once.

The gate

make native-entry-gate discovers test/native-entry/*.lg. Each fixture carries an exact .expect stdout and a structural .goexpect.json contract; a missing sidecar fails rather than skips. Per fixture it proves:

1. the fixture's own defn exists as a Go function matching an expected AST shape, exactly once;
2. the entry call site is classified — a direct fixture has one direct Go call and zero trampolines for that name, a vm fixture has the opposite;
3. lg-compile --entry-frame → lg -c → go build → run produces byte-exact expected stdout, exit 0;
4. three mutants, all located structurally through go/ast, never by hardcoded identifiers.

Inverted semantic mutant

For direct fixtures, mutating the generated Go body must change observed output — evidence the Go body is what runs. For vm fixtures the same mutation must not change output — evidence the Go body is dead and the trampoline is genuinely what executes. Both directions are positive, falsifiable claims; a surviving mutant on a vm row is required evidence, not a gap.

Fixtures

Fixture	Role
int_arith	typed direct lowering, unboxed int params
closure_capture	capture-carrying direct lowering
vm_backed	variadic callee stays on the trampoline; scopes the direct-call claim
toplevel_effect	top-level effect plus VM-backed callee; pins exactly-once replay

toplevel_effect and vm_backed are the regression fixtures for the frame defect: without the fix the first drops its banner and the second panics.

Validation

• Native-entry gate: 4 fixtures, 12 mutant sub-tests, all green
• TestLgCompileEntryFrameOptIn and TestNativeEntryMatrix (AOT native-entry: generate the Go entry frame, self-contain gogen, and stop the silent perf-cliff #425/feat(aot): generate native-entry Go frame from main/-main (#425 Item 1) #628 contracts) still green
• make generate + make check-generated — entry_frame.lg is a core source, artifacts regenerated
• internal/gofragment matcher: 16/16
• Full Go suite green

Relationship to #681

#681 gates strict capability and engine-output parity and explicitly declines to claim that fixture source executed as generated Go. This PR owns that claim, and is independent of #681.

Test infrastructure notes

• test/clojure-test-suite is a git submodule; jj workspaces (and plain git worktree) do not materialize it. scripts/link-clojure-test-suite.sh <workspace> [primary-worktree] symlinks it in idempotently, refuses to clobber a real directory, and is wired into make native-entry-gate. If it is missing, TestJankSuiteDirectABIGeneratedGo now fails loudly with both remedies rather than skipping.
• test/language_test.go walks test/** and executes every .lg file as a bytecode deftest file, skipping only compat, clojure-test-suite, benches, gogen. Adding fixtures under test/native-entry/ and the generator under test/tools/ therefore turned the suite red; both directories are now in that skip list, following the existing precedent.
• The generated Go for the jank direct-ABI test is written to t.TempDir() with its own go.mod; nothing generated is committed.

Rebase

Rebased onto main at 23f3ab1 (#712). pkg/rt/generated.sums conflicted and was resolved by regeneration, not by hand: make generate + make check-generated are clean at the tip.

[mparrett]

Verified independently at 679d9920 against merge-base 23f3ab15.

The frame fix matches the empty-NS lg -c bundle shape: emit rt.RunProgramMainChunk(unit) after LoadProgramNamespaces, with the existing exit-on-error shape. The early return keeps top-level forms once-only when the main chunk is already in NSOrder.

TestNativeEntryASTGate is green locally (four fixtures and the mutant sub-tests). Reverting just the new emit makes toplevel_effect and vm_backed fail with the documented Nil/CachedVarFn panic and the dropped banner, so the gate kills the defect it claims to pin. gofragment 16/16, and the existing TestLgCompileEntryFrameOptIn / TestNativeEntryMatrix CI lane is green on this head.

One non-blocking follow-up: wire TestNativeEntryASTGate (and, if you want the jank direct-ABI claim in the same lane, TestJankSuiteDirectABIGeneratedGo) into .github/workflows/go.yml. The current AOT step still only runs TestLgCompileEntryFrameOptIn|TestNativeEntryMatrix, and the matrix does not cover a VM-backed callee or a top-level side effect on an empty-NS bundle. entry_frame_test.lg protects the emitted call string; the end-to-end runtime claim lives only behind make native-entry-gate. The Makefile already strips ambient GOFLAGS=-short for that target, so extending the existing AOT -run list is enough.

Approving.

[mparrett]

@nnunley looks good. official review posted.

Btw- was thinking about knocking out the lg-compile remaining work. Unless you're already on that, I could probably squeeze it in today.

[nnunley]

Take it. If it's unclaimed, it's yours, though I think I have a PR touching it in flight

[nnunley]

Bench-ratchet: control-vs-branch at the rebased head

Run after rebasing onto 2fe51750 (#681). make bench-ratchet exits non-zero on this branch. The regressions it flags are present on the unmodified base and are not introduced here.

Method. Same command, same exclusive local lane, twice: once at the branch head 4c73840d, once at the PR's base 2fe51750 with no other changes.

Deterministic metrics (machine-independent)

metric	base→branch	base→control	branch vs control
pkg/compiler.BenchmarkInitFromLGB allocs/op	+271.6%	+271.6%	0.0000%
pkg/compiler.BenchmarkInitFromLGB bytes/op	+113.6%	+113.6%	0.0000%
pkg/ir.BenchmarkIRCompile [bytecode] allocs/op	+14.1%	+14.1%	−0.0009%
pkg/ir.BenchmarkIRCompile [gogen_ir] allocs/op	+35.7%	+35.7%	−0.0007%
pkg/ir.BenchmarkIRCompile [gogen_ir] bytes/op	+8.8%	+8.8%	−0.0016%

Raw counters are identical or differ by a single allocation (114254/114255, 151751/151752). This branch is allocation-neutral.

The wall-clock rows are not usable in either run

The calibration anchor moved +166.2% on the branch run and +85.9% on the control run — same machine, same lane. The two runs also disagree on which benchmarks qualify as REGRESSION (branch flags 2, control flags 3; IRCompile [gogen_ir] reads +1.6% on one and +7.4% on the other). Both captures are noise-dominated, so only the deterministic counters above carry signal.

Baseline provenance

The arm64/Apple M3 entry in docs/perf/baseline.json is a local-machine recapture:

captured_at:     2026-07-18T11:47:34Z
captured_at_sha: f154c7fb6bc9

That predates #645, #719, #726 and #681, so the base→current column is measuring four weeks of merged work rather than anything in this PR. Replacing local recapture with CI-timeline seeding is #651; the per-tier release reference is #597; the umbrella is #663.

One change this PR makes to measurement coverage

The branch produced 6 NEW records (BenchmarkClojureTestSuite*, …CompileOnly*) for suite steps where the control produced 0 records. scripts/link-clojure-test-suite.sh plus the Makefile change make those benchmarks execute; on the base the same steps yield nothing. Added coverage, no added cost — relevant to the silent-fallback class in #660/#663.

Gates green on 4c73840d

make check-generated · make native-entry-gate (4 fixtures × 12 mutant sub-tests) · TestJankSuiteDirectABIGeneratedGo · internal/gofragment (6) · TestLgCompileEntryFrameOptIn (4) · full ./test/ suite · pre-push hooks.

The generated.sums conflict from the rebase was resolved by regeneration, verified with make check-generated.

---

Delivery verified by local pr-claim-gate: exact PR/validated head 4c73840d4f3c306f69312f238eb407e20365295e; relevance claims pkg/rt/generated.sums occur in both the delivered delta and the PR feature delta; receipt /Users/ndn/.local/state/pr-claim-gate/nooga-let-go/pr-729.json.

[mparrett]

@nnunley This one looks ready to merge. Happy to do it, but didn't want to disrupt anything you have in flight. Also willing to take the non-blocking follow-up in my last comment.