Background
In cl runtime tests, many cases use a single ; in expect.txt. In practice this is often used for outputs that are not stable for textual golden comparison (for example, address-like values or environment-dependent formatting).
However, the current implementation treats this marker as skip the whole run, so tests appear green while the case is never executed.
Current Behavior
In cl/cltest/cltest.go (testRunFrom):
if bytes.Equal(expected, []byte{';'}) { // expected == ";" means skipping expect.txt
return
}
This returns before RunAndCaptureWithConf(...).
Why This Is a Problem
expect.txt == ";" effectively means “do not run”, not just “do not diff output”.
Consequences:
- build/runtime regressions can be hidden in
go test ./cl;
- false confidence: CI reports pass while manual
llgo run fails for the same case.
Expected Behavior
For expect.txt == ";":
- still execute
RunAndCaptureWithConf(...) (at least validate build/run success);
- skip only
output vs expect.txt textual diff.
If we still need “skip execution entirely”, introduce a separate explicit marker.
Reproduction Example
cd cl/_testgo/cgobasic
llgo run -target esp32c3-basic -emulator .
This fails with build/SSA errors, while the current go test ./cl path may not execute this case when expect.txt == ";".
If helpful, I can follow up with a PR that changes ; to run-only/no-diff semantics and adds a regression test.
Background
In
clruntime tests, many cases use a single;inexpect.txt. In practice this is often used for outputs that are not stable for textual golden comparison (for example, address-like values or environment-dependent formatting).However, the current implementation treats this marker as skip the whole run, so tests appear green while the case is never executed.
Current Behavior
In
cl/cltest/cltest.go(testRunFrom):This returns before
RunAndCaptureWithConf(...).Why This Is a Problem
expect.txt == ";"effectively means “do not run”, not just “do not diff output”.Consequences:
go test ./cl;llgo runfails for the same case.Expected Behavior
For
expect.txt == ";":RunAndCaptureWithConf(...)(at least validate build/run success);output vs expect.txttextual diff.If we still need “skip execution entirely”, introduce a separate explicit marker.
Reproduction Example
This fails with build/SSA errors, while the current
go test ./clpath may not execute this case whenexpect.txt == ";".If helpful, I can follow up with a PR that changes
;to run-only/no-diff semantics and adds a regression test.