Skip to content

test(cl): when expect.txt is ';', skip diff only, not execution #1742

@luoliwoshang

Description

@luoliwoshang

Background

In cl runtime tests, many cases use a single ; in expect.txt. In practice this is often used for outputs that are not stable for textual golden comparison (for example, address-like values or environment-dependent formatting).

However, the current implementation treats this marker as skip the whole run, so tests appear green while the case is never executed.

Current Behavior

In cl/cltest/cltest.go (testRunFrom):

if bytes.Equal(expected, []byte{';'}) { // expected == ";" means skipping expect.txt
    return
}

This returns before RunAndCaptureWithConf(...).

Why This Is a Problem

expect.txt == ";" effectively means “do not run”, not just “do not diff output”.

Consequences:

  • build/runtime regressions can be hidden in go test ./cl;
  • false confidence: CI reports pass while manual llgo run fails for the same case.

Expected Behavior

For expect.txt == ";":

  • still execute RunAndCaptureWithConf(...) (at least validate build/run success);
  • skip only output vs expect.txt textual diff.

If we still need “skip execution entirely”, introduce a separate explicit marker.

Reproduction Example

cd cl/_testgo/cgobasic
llgo run -target esp32c3-basic -emulator .

This fails with build/SSA errors, while the current go test ./cl path may not execute this case when expect.txt == ";".


If helpful, I can follow up with a PR that changes ; to run-only/no-diff semantics and adds a regression test.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions