Skip to content

A rust_test that compiles to nothing passes CI #462

Description

@chrysh

rust_test reports PASSED when the test binary contains zero tests, so a target
whose sources all get stripped by cfg is indistinguishable from a target whose
tests ran. Nothing in bazel test output or in ./pw ci flags it.

Found in PR #386: attest_producer_integration_test set features = ["test-support"], which is Bazel's own attribute, not rules_rust's
crate_features. The Rust feature was never set, #![cfg(feature = "test-support")] at the top of tests/producer_integration.rs stripped the
whole file, and the target compiled to an empty crate. It was green from the day
it was added and none of its nine tests had ever run. Fixing the attribute makes
all nine run and pass.

Two checks worth having, cheapest first:

  1. A presubmit check that rejects features = on rust_test, rust_library,
    rust_binary and rust_proc_macro targets, pointing at crate_features
    instead. First-party BUILD files currently have zero uses, so this starts
    clean and stays clean. It fits next to the existing checks in
    presubmit/presubmit.py QUICK.

  2. Fail a test target that runs zero tests. This catches the general case, not
    just the feature typo: an empty #[cfg(test)] module, a filter that matches
    nothing, a file that stops being compiled. The count is in the test log
    (test result: ok. 0 passed), so a wrapper or a log check in the ci_tests
    build in workflows.json can assert it is non-zero.

Check 1 alone would have caught #386. Check 2 is what stops the next variant.

One detail for check 2: do not read the console stream. On a cached target
Bazel prints only (cached) PASSED in 0.0s and streams no test output at all,
so --test_output=streamed shows nothing to parse. The per-target log is
written either way, cached or not, so the check should read
bazel-testlogs/<pkg>/<target>/test.log and assert the test result: line
reports a non-zero pass count. This is also why the zero-test case is invisible
in the CI logs for a re-run of the same commit.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions