Skip to content

chore(self-scan): schedule a baseline-freshness job to close the rest of the staleness class #1473

Description

@dekobon

Summary

#1465 added a warning when a .bca-baseline.toml entry has measured past
its recorded value. That warning cannot close the staleness class, by
construction, and this is the follow-up that can.

The residual gap

bca check's warning is emitted from filter_by_baseline, which only sees
violations that reached Baseline::classify. A function whose metric stopped
breaching its threshold entirely produces no Violation at all, so it
never reaches classify and its stale entry is invisible to that path.

So the warning covers offenders still above their limits; it cannot see an
entry whose offender is fully fixed. The looser such an entry, the longer it
survives, and nothing forces a refresh.

Proposal — option 3 from #1465

A scheduled workflow that regenerates and reports drift:

make self-scan-write-baseline-headroom
git diff --exit-code .bca-baseline.toml

A non-empty diff means the checked-in baseline no longer describes the tree.
File an issue (or open a PR with the regenerated file) when it does.

That catches both classes — decreases still above the limit, and entries
whose offender no longer violates at all — because it compares the whole
generated file rather than per-violation values.

Precedent to follow

.github/workflows/mutation-test.yml and benchmark.yml are both quarterly
crons with workflow_dispatch, permissions: {contents: read, issues: write}, and concurrency groups; mutation-test's "File issue on escaped
mutants" step is the template for the issue-filing half. Their cron dates are
deliberately offset two weeks from each other (benchmark.yml says so in a
comment) — pick a third non-colliding date and say why.

Notes

  • Use make self-scan-write-baseline-headroom, never the bare
    self-scan-write-baseline: the checked-in file carries tier = "soft", headroom = 0.95 provenance, and the bare target would make the soft-tier
    gate re-fire on untouched files.
  • bca diff-baseline already models a downward move (it has an improved
    bucket) — reuse its vocabulary so the two features speak the same language.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions