Skip to content

chore(utils): the build's own gate scripts are unlinted and untyped #1477

Description

@dekobon

Summary

The 29 Python scripts in utils/ are reached by no lint, format or
type gate. They are the scripts that gate everything else — the gates are
themselves ungated.

Where the scoping excludes them

  • .pre-commit-config.yaml:26-30 — both ruff-check and ruff-format
    carry files: ^big-code-analysis-py/.*\.py$.
  • Makefilepy-lint / py-fmt-check run ruff over $(BCA_PY_DIR)
    (big-code-analysis-py) and $(SMOKE_PY_DIR) (scripts/smoke) only;
    py-typecheck runs mypy --strict python tests examples $(SMOKE_LIB_PY)
    from inside big-code-analysis-py.

utils/ is in none of those paths.

Measured

Using the bindings crate's own ruff and config
(big-code-analysis-py/.venv/bin/ruff --config big-code-analysis-py/pyproject.toml):

Check Result
ruff check utils 835 errors
ruff format --check utils 27 of 29 files would be reformatted
mypy --strict utils 29 errors in 16 files

The 835 is misleading, and that decides the fix

Rule breakdown of the ruff findings:

Code Count What
PT009 744 use a plain assert instead of unittest's assertEqual
PT027 41 use pytest.raises instead of assertRaises
PT018 13 split a composite assertion
PT008 4 patch-style monkeypatching
everything else 33 E501, UP037, PERF401, SIM117, UP006, SIM118, …

802 of 835 are flake8-pytest-style rules fired at code that
deliberately uses unittest.
The utils/*-test.py self-tests are
unittest.TestCase suites run directly (python3 utils/<gate>-test.py),
not under pytest — that is why they are shaped this way, and the bindings
crate's config enables the PT family because it is a pytest suite.

So adopting the bindings config wholesale for utils/ is the wrong move:
it would demand rewriting ~800 assertions to satisfy a rule about a test
runner these files do not use. The real backlog is the 33 non-PT
findings plus the formatting and typing gaps.

Why it matters

These are not incidental scripts. AGENTS.md lists them as the gates run
by make pre-commit / make ci: check-versions.py,
check-snapshot-anchors.py, check-rustfmt-bail.py,
check-manpage-drift.py, check-diagnostic-prefix.py,
check-safety-doc-pin.py, check-grammar-marker-sync.py,
check-excluded-manifests.py, check-ruff-lockstep.py,
check-publish-metadata.py, and now check-feature-gates.py.

A gate that silently stops working is the worst failure mode in the
repository — check-snapshot-anchors.py already has a lesson attached to
exactly that (#1192, where a mis-lexed char literal made the gate report
clean files). Type errors and unreviewed formatting in that code are
carrying real risk.

Suggested shape

  1. A utils/-scoped ruff config that inherits the house style but
    drops flake8-pytest-style, since these suites are unittest by
    design. Either a [tool.ruff] section in a utils/pyproject.toml or a
    per-directory extend-per-file-ignores.
  2. Fix the 33 real findings and run ruff format — a one-time
    reformat of 27 files, best landed on its own so it does not hide behind
    a functional change.
  3. mypy --strict over utils/, fixing the 29 errors. Most are
    no-untyped-call on the shared _load_module helper the *-test.py
    files use to import a hyphenated script, so typing that one helper
    likely clears a large fraction.
  4. Wire it in: extend the pre-commit files: patterns and add a
    utils/ arm to make py-lint / py-fmt-check / py-typecheck.

One constraint

AGENTS.md's "One ruff version, four declarations" rule applies:
big-code-analysis-py/uv.lock is the anchor, and the rev: in
.pre-commit-config.yaml, the requirements/dev.txt export and
pyproject.toml's bound must follow it. Whatever config utils/ gets
must not introduce a fifth place where a ruff version is named —
make check-ruff-lockstep only knows about the four.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions