Skip to content

gates.json: never_required_contexts cannot catch default-setup CodeQL by name #1033

Description

@hyperpolymath

What

config/rulesets/gates.json carries never_required_contexts, a list of check-run
names that must never be promoted to a required status check. It lists
"CodeQL (default setup)".

That entry does not catch what it is aimed at. Default-setup CodeQL does not emit
a check named CodeQL (default setup) — it emits one check per analysed language,
named analyze (<language>, <build-mode>).

Measured, 2026-09-23 on metadatastician/paint-type

Deriving the required-context set with scripts/apply-branch-gates.sh:

  • gate_files=620 contexts
  • never_required_contexts excluded 3
  • analyze (actions, none) survived the exclusion and was carried into the
    derived set

It only dropped out later, and for an unrelated reason: --require-green 3 saw it
red at the time.

Why the list cannot be fixed by adding names

The context name is a function of the languages the repo happens to contain and
the build mode CodeQL selects. actions, javascript-typescript, rust,
python, go, java-kotlin, c-cpp, csharp, ruby, swift — each with its own
build-mode suffix (none, autobuild, manual). Enumerating them is a list that
goes stale the moment a repo gains a language.

This is a canon gap, not a bug in the applier: the applier faithfully applies the
list it is given.

Options

  1. Pattern matching. Let never_required_contexts entries be anchored regexes,
    or add a sibling never_required_context_patterns. ^analyze \(.*\)$ closes it
    in one line and stays closed.
  2. Source-based exclusion. Exclude by the emitting integration/app id rather than
    by name — default-setup CodeQL's check runs carry a stable app id, and the applier
    already reads integration_id when it builds the payload.
  3. Accept it and require them. Default-setup CodeQL is green on paint-type today.
    ⚠ This is the weakest arm: the context set silently changes shape whenever a repo
    gains or loses a language, and a required context that stops being emitted blocks
    every PR forever.

Option 2 is the most elegant long-term — it keys on what emitted the check rather
than on a string the emitter is free to change — but it is also the only one that
needs a new field in gates.json.

Acceptance criteria

  • An owner ruling between the three arms recorded on hyperpolymath/standards#787.
  • gates.json updated to match, with "CodeQL (default setup)" either replaced
    or kept alongside whatever mechanism is chosen.
  • A control in scripts/tests/branch-gates-apply-test.sh that feeds the applier a
    jobs list containing analyze (actions, none) and asserts it is absent from
    the derived contexts — plus the matching mutant, so a regression is caught.
  • Re-derive paint-type's context set and confirm the count drops by one.

🤖 Generated with Claude Code

https://claude.ai/code/session_01YSq3UodR3CjsuAK5yoTzHF

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions