Skip to content

feat(ci): detect unsupported ecosystems without silencing the zero-denominator refusal - #970

Merged
hyperpolymath merged 1 commit into
mainfrom
ci/detect-unsupported-ecosystems
Sep 22, 2026
Merged

hyperpolymath merged 1 commit into
mainfrom
ci/detect-unsupported-ecosystems

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

The problem

ci-pipeline.yml's detect job recognised six ecosystems: Rust, Nickel,
ReScript, V, Haskell, Deno. A repository built on Julia, Zig, Elixir, Lean,
Ada, Ruby, PHP, Gleam, Agda, Idris2, Isabelle or Bun
matched none of them,
so TOTAL came out 0 and the pipeline refused with "No known ecosystem
detected"
.

That sentence is true of the detector and false of the repository. It
is also the sentence the planned adoption ledger would have been seeded from —
so the ledger would have recorded detector blindness as repository debt.

Tracked as #967.

What was NOT done, and why

The original framing of #967 said "drop the Deno probe, add Bun". The Deno
probe is kept.
It is not an attempt to lint Deno — it is the trigger for the
deno refusal job, i.e. the enforcement teeth of the standing no-Deno
ruling, with slugs still live in .machine_readable/deno-allow.txt. Deleting
it would not have dropped support; it would have silently un-enforced a ban,
because the refusal stops firing and the repo goes green. A comment now says
this at the probe so the next reader does not "tidy" it away.

The design question, and the answer

The obvious implementation — add has_julia, has_zig, … to the existing
counter — is a trap. No lint job consumes them, so they would lift TOTAL
above zero and silence the zero-denominator refusal while nothing whatsoever
was checked
. A vacuous gate is worse than an honest blind spot: the blind
spot reports nothing, the vacuous gate reports success.

So the twelve new probes accumulate into a separate N_UNSUPPORTED, and
TOTAL keeps its existing meaning — ecosystems some job in this file actually
examines
. Deno counts toward TOTAL because a refusal is a check.

The refusal still fires when nothing is checkable. It now distinguishes two
cases that need different cures:

Condition Message
TOTAL == 0, N_UNSUPPORTED == 0 unchanged — repository not recognised at all
TOTAL == 0, N_UNSUPPORTED > 0 names the ecosystems and cites #967 — measured debt

Same fail_on_no_ecosystem switch; no second input.

Honest scope

This PR does not shrink the red set — it classifies it. No lint job is
added for any of the twelve. Post-merge, the adoption ledger holds two classes:
no ecosystem at all and detected but unsupported. Two different cures, both
now countable. That is the point; the debt becomes separable instead of being
reported as one undifferentiated failure.

Bun lands as detected-unsupported. bun.lock*/bunfig.toml are positive
identification; a bare package.json is an assumption stated in the comment
rather than hidden — under the bun-only ruling a bare package.json in this
estate is presumed Bun. Safe while the verdict is refusal; flagged for
re-examination the day a Bun lint job exists, because then it would start
selecting which repositories get linted.

Detail worth noting

The report job gets prose, not a row() call. row()'s catch-all branch
prints the word ran for an unset result — and a job that does not exist has
no needs.*.result. A row for Julia would have rendered | Julia | | ran |,
asserting the exact opposite of the truth.

Tests

scripts/tests/detect-ecosystem-test.sh — 33 assertions, 33 pass.
Auto-discovered by scripts/run-shell-test-suite.sh (52 test file(s) passed
locally, no regressions). Mode 100755.

It extracts and executes the shipped YAML, not a re-implementation — a test
that copies the logic it tests passes forever after the real thing breaks. The
extractor fails loudly if its anchors stop matching. No Python: LANGUAGE-POLICY
bans it, so extraction is awk.

Two mutants, because a suite that only ever goes green proves nothing:

Mutant Must happen Observed
delete the Julia probe fixture 3 reverts to the old "no known ecosystem" message ✅
fold N_UNSUPPORTED into TOTAL the refusal goes silent, rc 1 → 0 ✅

The second is the real control: it demonstrates that the separation, not
something incidental, is what keeps the gate from becoming vacuous.

Blast radius

detect outputs are added, never changed — has_*, n_deno and total
keep their meanings, so no existing consumer moves. No new uses:, so
actions.lock is untouched. ci-pipeline.yml is workflow_call:-only and
still has zero callers, so nothing in the estate changes until rollout.

⚠ Rollout (the 5-repo pilot) is gated on this merging — callers must pin a
merge SHA on main, and a dry-run before this lands reds Julia/Zig/Elixir
repos for the wrong reason and its numbers mean nothing.

Refs #967

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ji1bq3TypfycfUPAR7hSxR

`detect` recognised six ecosystems. Repos built on Julia, Zig, Elixir,
Lean, Ada, Ruby, PHP, Gleam, Agda, Idris2, Isabelle or Bun matched none
of them, so `TOTAL` was 0 and the pipeline refused with "no known
ecosystem detected" — true of the detector, false of the repository.

Twelve manifest-keyed probes are added. They accumulate into a SEPARATE
`N_UNSUPPORTED`, never into `TOTAL`.

That separation is the change. Adding them to `TOTAL` would lift it
above zero and silence the zero-denominator refusal while nothing had
been examined — a vacuous gate, which is worse than a blind spot
because it reports success. `TOTAL` therefore stays defined as
"ecosystems some job in this file actually checks". The refusal still
fires; it now says which of the two things went wrong, so the adoption
ledger can measure real debt instead of detector blindness.

The Deno probe is KEPT. It is not a lint gate, it is the trigger for
the refusal job, i.e. the teeth of the no-Deno ruling; removing it
would not drop support, it would silently un-enforce a ban. A comment
now says so at the probe.

Bun lands as detected-unsupported: `bun.lock*`/`bunfig.toml` are
positive identification, a bare `package.json` is a stated assumption
under the bun-only ruling. Safe while the verdict is refusal; to be
re-examined when a Bun lint job exists.

The report job gets prose, not a `row()` — row()'s catch-all branch
prints "ran" for an unset result, which for a job that does not exist
would assert the opposite of the truth.

scripts/tests/detect-ecosystem-test.sh, 33 assertions, extracts and
runs the SHIPPED yaml rather than a copy. Two mutants: deleting the
Julia probe flips fixture 3 back to the old message; folding
N_UNSUPPORTED into TOTAL silences the refusal (rc 1 -> 0), which is
what proves the separation is load-bearing.

Refs: #967

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ji1bq3TypfycfUPAR7hSxR
@coderabbitai

coderabbitai Bot commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 10 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 6856404a-3715-4b00-a292-0c4ba16cff8f

📥 Commits

Reviewing files that changed from the base of the PR and between 12d76e8 and 3f654c6.

📒 Files selected for processing (2)
  • .github/workflows/ci-pipeline.yml
  • scripts/tests/detect-ecosystem-test.sh

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant