feat(ci): detect unsupported ecosystems without silencing the zero-denominator refusal - #970
Merged
Merged
Conversation
`detect` recognised six ecosystems. Repos built on Julia, Zig, Elixir, Lean, Ada, Ruby, PHP, Gleam, Agda, Idris2, Isabelle or Bun matched none of them, so `TOTAL` was 0 and the pipeline refused with "no known ecosystem detected" — true of the detector, false of the repository. Twelve manifest-keyed probes are added. They accumulate into a SEPARATE `N_UNSUPPORTED`, never into `TOTAL`. That separation is the change. Adding them to `TOTAL` would lift it above zero and silence the zero-denominator refusal while nothing had been examined — a vacuous gate, which is worse than a blind spot because it reports success. `TOTAL` therefore stays defined as "ecosystems some job in this file actually checks". The refusal still fires; it now says which of the two things went wrong, so the adoption ledger can measure real debt instead of detector blindness. The Deno probe is KEPT. It is not a lint gate, it is the trigger for the refusal job, i.e. the teeth of the no-Deno ruling; removing it would not drop support, it would silently un-enforce a ban. A comment now says so at the probe. Bun lands as detected-unsupported: `bun.lock*`/`bunfig.toml` are positive identification, a bare `package.json` is a stated assumption under the bun-only ruling. Safe while the verdict is refusal; to be re-examined when a Bun lint job exists. The report job gets prose, not a `row()` — row()'s catch-all branch prints "ran" for an unset result, which for a job that does not exist would assert the opposite of the truth. scripts/tests/detect-ecosystem-test.sh, 33 assertions, extracts and runs the SHIPPED yaml rather than a copy. Two mutants: deleting the Julia probe flips fixture 3 back to the old message; folding N_UNSUPPORTED into TOTAL silences the refusal (rc 1 -> 0), which is what proves the separation is load-bearing. Refs: #967 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ji1bq3TypfycfUPAR7hSxR
Contributor
|
Warning Review limit reachedNext included review available in 10 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
5 tasks
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



The problem
ci-pipeline.yml'sdetectjob recognised six ecosystems: Rust, Nickel,ReScript, V, Haskell, Deno. A repository built on Julia, Zig, Elixir, Lean,
Ada, Ruby, PHP, Gleam, Agda, Idris2, Isabelle or Bun matched none of them,
so
TOTALcame out 0 and the pipeline refused with "No known ecosystemdetected".
That sentence is true of the detector and false of the repository. It
is also the sentence the planned adoption ledger would have been seeded from —
so the ledger would have recorded detector blindness as repository debt.
Tracked as #967.
What was NOT done, and why
The original framing of #967 said "drop the Deno probe, add Bun". The Deno
probe is kept. It is not an attempt to lint Deno — it is the trigger for the
denorefusal job, i.e. the enforcement teeth of the standing no-Denoruling, with slugs still live in
.machine_readable/deno-allow.txt. Deletingit would not have dropped support; it would have silently un-enforced a ban,
because the refusal stops firing and the repo goes green. A comment now says
this at the probe so the next reader does not "tidy" it away.
The design question, and the answer
The obvious implementation — add
has_julia,has_zig, … to the existingcounter — is a trap. No lint job consumes them, so they would lift
TOTALabove zero and silence the zero-denominator refusal while nothing whatsoever
was checked. A vacuous gate is worse than an honest blind spot: the blind
spot reports nothing, the vacuous gate reports success.
So the twelve new probes accumulate into a separate
N_UNSUPPORTED, andTOTALkeeps its existing meaning — ecosystems some job in this file actuallyexamines. Deno counts toward
TOTALbecause a refusal is a check.The refusal still fires when nothing is checkable. It now distinguishes two
cases that need different cures:
TOTAL == 0,N_UNSUPPORTED == 0TOTAL == 0,N_UNSUPPORTED > 0Same
fail_on_no_ecosystemswitch; no second input.Honest scope
This PR does not shrink the red set — it classifies it. No lint job is
added for any of the twelve. Post-merge, the adoption ledger holds two classes:
no ecosystem at all and detected but unsupported. Two different cures, both
now countable. That is the point; the debt becomes separable instead of being
reported as one undifferentiated failure.
Bun lands as detected-unsupported.
bun.lock*/bunfig.tomlare positiveidentification; a bare
package.jsonis an assumption stated in the commentrather than hidden — under the bun-only ruling a bare
package.jsonin thisestate is presumed Bun. Safe while the verdict is refusal; flagged for
re-examination the day a Bun lint job exists, because then it would start
selecting which repositories get linted.
Detail worth noting
The report job gets prose, not a
row()call.row()'s catch-all branchprints the word
ranfor an unset result — and a job that does not exist hasno
needs.*.result. A row for Julia would have rendered| Julia | | ran |,asserting the exact opposite of the truth.
Tests
scripts/tests/detect-ecosystem-test.sh— 33 assertions, 33 pass.Auto-discovered by
scripts/run-shell-test-suite.sh(52 test file(s) passedlocally, no regressions). Mode
100755.It extracts and executes the shipped YAML, not a re-implementation — a test
that copies the logic it tests passes forever after the real thing breaks. The
extractor fails loudly if its anchors stop matching. No Python: LANGUAGE-POLICY
bans it, so extraction is awk.
Two mutants, because a suite that only ever goes green proves nothing:
N_UNSUPPORTEDintoTOTALThe second is the real control: it demonstrates that the separation, not
something incidental, is what keeps the gate from becoming vacuous.
Blast radius
detectoutputs are added, never changed —has_*,n_denoandtotalkeep their meanings, so no existing consumer moves. No new
uses:, soactions.lockis untouched.ci-pipeline.ymlisworkflow_call:-only andstill has zero callers, so nothing in the estate changes until rollout.
⚠ Rollout (the 5-repo pilot) is gated on this merging — callers must pin a
merge SHA on
main, and a dry-run before this lands reds Julia/Zig/Elixirrepos for the wrong reason and its numbers mean nothing.
Refs #967
🤖 Generated with Claude Code
https://claude.ai/code/session_01Ji1bq3TypfycfUPAR7hSxR