test(cli): self-diagnosing consent-down test (fix the red cell on main) - #112
Merged
Merged
Conversation
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
…n non-fatal
main is red on one cell, and the log archive is not reachable from anywhere
I can read, so the fix has to carry its own diagnosis.
`test run/2 confirm gate a yes with the consent service down sends nothing
and exits non-zero` was the single failure across all three cells. Exit
code 2 from `mix test` is not a plain assertion failure: it is the branch
where an exception escapes the suite, and the only way for that to happen
here is teardown. So the `on_exit` that puts the consent service's name
back is now best-effort and cannot raise — an exception there reports
against whichever test happens to be running, which is how one broken
test hides behind an innocent one.
The assertions now name both values they compare, in the message:
* a wrong exit code means the refusal branch was never taken, which
would mean the CLI minted a capability with the service down, i.e.
the gate failed open;
* an empty stderr means the branch was taken but said nothing, which
would mean the refusal is invisible to the person it is refusing.
Two diagnostics worth having either way: if this still fails, the next
run says which of the two it is, instead of only naming the test.
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
arena-ai-coding-agent
Bot
requested a review
from hyperpolymath
as a code owner
September 29, 2026 17:17
Contributor
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The floor is stated as "cowlib 2.20 does not compile on OTP 26", which was a reading of a version constraint. It is now a measurement: on 2026-09-19 every OTP 26 cell in CI (Elixir 1.15, 1.16 and 1.17) failed with `function parse/1 undefined`, while the same commit's OTP 27 cell passed cleanly. Both places that carry the rationale now carry the evidence — the README where the floor is promised, and the CI matrix comment where it is enforced and where someone will look when they wonder why the old Elixir versions are gone. Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
hyperpolymath
approved these changes
Sep 29, 2026
hyperpolymath
approved these changes
Sep 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
main is currently red: all three Elixir cells fail on exactly one test,
run/2 confirm gate a yes with the consent service down sends nothing and exits non-zero.Why PR #111 was needed first: log archives live on
results-receiver.actions.githubusercontent.com, which the sandbox cannot reach, so the failure could not be read the way a human would. That PR (already merged) re-emits the failing block as a check annotation.The diagnosis from
mix testexit code 2: that is not a plain assertion failure — it is the branch where an exception escapes the suite, and the only candidate here is theon_exitthat restores the consent service's registered name. It is now best-effort and cannot raise. The assertions also name the values they compare, so if this still fails the next run says which of the two behaviours is wrong rather than only which test.No behaviour changed; nothing but the test moved.