fix(gates): strict-by-install Action + pre-commit, robust sticky scorecard (F3/F8) - #29
Merged
Merged
Conversation
…idation runs The GitHub Action installed the bare package (no [schemas] extra) on both install paths and the pre-commit loop-doctor hook declared no deps, so both gate surfaces ran loop/contract.py's pure-stdlib structural fallback — a type-invalid contract passed the shipped gates. The CLI's structural default is deliberate and unchanged; strictness now lands by-install on the gates only. - action.yml: both paths install "loop-engineer[schemas,yaml]" (PyPI-pinned and action-checkout). Local-path-with-extras syntax proven in a scratch venv. - action.yml doctor step asserts the report's validation_mode is "jsonschema" so a future packaging regression that drops the extras fails loudly. - .pre-commit-hooks.yaml: additional_dependencies jsonschema>=4, pyyaml>=6. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The scorecard lived in an inline action heredoc that did int(sys.argv[2]) — a
non-integer fail-under-score ("80.5"/"abc") tracebacked instead of failing
cleanly — and the PR-comment step POSTed a fresh comment every run.
- Extract the logic to scripts/action_scorecard.py (invoked from the action
checkout at github.action_path). It validates fail-under as an integer 0-100
and emits a clear ::error with a distinct exit code (2) on bad input, separate
from the fail-under-breach code (1); same summary/scorecard.md/warning/
fail-under behavior otherwise. TDD-covered in test_action_scorecard.py.
- Sticky PR comment: the rendered body carries a <!-- loop-engineer-scorecard -->
marker; the comment step finds that comment and PATCHes it in place, else
POSTs. Stays non-fatal on API failure.
- Document honestly (fail-under-score input + README): the inspect score is an
advisory heuristic and can be gamed; loop doctor is the hard gate.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes external-review findings F3 and F8: the shipped gate surfaces (GitHub Action, pre-commit hook) silently ran structural-fallback validation, and the Action's scorecard step was fragile.
action.ymlinstalls[schemas,yaml]extras in both install paths (PyPI-versioned and action-checkout). The local-path-with-extras pip syntax was proven in a scratch venv, and an independent verifier reproduced the proof — strict mode catches a type-invalid contract (plan_versionas string) that the fallback misses..pre-commit-hooks.yamlloop-doctorhook declaresadditional_dependencies: [jsonschema>=4, pyyaml>=6], pinned by test.scripts/action_scorecard.py;fail-under-scoreis validated as an integer 0–100 with a clear::errorinstead of anint()traceback.<!-- loop-engineer-scorecard -->marker is embedded and the step updates the existing comment instead of posting a new one each run. Still non-fatal on API failure.loop doctoris the hard gate.Test plan
action.ymlyaml-lints; referenced files exist atgithub.action_pathin both install modes🤖 Generated with Claude Code