Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
name: CI

on:
Expand Down Expand Up @@ -27,6 +27,16 @@
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: ./
- id: sheriff
uses: ./
with:
base: HEAD^
comment: "false"
mode: advisory
- name: Verify Action outputs
env:
RISK: ${{ steps.sheriff.outputs.risk }}
POLICY_PASSED: ${{ steps.sheriff.outputs.policy-passed }}
run: |
test -n "$RISK"
test "$POLICY_PASSED" = "true" -o "$POLICY_PASSED" = "false"
3 changes: 2 additions & 1 deletion .pr-sheriff.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,6 @@
"require_tests_after_lines": 120,
"test_patterns": ["tests/**", "test/**", "**/*_test.*", "**/*.test.*", "**/*.spec.*"],
"sensitive_patterns": [".github/workflows/**", "**/auth/**", "**/security/**", "**/migrations/**", "Dockerfile", "**/Dockerfile", "package-lock.json", "poetry.lock"],
"ignore_patterns": ["**/*.md", "docs/**"]
"ignore_patterns": ["**/*.md", "docs/**"],
"path_rules": []
}
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@

All notable changes to PR Sheriff are documented here.

## 0.4.0 - 2026-06-06

- Add path-specific thresholds and test requirements.
- Show an explainable risk score breakdown in every report.
- Add advisory mode for gradual, non-blocking adoption.
- Validate numeric thresholds and nested path rule configuration.
- Expose `policy-passed` as a GitHub Action output.

## 0.3.1 - 2026-06-06

- Publish PR Sheriff to the GitHub Actions Marketplace.
Expand Down
64 changes: 61 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,12 @@ Policy check passed.
The command exits with `1` when policy violations exist, making it suitable for
CI and pre-push hooks. Use `--json` for integrations.

Start with `--advisory` to report violations without blocking contributors:

```bash
pr-sheriff check --base origin/main --advisory
```

## What it checks

- Maximum changed lines and files
Expand Down Expand Up @@ -75,7 +81,7 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: Hayal08/pr-sheriff@v0.3.1
- uses: Hayal08/pr-sheriff@v0.4.0
with:
base: origin/${{ github.base_ref }}
```
Expand All @@ -92,9 +98,26 @@ comment, and fails when policy is violated.
| `head` | `HEAD` | Head git ref for the three-dot diff |
| `config` | `.pr-sheriff.json` | Path to repository policy |
| `comment` | `true` | Create or update a report comment on pull requests |
| `mode` | `enforce` | Use `advisory` to report without failing the check |

The Action exposes `risk`, `score`, `changed-files`, `changed-lines`, and
`tests-changed` outputs for later workflow steps.
`tests-changed` outputs for later workflow steps. The `policy-passed` output is
`false` when violations exist, including in advisory mode.

### Gradual rollout

Use advisory mode to learn what the policy would flag before making it a
required check:

```yaml
- uses: Hayal08/pr-sheriff@v0.4.0
with:
base: origin/${{ github.base_ref }}
mode: advisory
```

Advisory mode keeps the Job Summary, PR comment, outputs, and annotations, but
turns policy errors into warnings and returns a successful exit code.

## Configuration

Expand All @@ -107,13 +130,48 @@ Run `pr-sheriff init` or add `.pr-sheriff.json` manually:
"require_tests_after_lines": 120,
"test_patterns": ["tests/**", "**/*_test.*", "**/*.test.*"],
"sensitive_patterns": [".github/workflows/**", "**/auth/**", "**/migrations/**"],
"ignore_patterns": ["**/*.md", "docs/**"]
"ignore_patterns": ["**/*.md", "docs/**"],
"path_rules": []
}
```

Unknown configuration keys are rejected so typos cannot silently weaken a
policy.

### Path-specific rules

Different parts of a repository can have stricter review policies. Each matched
rule is shown separately in the report:

```json
{
"path_rules": [
{
"name": "database migrations",
"patterns": ["**/migrations/**"],
"max_changed_lines": 100,
"require_tests_after_lines": 1
},
{
"name": "frontend",
"patterns": ["web/**"],
"max_changed_files": 15,
"require_tests_after_lines": 80
}
]
}
```

Path rules support `max_changed_lines`, `max_changed_files`, and
`require_tests_after_lines`. Global limits still apply.

### Explainable risk score

Human-readable, JSON, Job Summary, and PR comment reports now show how changed
lines, changed files, and sensitive paths contribute to the risk score. JSON
consumers can use `score_breakdown` and `path_rule_results` for custom
dashboards or workflow decisions.

## Try it safely

Open a pull request that changes more than `require_tests_after_lines` without
Expand Down
7 changes: 5 additions & 2 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,19 @@ review. The roadmap is intentionally small and driven by maintainer feedback.
## Next: better policy checks

- Require changelog entries for user-visible changes.
- Support path-specific thresholds and policies.
- Detect dependency and public API changes.

Recently completed:

- Support path-specific thresholds and policies.
- Explain how each part of the risk score was calculated.
- Add a non-blocking advisory mode for gradual adoption.

## Later: larger repositories

- Add first-class monorepo support.
- Report risk separately for affected packages.
- Support reusable organization-wide policy presets.
- Explore a non-blocking advisory mode for gradual adoption.

## Not planned

Expand Down
18 changes: 17 additions & 1 deletion action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ inputs:
description: Create or update a report comment on pull requests
required: false
default: "true"
mode:
description: Policy mode, either enforce or advisory
required: false
default: enforce

outputs:
risk:
Expand All @@ -40,6 +44,9 @@ outputs:
tests-changed:
description: Whether the pull request changes tests
value: ${{ steps.check.outputs.tests-changed }}
policy-passed:
description: Whether the pull request satisfies every configured policy
value: ${{ steps.check.outputs.policy-passed }}

runs:
using: composite
Expand All @@ -51,17 +58,26 @@ runs:
PR_SHERIFF_HEAD: ${{ inputs.head }}
PR_SHERIFF_CONFIG: ${{ inputs.config }}
PR_SHERIFF_COMMENT: ${{ inputs.comment }}
PR_SHERIFF_MODE: ${{ inputs.mode }}
GITHUB_TOKEN: ${{ github.token }}
run: |
comment_args=()
if [[ "$PR_SHERIFF_COMMENT" == "true" ]]; then
comment_args+=(--github-comment)
fi
mode_args=()
if [[ "$PR_SHERIFF_MODE" == "advisory" ]]; then
mode_args+=(--advisory)
elif [[ "$PR_SHERIFF_MODE" != "enforce" ]]; then
echo "::error::mode must be either enforce or advisory"
exit 2
fi
PYTHONPATH="$GITHUB_ACTION_PATH/src" python -m pr_sheriff check \
--base "$PR_SHERIFF_BASE" \
--head "$PR_SHERIFF_HEAD" \
--config "$PR_SHERIFF_CONFIG" \
--github-summary "$GITHUB_STEP_SUMMARY" \
--github-output "$GITHUB_OUTPUT" \
--github-annotations \
"${comment_args[@]}"
"${comment_args[@]}" \
"${mode_args[@]}"
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "pr-sheriff"
version = "0.3.1"
version = "0.4.0"
description = "Deterministic pull request risk checks for busy maintainers"
readme = "README.md"
requires-python = ">=3.10"
Expand Down
2 changes: 1 addition & 1 deletion src/pr_sheriff/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"""PR Sheriff: deterministic pull request risk checks."""

__version__ = "0.3.1"
__version__ = "0.4.0"
57 changes: 49 additions & 8 deletions src/pr_sheriff/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ def build_parser() -> argparse.ArgumentParser:
check.add_argument("--head", default="HEAD", help="head git ref")
check.add_argument("--config", default=".pr-sheriff.json", type=Path)
check.add_argument("--json", action="store_true", dest="as_json")
check.add_argument(
"--advisory", action="store_true", help="report violations without failing"
)
check.add_argument("--github-summary", type=Path, help=argparse.SUPPRESS)
check.add_argument("--github-output", type=Path, help=argparse.SUPPRESS)
check.add_argument("--github-annotations", action="store_true", help=argparse.SUPPRESS)
Expand All @@ -32,6 +35,13 @@ def build_parser() -> argparse.ArgumentParser:
def print_report(report) -> None:
print(f"PR risk: {report.risk.upper()} ({report.score}/100)")
print(f"Changed: {report.changed_files} files, {report.changed_lines} lines")
if report.score_breakdown:
print(
"Score: "
f"lines +{report.score_breakdown['changed_lines']}, "
f"files +{report.score_breakdown['changed_files']}, "
f"sensitive +{report.score_breakdown['sensitive_files']}"
)
if report.sensitive_files:
print("Sensitive files:")
for path in report.sensitive_files:
Expand All @@ -44,8 +54,8 @@ def print_report(report) -> None:
print("Policy check passed.")


def markdown_report(report) -> str:
status = "Failed" if report.violations else "Passed"
def markdown_report(report, advisory: bool = False) -> str:
status = "Advisory" if advisory and report.violations else "Failed" if report.violations else "Passed"
tests = "yes" if report.tests_changed else "no"
lines = [
"## PR Sheriff report",
Expand All @@ -56,6 +66,31 @@ def markdown_report(report) -> str:
"| ---: | ---: | :---: |",
f"| {report.changed_files} | {report.changed_lines} | {tests} |",
]
if report.score_breakdown:
breakdown = report.score_breakdown
lines.extend(
[
"",
"<details>",
"<summary>Risk score breakdown</summary>",
"",
"| Changed lines | Changed files | Sensitive files | Cap adjustment | Total |",
"| ---: | ---: | ---: | ---: | ---: |",
f"| +{breakdown['changed_lines']} | +{breakdown['changed_files']} | "
f"+{breakdown['sensitive_files']} | {breakdown['cap_adjustment']} | "
f"**{breakdown['total']}** |",
"",
"</details>",
]
)
if report.path_rule_results:
lines.extend(["", "### Matched path rules"])
for result in report.path_rule_results:
status_text = "violated" if result["violations"] else "passed"
lines.append(
f"- **{result['name']}**: {result['changed_files']} files, "
f"{result['changed_lines']} lines ({status_text})"
)
if report.sensitive_files:
lines.extend(["", "### Sensitive files"])
lines.extend(f"- `{path}`" for path in report.sensitive_files)
Expand All @@ -76,17 +111,19 @@ def write_github_output(path: Path, report) -> None:
"changed-files": report.changed_files,
"changed-lines": report.changed_lines,
"tests-changed": str(report.tests_changed).lower(),
"policy-passed": str(not report.violations).lower(),
}
with path.open("a", encoding="utf-8") as output:
for key, value in values.items():
output.write(f"{key}={value}\n")


def print_github_annotations(report) -> None:
def print_github_annotations(report, advisory: bool = False) -> None:
for path in report.sensitive_files:
print(f"::warning file={github_escape(path)}::Sensitive file changed")
for violation in report.violations:
print(f"::error::{github_escape(violation)}")
level = "warning" if advisory else "error"
print(f"::{level}::{github_escape(violation)}")


def main(argv: list[str] | None = None) -> int:
Expand All @@ -111,11 +148,11 @@ def main(argv: list[str] | None = None) -> int:
print_report(report)
if args.github_summary:
with args.github_summary.open("a", encoding="utf-8") as summary:
summary.write(markdown_report(report))
summary.write(markdown_report(report, args.advisory))
if args.github_output:
write_github_output(args.github_output, report)
if args.github_annotations:
print_github_annotations(report)
print_github_annotations(report, args.advisory)
if args.github_comment:
try:
event_path = os.environ.get("GITHUB_EVENT_PATH")
Expand All @@ -126,7 +163,11 @@ def main(argv: list[str] | None = None) -> int:
number = pull_request_number(Path(event_path))
if number:
result = upsert_pull_request_comment(
markdown_report(report), token, repository, number, api_url
markdown_report(report, args.advisory),
token,
repository,
number,
api_url,
)
print(f"PR comment {result}.")
else:
Expand All @@ -135,7 +176,7 @@ def main(argv: list[str] | None = None) -> int:
print("PR comment skipped: GitHub environment is incomplete.")
except (OSError, RuntimeError, ValueError, json.JSONDecodeError) as exc:
print(f"::warning::PR comment skipped: {github_escape(str(exc))}")
return 1 if report.violations else 0
return 1 if report.violations and not args.advisory else 0


if __name__ == "__main__":
Expand Down
Loading
Loading