Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
state: open

labels:
required: []
forbidden:
- risk/elevated
- risk/high
- risk/critical
risk_label_required: true

findings:
required: []
forbidden:
- file-naming-convention

max_turns: 40
max_cost_usd: 4.00

review_expectations: |
The PR adds `.codecov.yml`, a tool-owned configuration filename that
Codecov recognizes. Although existing root-level YAML files use `.yaml`,
the review must not recommend renaming `.codecov.yml` for consistency.
The review should approve or comment without a file-naming-convention
finding.
18 changes: 18 additions & 0 deletions eval/review/cases/004-tool-owned-codecov-filename/input.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
forge: github

fixture:
type: pull_request
title: "ci: add Codecov configuration"
body: |
Add repository-level Codecov thresholds.
files:
- path: .codecov.yml
content: |
coverage:
status:
project:
default:
target: auto
patch:
default:
target: 80%
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Codecov fixture

This repository uses `.yaml` for project-owned YAML files.
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
name: example
environment: production
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
channel: stable
enabled: true
22 changes: 22 additions & 0 deletions eval/review/eval.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,26 @@ judges:
return False, f"Forbidden labels present: {present} (actual: {actual})"
return True, f"No forbidden labels found (checked: {forbidden})"

- name: finding_expectations
description: Required findings must appear and forbidden findings must not appear
check: |
import json
raw = outputs["files"].get("output/fixture-state.json")
if not raw:
return False, "fixture-state.json not found"
state = json.loads(raw)
bodies = [item.get("body", "") for item in state.get("comments", [])]
bodies.extend(item.get("body", "") for item in state.get("reviews", []))
review_text = "\n".join(bodies).lower()
expected = outputs.get("annotations", {}).get("findings", {})
required = [finding.lower() for finding in expected.get("required", [])]
forbidden = [finding.lower() for finding in expected.get("forbidden", [])]
missing = [finding for finding in required if f"[{finding}]" not in review_text]
present = [finding for finding in forbidden if f"[{finding}]" in review_text]
if missing or present:
return False, f"Finding mismatch: missing={missing}, forbidden_present={present}"
return True, f"Finding expectations met: required={required}, forbidden={forbidden}"

- name: risk_label_present
description: When risk_label_required is set, at least one risk/* label must be present
check: |
Expand Down Expand Up @@ -212,6 +232,8 @@ thresholds:
min_pass_rate: 1.0
forbidden_labels:
min_pass_rate: 1.0
finding_expectations:
min_pass_rate: 1.0
max_turns:
min_pass_rate: 1.0
max_cost:
Expand Down
12 changes: 12 additions & 0 deletions skills/pr-review/sub-agents/style-conventions.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,18 @@ not from general best practices.

**Do not own:** Logic correctness, security, documentation content/staleness.

## Tool-owned filenames

Treat filenames recognized by external tools as compatibility contracts. Do
Comment thread
qodo-code-review[bot] marked this conversation as resolved.
not recommend renaming a tool configuration file solely to match repository
conventions. Only raise a filename-convention finding when repository evidence
confirms that the proposed alternative is supported; if support cannot be
established with the available tools, report no finding.
Comment thread
shairevivo marked this conversation as resolved.

Examples include `.codecov.yml`, `.eslintrc.yml`, `.prettierrc`,
`.editorconfig`, `Dockerfile`, and `Makefile`. This list is illustrative, not
exhaustive.

## Exploration budget

Before exploring context files, assess the diff size and nature.
Expand Down
Loading