feat(#568): make protected paths configurable via env var - #569
Conversation
PR Summary by QodoMake protected review paths configurable via REVIEW_PROTECTED_PATHS
AI Description
Diagram
High-Level Assessment
Files changed (4)
|
|
🤖 Review · |
Code Review by Qodo
1.
|
|
🤖 Review · |
|
🤖 Review · |
|
🤖 Review · |
|
🤖 Finished Review · ✅ Success · Started 5:48 PM UTC · Completed 6:08 PM UTC |
ReviewFindingsMedium
Low
Previous runReviewFindingsMedium
Low
Previous run (2)ReviewFindingsMedium
Low
Previous run (3)ReviewFindingsMedium
Low
Previous run (4)ReviewFindingsMedium
Low
Previous run (5)ReviewFindingsMedium
Low
Labels: PR modifies review agent infrastructure (post-review.sh, harness config, skill definitions, eval runner) Next steps:
Previous run (6)ReviewFindingsMedium
Low
Previous run (7)ReviewFindingsMedium
Low
Previous run (8)ReviewFindingsMedium
Low
Previous run (9)ReviewFindingsHigh
Medium
Low
Next steps:
Previous run (10)ReviewFindingsMedium
Low
Previous run (11)ReviewFindingsMedium
Low
Previous run (12)ReviewFindingsHigh
Medium
Low
Labels: PR modifies review agent infrastructure (post-review.sh, SKILL.md, security-triage.md, harness config) Next steps:
|
|
🤖 Finished Fix · ❌ Failure · Started 6:10 PM UTC · Completed 6:19 PM UTC |
|
🤖 Review · |
|
The fix agent completed, but the post-fix script failed before finishing. Workflow run: https://github.com/fullsend-ai/.fullsend/actions/runs/30569103309 Details: |
|
🤖 Review · |
|
🤖 Review · |
|
🤖 Review · |
|
🤖 Review · |
|
🤖 Finished Review · ✅ Success · Started 7:21 PM UTC · Completed 7:42 PM UTC |
Superseded by updated review
waynesun09
left a comment
There was a problem hiding this comment.
Review-only pass (no approve/request-changes). Two additional findings not yet covered by prior review threads or the automated review agent's summary on the current head commit.
|
🤖 Finished Review · ✅ Success · Started 9:09 PM UTC · Completed 9:30 PM UTC |
|
🤖 Finished Fix · ✅ Success · Started 9:31 PM UTC · Completed 9:43 PM UTC |
- Remove empty-string-disables-protection code path from post-review.sh; REVIEW_PROTECTED_PATHS="" now triggers fail-closed instead of silently disabling all protection (addresses High fail-open finding) - Add zero-length check after reading defaults file to prevent silent protection bypass when file contains only comments/blanks - Add env/ to default protected paths so the defaults file itself is protected from unreviewed modification - Update SKILL.md to say "if set and non-empty" for REVIEW_PROTECTED_PATHS to prevent LLM misinterpretation of empty-but-defined env var - Update eval runner to populate REVIEW_PROTECTED_PATHS with defaults from file when caller does not provide a value (avoids empty-string ambiguity) - Update docs/review.md to document fail-closed semantics for empty string - Replace explicit-empty-string-no-downgrade test with explicit-empty-string-aborts - Add file-fallback-comments-only-aborts test for defaults file edge case Addresses review feedback on #569
- Remove empty-string-disables-protection code path from post-review.sh; REVIEW_PROTECTED_PATHS="" now triggers fail-closed instead of silently disabling all protection (addresses High fail-open finding) - Add zero-length check after reading defaults file to prevent silent protection bypass when file contains only comments/blanks - Add env/ to default protected paths so the defaults file itself is protected from unreviewed modification - Update SKILL.md to say "if set and non-empty" for REVIEW_PROTECTED_PATHS to prevent LLM misinterpretation of empty-but-defined env var - Update eval runner to populate REVIEW_PROTECTED_PATHS with defaults from file when caller does not provide a value (avoids empty-string ambiguity) - Update docs/review.md to document fail-closed semantics for empty string - Replace explicit-empty-string-no-downgrade test with explicit-empty-string-aborts - Add file-fallback-comments-only-aborts test for defaults file edge case Addresses review feedback on #569
- docs/review.md: REVIEW_PROTECTED_PATHS description said empty string is "treated the same as unset (fail-closed)" but unset reads defaults while empty string aborts — these are different behaviors. Clarify. - eval case 003 annotation claimed to verify the direct file-read fallback branch in post-review.sh, but the eval harness always pre-populates REVIEW_PROTECTED_PATHS, so only the env-var branch runs. Correct the annotation to describe what's actually exercised. Addresses review feedback on #569 Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
…ECTED_PATHS REVIEW_PROTECTED_PATHS="" now disables protected-path enforcement entirely (deliberate operator opt-out), instead of failing closed. A value that parses to zero entries after trimming (e.g. stray or consecutive commas) still fails closed, since that's more likely a misconfiguration than an intentional opt-out — the abort message now includes the raw value to make that easier to diagnose. Addresses review feedback on #569. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
… tests Two review findings from PR #569: - eval/scripts/run-fullsend.sh: when the default protected-paths file is missing, emit_env'ing REVIEW_PROTECTED_PATHS="" gets interpreted by post-review.sh as a deliberate opt-out (protection disabled) rather than a fail-closed error. Exit 1 instead, matching post-review.sh's own missing-file guard. - scripts/post-review-test.sh: run_protected_paths_test only exported REVIEW_PROTECTED_PATHS when the test's protected_paths argument was non-empty, so file-fallback tests could silently inherit a stale value from the calling environment instead of exercising the defaults-file path. Unset it explicitly in the empty-argument case, with a new test that reproduces the leak. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
agents/code.md and agents/fix.md said protected paths are "defined in" post-review.sh, but the list is configured via REVIEW_PROTECTED_PATHS in harness/review.yaml and only enforced by post-review.sh. Clarify that split. Also rename post-review.sh's leading-underscore locals (_trimmed, _entry, _sanitized_paths) to match the file's existing naming convention, per review feedback on PR #569. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
Rename the internal PROTECTED_PATHS array to REVIEW_ACTIVE_PROTECTED_PATHS to follow the codebase's namespace-qualified naming convention (e.g. REVIEW_CONTROL_LABELS) and to avoid reading like the REVIEW_PROTECTED_PATHS env var it's derived from, per review feedback on PR #569. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
REVIEW_PROTECTED_PATHS' default is now hardcoded verbatim in three places (harness/review.yaml's env.runner and env.sandbox, and this test file) with no structural source of truth since env/default-review-protected-paths.txt was removed. Add a test that compares this file's default against harness/review.yaml via yq so a future edit that updates one copy and misses another fails loudly instead of silently testing against a stale default. Skips (doesn't fail) when yq is unavailable, per review feedback on PR #569. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
- Remove empty-string-disables-protection code path from post-review.sh; REVIEW_PROTECTED_PATHS="" now triggers fail-closed instead of silently disabling all protection (addresses High fail-open finding) - Add zero-length check after reading defaults file to prevent silent protection bypass when file contains only comments/blanks - Add env/ to default protected paths so the defaults file itself is protected from unreviewed modification - Update SKILL.md to say "if set and non-empty" for REVIEW_PROTECTED_PATHS to prevent LLM misinterpretation of empty-but-defined env var - Update eval runner to populate REVIEW_PROTECTED_PATHS with defaults from file when caller does not provide a value (avoids empty-string ambiguity) - Update docs/review.md to document fail-closed semantics for empty string - Replace explicit-empty-string-no-downgrade test with explicit-empty-string-aborts - Add file-fallback-comments-only-aborts test for defaults file edge case Addresses review feedback on fullsend-ai#569
…cted paths - docs/review.md: REVIEW_PROTECTED_PATHS description said empty string is "treated the same as unset (fail-closed)" but unset reads defaults while empty string aborts — these are different behaviors. Clarify. - eval case 003 annotation claimed to verify the direct file-read fallback branch in post-review.sh, but the eval harness always pre-populates REVIEW_PROTECTED_PATHS, so only the env-var branch runs. Correct the annotation to describe what's actually exercised. Addresses review feedback on fullsend-ai#569 Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
…REVIEW_PROTECTED_PATHS REVIEW_PROTECTED_PATHS="" now disables protected-path enforcement entirely (deliberate operator opt-out), instead of failing closed. A value that parses to zero entries after trimming (e.g. stray or consecutive commas) still fails closed, since that's more likely a misconfiguration than an intentional opt-out — the abort message now includes the raw value to make that easier to diagnose. Addresses review feedback on fullsend-ai#569. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
…ient env in tests Two review findings from PR fullsend-ai#569: - eval/scripts/run-fullsend.sh: when the default protected-paths file is missing, emit_env'ing REVIEW_PROTECTED_PATHS="" gets interpreted by post-review.sh as a deliberate opt-out (protection disabled) rather than a fail-closed error. Exit 1 instead, matching post-review.sh's own missing-file guard. - scripts/post-review-test.sh: run_protected_paths_test only exported REVIEW_PROTECTED_PATHS when the test's protected_paths argument was non-empty, so file-fallback tests could silently inherit a stale value from the calling environment instead of exercising the defaults-file path. Unset it explicitly in the empty-argument case, with a new test that reproduces the leak. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
agents/code.md and agents/fix.md said protected paths are "defined in" post-review.sh, but the list is configured via REVIEW_PROTECTED_PATHS in harness/review.yaml and only enforced by post-review.sh. Clarify that split. Also rename post-review.sh's leading-underscore locals (_trimmed, _entry, _sanitized_paths) to match the file's existing naming convention, per review feedback on PR fullsend-ai#569. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
Rename the internal PROTECTED_PATHS array to REVIEW_ACTIVE_PROTECTED_PATHS to follow the codebase's namespace-qualified naming convention (e.g. REVIEW_CONTROL_LABELS) and to avoid reading like the REVIEW_PROTECTED_PATHS env var it's derived from, per review feedback on PR fullsend-ai#569. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
REVIEW_PROTECTED_PATHS' default is now hardcoded verbatim in three places (harness/review.yaml's env.runner and env.sandbox, and this test file) with no structural source of truth since env/default-review-protected-paths.txt was removed. Add a test that compares this file's default against harness/review.yaml via yq so a future edit that updates one copy and misses another fails loudly instead of silently testing against a stale default. Skips (doesn't fail) when yq is unavailable, per review feedback on PR fullsend-ai#569. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
Summary
REVIEW_PROTECTED_PATHSenvironment variable to override the hardcoded protected-path list inpost-review.sh. Comma-separated path prefixes, whitespace-trimmed.REVIEW_FINDING_SEVERITY_THRESHOLD's wiring intoharness/review.yaml's runner/sandbox env predates this branch. This PR's only change to that file is adding the twoREVIEW_PROTECTED_PATHSentries.env/default-review-protected-paths.txtfile thatpost-review.sh,run-fullsend.sh, andSKILL.mdeach had to independently resolve via a three-way (set / set-empty / unset) ladder. That's now replaced with a single literal default declared directly inharness/review.yaml'senv.runner/env.sandboxstanzas (matching the existing constant-value pattern already used for things likeMAX_RETRIESinharness/fix.yaml). Repos needing a different list override it via harness composition instead of an env var. Unset is now a hard misconfiguration error rather than a file-read fallback.env/default-review-protected-paths.txt.post-review.sh: collapsed to two cases (non-empty / explicitly-empty).run-fullsend.sh: removed the now-dead default-computation block.SKILL.md,docs/review.md, and the003-protected-path-downgradeeval case's annotations to match.skills/pr-review/SKILL.mdanddocs/review.mdto document the variable.Closes #568
Test plan
bash scripts/post-review-test.sh)bash scripts/validate-output-schema-test.sh)🤖 Generated with Claude Code