feat: add generic refine agent - #86
Conversation
|
🤖 Finished Review · ✅ Success · Started 5:46 PM UTC · Completed 5:57 PM UTC |
PR Summary by QodoAdd platform-agnostic refine agent with harness, policy, schema, and scripts
AI Description
Diagram
High-Level Assessment
Files changed (14)
|
Code Review by Qodo
1.
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
ReviewChanges since prior reviewThe 2 commits since Key changes:
FindingsHigh
Medium
Low
Next steps:
Previous runReviewChanges since prior reviewThe commit since
FindingsHigh
Medium
Low
Next steps:
Previous run (2)ReviewChanges since prior reviewThe commit since
FindingsHigh
Medium
Low
Next steps:
Previous run (3)ReviewChanges since prior reviewThe commit since
FindingsHigh
Medium
Low
Next steps:
Previous run (4)ReviewChanges since prior reviewThe commit since
FindingsHigh
Medium
Low
Next steps:
Previous run (5)ReviewChanges since prior reviewThe commit since
FindingsHigh
Medium
Low
Next steps:
Previous run (6)ReviewChanges since prior reviewThe commits since
FindingsHigh
Medium
Low
Next steps:
Previous run (7)ReviewChanges since prior reviewThe commits since
FindingsHigh
Medium
Low
Next steps:
Previous run (8)ReviewChanges since prior reviewThe commits since
FindingsHigh
Medium
Low
Notes
Next steps:
Previous run (9)ReviewFindingsHigh
Medium
Low
Notes
|
![fullsend-ai-review[bot]](https://avatars.githubusercontent.com/in/3478501?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
🤖 Finished Review · ✅ Success · Started 6:12 PM UTC · Completed 6:23 PM UTC |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
🤖 Finished Review · ✅ Success · Started 7:57 PM UTC · Completed 8:10 PM UTC |
Superseded by updated review
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
fullsend-ai-review
Bot
added
requires-manual-review
There was a problem hiding this comment.
Same deal as the other PR, move this to the harness
rh-hemartin
left a comment
rh-hemartin
left a comment
There was a problem hiding this comment.
I want to move some envs to the harness file.
Update for konflux-ci/refinement
|
|
🤖 Review · |
7a404e0 to
ab73ea1
Compare
CI fix pass (
|
|
🤖 Review · |
Introduce refine harness, agent prompt, scripts, and schema. Consume optional ORG_KNOWLEDGE for install-injected org context without failing when the pack is absent. Resolve companion script helpers for base-composition installs. Signed-off-by: Adam Scerra <ascerra@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com>
ab73ea1 to
e31798d
Compare
|
Rebased onto latest |
|
🤖 Finished Review · ✅ Success · Started 8:13 PM UTC · Completed 8:29 PM UTC |
Stop treating every assumption as a reply-required Open Question. Require resolution=needs_human|research_spike|assumed_default, render sticky sections accordingly, and ban triple-booking assume+spike+ask. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
🤖 Finished Review · ✅ Success · Started 9:19 PM UTC · Completed 9:38 PM UTC |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
🤖 Finished Review · ✅ Success · Started 1:12 AM UTC · Completed 1:30 AM UTC |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Phase 0 duplicate gate emits blocked_duplicate with empty children and skips ready-to-critique. Second /fs-refine after the sticky marker overrides. Syncs comment-helpers + explore companions for same-commit fetch. Co-authored-by: Cursor <cursoragent@cursor.com>
|
🤖 Finished Review · ✅ Success · Started 11:51 AM UTC · Completed 12:12 PM UTC |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
🤖 Finished Review · ✅ Success · Started 8:19 PM UTC · Completed 8:38 PM UTC |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Phase 0 cats /tmp/workspace/duplicate-gate.json; also try the sandbox mount path and report clearly when neither exists so override=true is not silently missed. Signed-off-by: Adam Scerra <ascerra@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com>
|
🤖 Finished Review · ✅ Success · Started 9:44 PM UTC · Completed 10:06 PM UTC |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Align refine (and explore schema copy) with 0.0–5.0 confidence and issue_id/project.id naming so ticket IDs survive secret redaction. Assisted-by: Cursor Signed-off-by: Adam Scerra <ascerra@redhat.com>
|
🤖 Finished Review · ✅ Success · Started 5:04 PM UTC · Completed 5:21 PM UTC Commit: |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Companion fetch must match explore tip so refine runs do not regress rate-limit / private-repo guards. Signed-off-by: Adam Scerra <ascerra@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com>
|
🤖 Review · Commit: |
Ask Story/Task children for 1–3 concrete repo file/package anchors from explore evidence (REPO-INDEX / accessed sources), never invented paths. Signed-off-by: Adam Scerra <ascerra@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com>
|
🤖 Finished Review · ✅ Success · Started 11:14 PM UTC · Completed 11:36 PM UTC Commit: |
| url = mo.group(0).rstrip(").,;]") | ||
| return f"[{url}]({url})" | ||
|
|
||
| out = re.sub(r"https?://[^\s<>{}\"|\\^`\[\]]+", repl_url, out) |
There was a problem hiding this comment.
[medium] double-linkify-corruption
In the linkify function generic fallback path, repl_url runs on the output of repl_jira, double-wrapping URLs inside markdown links and producing corrupted markdown.
Suggested fix: Run repl_url before repl_jira, or make repl_url skip URLs already inside markdown link syntax.
| } | ||
|
|
||
| RESULT_FILE="" | ||
| for dir in iteration-*/output; do |
There was a problem hiding this comment.
[medium] result-file-discovery-pattern
Result file discovery uses only the iteration-*/output glob fallback. Every other post-script first checks FULLSEND_VALIDATED_ITERATION_DIR as a trust boundary.
Suggested fix: Add the standard FULLSEND_VALIDATED_ITERATION_DIR block before the glob fallback.
| agent: agents/refine.md | ||
| doc: docs/refine.md | ||
| model: opus | ||
| image: ghcr.io/fullsend-ai/fullsend-sandbox:latest |
There was a problem hiding this comment.
[medium] image-not-pinned
All existing harness definitions pin the sandbox image by digest (@sha256:...). The refine harness uses :latest, which is non-reproducible and breaks the codebase convention for deterministic builds.
Suggested fix: Pin the image to a specific sha256 digest.
| for c in "${CLONED[@]+"${CLONED[@]}"}"; do | ||
| [[ "$c" == "$ref" ]] && already=true && break | ||
| done | ||
| $already || CLONED+=("$ref") |
There was a problem hiding this comment.
[low] gha-command-injection
The ::notice:: command interpolates ${CLONED[*]} without sanitization. Although GitHub org/repo naming rules make injection impractical, the codebase convention is to sanitize all values interpolated into GHA workflow commands.
| extract_repo_refs_from_text() { | ||
| TEXT_CONTENT=$(jq -r ' | ||
| [.description // "", (.comments // [] | .[].body // "")] | join("\n") | ||
| ' "$WORKSPACE/issue-context.json") |
There was a problem hiding this comment.
[low] gha-command-injection
The ::warning:: in validate_repo interpolates ${ref} without sanitization.
| if [[ "${ISSUE_SOURCE:-}" == "jira" && -n "${JIRA_HOST:-}" && -n "${JIRA_EMAIL:-}" && -n "${JIRA_API_TOKEN:-}" ]]; then | ||
| _jira_auth=$(printf '%s:%s' "$JIRA_EMAIL" "$JIRA_API_TOKEN" | base64 -w0) | ||
| _probe_codes="" | ||
| for _proj in KFLUXUI KFLUXSE STONEINTG; do |
There was a problem hiding this comment.
[low] hardcoded-project-keys
Jira API hints block probes three hardcoded project keys specific to a single Jira instance.
Closing without merge — keep the branchClosing #86 (refine). Refine is not cancelled. Same situation as #11: an open draft PR used as a pin target meant every branch update re-triggered fullsend review. Extending Team decision 2026-08-10 (notes). Depends on explore shared scripts from #11 ( |
|
🤖 Finished Retro · ✅ Success · Started 12:41 AM UTC · Completed 12:52 AM UTC Commit: |
Retro: PR #86 — feat: add generic refine agentOutcome: Closed without merge after 33 days and 19 review agent runs. Timeline
Review quality assessmentEarly rounds were high-value. The review agent identified genuine correctness bugs, security issues, and schema deficiencies — 11 findings that the author fixed. The challenger sub-agent and multi-dimension approach worked well for first-pass review. Later rounds delivered diminishing returns. Rounds 4–19 produced ~85 additional inline findings, but nearly all were repeats of the same ~9 persisting issues. The author had responded to each with explanations ("Acknowledged: intentional," "Fixed earlier," "Depends on PR #11"), but the review agent did not incorporate these responses. This noise-to-signal ratio degraded trust and ultimately caused the author to close the PR. One human-only catch. rh-hemartin identified a repo convention (env vars should live in the harness config, not a separate env file) that the review agent missed. This convention is not documented in AGENTS.md. CI failures went unaddressed. The CI Evidence for existing issues (no new proposals filed)All problems identified in this retro are already tracked by existing open issues. PR #86 provides additional evidence for the severity and user impact of the review dedup/noise cluster:
|
2 participants
Summary
comment-helpers.sh,pre-explore.sh, andmarkdown-to-adf.pyRelated Issue
Continuation of agent migration from
konflux-ci/refinementto generic agents repo.Changes
agents/refine.mdharness/refine.yamlenv.runner/env.sandboxformat, human-directive mountpolicies/refine.yamlschemas/refine-result.schema.jsonscripts/pre-refine.shscripts/post-refine.shscripts/post-refine-test.shscripts/platform-{jira,github,gitlab}.mdenv/refine.envdocs/refine.mdconfig.yamlharness/refine.yamlREADME.mdTesting
bash scripts/post-refine-test.sh— 29/29 tests passingChecklist
PUTverb blocked indisallowedToolsenv.runner/env.sandboxformatMade with Cursor