Skip to content

TRT-2887: remove analyze-disruption command to prevent model freelancing - #683

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift-eng:mainfrom
smg247:fix/slim-disruption-command
Aug 6, 2026
Merged

openshift-merge-bot[bot] merged 1 commit into
openshift-eng:mainfrom
smg247:fix/slim-disruption-command

Conversation

@smg247

@smg247 smg247 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Follows up on #679

The verbose command file gave the model enough context to improvise with curl/WebFetch instead of loading and following the SKILL.md steps. Skills are slash-invokable so the command isn't needed.

Summary by CodeRabbit

  • Documentation
    • Simplified the ci:analyze-disruption command synopsis.
    • Clarified that the command loads the analysis skill and forwards provided arguments.
    • Updated supported inputs to include Prow job or Grafana URLs, optional backend selection, and JIRA skipping.
    • Refreshed CI plugin metadata to version 0.0.82.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Aug 6, 2026
@openshift-ci-robot

openshift-ci-robot commented Aug 6, 2026

Copy link
Copy Markdown

@smg247: This pull request references TRT-2887 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Follows up on #679

The verbose command file gave the model enough context to improvise with curl/WebFetch instead of loading and following the SKILL.md steps. Stripped to bare minimum: frontmatter, synopsis, and a directive to load the skill.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 6, 2026
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

The CI plugin version changes from 0.0.81 to 0.0.82. The ci:analyze-disruption command is removed from the documentation index and its command documentation is deleted.

Changes

CI plugin update

Layer / File(s) Summary
Update plugin metadata and remove command
.claude-plugin/marketplace.json, plugins/ci/.claude-plugin/plugin.json, docs/index.html
The CI plugin version changes to 0.0.82. The documentation index no longer defines ci:analyze-disruption. The command documentation is removed from plugins/ci/commands/analyze-disruption.md.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related PRs

Suggested reviewers: brandisher, theobarberbany, stleerh

🚥 Pre-merge checks | ✅ 9 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Ai-Helpers Overlap Detection ❓ Inconclusive Evidence collection is still in progress. Inspect the current command, related skill, existing content, and open PRs before deciding.
✅ Passed checks (9 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
No Real People Names In Style References ✅ Passed The commit adds only version metadata and deletes the command; neither the deleted command nor the surviving analyze-disruption skill contains a real person's name or style reference.
No Assumed Git Remote Names ✅ Passed The PR only bumps CI versions and removes the disruption command; no added line contains origin, upstream, or a hardcoded Git remote operation.
Git Push Safety Rules ✅ Passed The PR only updates plugin versions and removes the command; added lines contain no git push, force-push, main, or master instructions.
No Untrusted Mcp Servers ✅ Passed The HEAD^..HEAD diff adds only CI version strings; it adds no MCP server, package installation, repository URL, or dependency, and changes no MCP config paths.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the removed command and states the primary reason for the change.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/ci/commands/analyze-disruption.md`:
- Around line 11-19: Update the analyze-disruption command synopsis to
explicitly show either one or more Prow job URLs or a single Grafana disruption
dashboard URL, without implying mixed or repeated URL types beyond the skill
contract. Revise the description to mention both supported inputs, then
regenerate docs/index.html to reflect the updated command documentation.
- Around line 11-19: Update the Skillsaw command-schema validation for the
analyze-disruption command to require the Name, Synopsis, Description, and
Implementation sections when that structure is mandated. Ensure validation
covers command documents such as analyze-disruption.md without altering the
existing command or skill paths and argument handling.
- Around line 18-19: Add the required overlap report to the analyze-disruption
command change, documenting PR `#655` and
plugins/ci/skills/prow-job-analysis/SKILL.md as LOW overlap, excluding PR `#683`,
and explicitly reporting no HIGH or MODERATE overlaps.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 688f33ad-655e-40e0-8653-3d6b04d5ccf1

📥 Commits

Reviewing files that changed from the base of the PR and between 7fba9b0 and 1918df2.

📒 Files selected for processing (2)
  • docs/index.html
  • plugins/ci/commands/analyze-disruption.md

Comment thread plugins/ci/commands/analyze-disruption.md Outdated
Comment thread plugins/ci/commands/analyze-disruption.md Outdated
@smg247
smg247 force-pushed the fix/slim-disruption-command branch from 1918df2 to b037275 Compare August 6, 2026 10:47
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

Comment thread plugins/ci/commands/analyze-disruption.md Outdated
…lancing

Skills are directly slash-invokable, so the command file was redundant.
Worse, its description gave the model enough context to improvise with
curl/WebFetch instead of loading and following the SKILL.md steps.
Removing the command file means /ci:analyze-disruption loads the
SKILL.md directly with no intermediary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@smg247
smg247 force-pushed the fix/slim-disruption-command branch from b037275 to 2438fcd Compare August 6, 2026 11:19
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@smg247 smg247 changed the title TRT-2887: slim down analyze-disruption command to prevent model freelancing TRT-2887: remove analyze-disruption command to prevent model freelancing Aug 6, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs/index.html (1)

554-554: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Keep the ci:analyze-disruption entry in the generated documentation.

plugins/ci/skills/analyze-disruption/SKILL.md still defines the skill, and PLUGINS.md still advertises /ci:analyze-disruption. Deleting this entry removes a supported command from the generated documentation. Restore the entry and update its body to direct users to SKILL.md, or remove the command from the plugin and all other documentation in the same change.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/index.html` at line 554, Restore the ci:analyze-disruption entry in the
generated documentation and update its content to direct users to
plugins/ci/skills/analyze-disruption/SKILL.md, preserving the documented
supported command.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@docs/index.html`:
- Line 554: Restore the ci:analyze-disruption entry in the generated
documentation and update its content to direct users to
plugins/ci/skills/analyze-disruption/SKILL.md, preserving the documented
supported command.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 46265c3f-89c2-473d-9052-3ed1d73ddac1

📥 Commits

Reviewing files that changed from the base of the PR and between 838106a and 2438fcd.

📒 Files selected for processing (4)
  • .claude-plugin/marketplace.json
  • docs/index.html
  • plugins/ci/.claude-plugin/plugin.json
  • plugins/ci/commands/analyze-disruption.md
💤 Files with no reviewable changes (1)
  • plugins/ci/commands/analyze-disruption.md
🚧 Files skipped from review as they are similar to previous changes (2)
  • .claude-plugin/marketplace.json
  • plugins/ci/.claude-plugin/plugin.json

@stbenjam

stbenjam commented Aug 6, 2026

Copy link
Copy Markdown
Member

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Aug 6, 2026
@openshift-ci

openshift-ci Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: smg247, stbenjam

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot
openshift-merge-bot Bot merged commit 2ab1887 into openshift-eng:main Aug 6, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants