Skip to content

fix(skills): correct using-agent-relay drift and stale MCP names in sdk rule - #1423

Open
willwashburn wants to merge 2 commits into
mainfrom
claude/agent-relay-skills-review-a8mgtd-skill-drift
Open

fix(skills): correct using-agent-relay drift and stale MCP names in sdk rule#1423
willwashburn wants to merge 2 commits into
mainfrom
claude/agent-relay-skills-review-a8mgtd-skill-drift

Conversation

@willwashburn

@willwashburn willwashburn commented Aug 3, 2026

Copy link
Copy Markdown
Member

Summary

Both vendored copies of using-agent-relay (.claude/skills/ and .agents/skills/) had drifted from the published skill, in four ways that all mislead an agent reading them:

  1. A dead security warning was suppressing working content. The CLI reference omits every startup and status command, justified by: "Published Agent Relay versions through 11.3.0 can print live workspace credentials … Upgrade to Agent Relay 11.3.1 or later." This repo is on 11.4.0 and 11.3.1 shipped, so the note describes a fixed defect while still hiding commands agents need.
  2. The MCP tool prefix was wrong. It documented mcp__relaycast__send_dm and mcp__agent_relay__send_dm. The prefix comes from the configured server key, which is agent-relay, so the real decorated form is mcp__agent-relay__send_dm.
  3. list_dms was missing from the tool table — while orchestrating-agent-relay tells agents to use exactly that tool to re-read consumed DM history.
  4. The base URL example was gateway.relaycast.dev, the legacy host. Every default in this repo and in relaycast is cast.agentrelay.com.

The published version is correct on all four and is a strict superset of the vendored CLI reference — it also documents message inbox mark_read, message dm list, message get_thread, channel list, node up, node status, and the deprecated local alias. I diffed both directions before syncing; the vendored copies had nothing unique except the stale warning.

Bonus: the same drift in a rule file

.claude/rules/sdk.md listed these as the "Primary" communication protocol:

mcp__relaycast__message_dm_send, mcp__relaycast__message_inbox_check,
mcp__relaycast__agent_list, mcp__relaycast__agent_add, mcp__relaycast__agent_remove

Those are the old category-expanded names that using-agent-relay explicitly tells agents not to use, and agent-relay mcp does not register any of them — the registered names are flat (send_dm, check_inbox, list_agents, add_agent, remove_agent, verified in agent-relay-mcp.ts and mcp/messaging-tools.ts). Two files in this repo were giving agents opposite instructions, and the rule file was the wrong one. Corrected.

Not addressed here — needs a decision

Five more skills exist in three copies (skills repo, .claude/skills/, .agents/skills/) with three different hashes each. writing-agent-relay-workflows is 2277 / 2093 / 1523 lines; even after normalizing markdown syntax, the .claude and .agents copies differ by ~630 lines. That is content divergence, not formatting.

I fixed using-agent-relay here because it was unambiguous — the published copy is strictly better on every point. The rest needs a call I did not want to make unilaterally, because the vendored copies are newer by commit date but worse in content, so neither side is simply canonical:

  • Make the skills repo the single source and generate the two vendored trees from it in CI, or
  • Accept the divergence and delete the vendored copies, relying on prpm install, or
  • Keep hand-maintaining all three.

Happy to implement whichever you prefer as a follow-up.

Part of a series from a review of the Agent Relay skills and plugins.

Test Plan

  • Confirmed package.json is at 11.4.0, so the 11.3.0 warning is obsolete
  • Confirmed the MCP server key is agent-relay (plugin .mcp.json) and that the registered tool names are flat, by grepping the registrations in agent-relay-mcp.ts and mcp/messaging-tools.ts
  • Confirmed cast.agentrelay.com is the default engine URL across both repos
  • Diffed the vendored copies against the published skill in both directions to confirm nothing unique was dropped
  • Verified .claude/skills/ and .agents/skills/ copies are byte-identical after the sync
  • Tests added/updated — n/a, documentation and agent-instruction files with no code or build dependency

Screenshots

n/a


Generated by Claude Code

Review in cubic

Both vendored copies of `using-agent-relay` had drifted from the
published skill and were wrong in four ways:

- **A dead security warning was suppressing content.** The CLI reference
  omitted every startup and status command because "Published Agent Relay
  versions through 11.3.0 can print live workspace credentials … Upgrade
  to Agent Relay 11.3.1 or later." This repo is on 11.4.0 and 11.3.1
  shipped, so the warning describes a fixed defect while still hiding
  working commands from agents.
- **The MCP tool prefix was wrong.** It documented
  `mcp__relaycast__send_dm` and `mcp__agent_relay__send_dm`. The prefix
  comes from the server key, which is `agent-relay`, so the real form is
  `mcp__agent-relay__send_dm`.
- **`list_dms` was missing** from the tool table, even though
  `orchestrating-agent-relay` tells agents to use it to re-read consumed
  DM history.
- **The base URL example was the legacy `gateway.relaycast.dev` host.**
  Every default in this repo and in relaycast is `cast.agentrelay.com`.

The published version is correct on all four and is a strict superset of
the vendored CLI reference (it also documents `message inbox mark_read`,
`message dm list`, `message get_thread`, `channel list`, `node up`,
`node status`, and the deprecated `local` alias). Both copies are synced
to it; nothing unique was lost.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jmke9G9s7ftrN49opNmdx1
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The changes align Agent Relay documentation with canonical flat MCP tool names, add list_dms, and replace the CLI reference with participant messaging, workspace, broker, environment, and durable-message guidance.

Changes

Agent Relay documentation

Layer / File(s) Summary
Canonical MCP tool naming
.agents/skills/using-agent-relay/SKILL.md, .claude/skills/using-agent-relay/SKILL.md, .claude/rules/sdk.md
The documentation uses flat MCP names, optional server-key prefixes, and bare names for Codex and opencode.
Messaging and broker workflows
.agents/skills/using-agent-relay/SKILL.md, .claude/skills/using-agent-relay/SKILL.md
The guidance adds list_dms and documents participant messaging, workspace identity, broker lifecycle, environment variables, and durable-message retrieval commands.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related PRs

Suggested reviewers: khaliqgant

Poem

A rabbit updates names in a row,
Flat tools now tell agents where to go.
DMs list, brokers start,
Durable messages play their part.
Clear relay guidance helps work flow.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the MCP naming corrections and Agent Relay skill drift fixes.
Description check ✅ Passed The description explains the changes, provides verification details, and documents that tests are not applicable.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/agent-relay-skills-review-a8mgtd-skill-drift

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.agents/skills/using-agent-relay/SKILL.md:
- Around line 201-209: Remove inline workspace keys and agent tokens from the
messaging command examples, replacing them with the documented
RELAY_WORKSPACE_KEY and RELAY_AGENT_TOKEN environment variables in
.agents/skills/using-agent-relay/SKILL.md lines 201-209 and
.claude/skills/using-agent-relay/SKILL.md lines 201-209; apply the same
correction to every listed command while preserving their arguments and
behavior.
- Line 204: Replace the angle-bracket conversation identifier placeholder in the
`agent-relay message dm list` example with the shell-safe `CONVERSATION_ID`
placeholder at `.agents/skills/using-agent-relay/SKILL.md:204-204` and apply the
same change at `.claude/skills/using-agent-relay/SKILL.md:204-204`.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1492dc5b-bdee-475c-b21d-0c3261dd29ac

📥 Commits

Reviewing files that changed from the base of the PR and between 86631da and 848cf17.

📒 Files selected for processing (3)
  • .agents/skills/using-agent-relay/SKILL.md
  • .claude/rules/sdk.md
  • .claude/skills/using-agent-relay/SKILL.md

Comment on lines 201 to +209
agent-relay message inbox check --workspace-key rk_live_... --token at_live_...
agent-relay message inbox mark_read msg_123 --workspace-key rk_live_... --token at_live_...
agent-relay message dm send Lead "ACK: I am online." --workspace-key rk_live_... --token at_live_...
agent-relay message dm list <conversationId> --workspace-key rk_live_... --token at_live_... # persistent DM history (unlike unread-only inbox check)
agent-relay message post general "Status update" --workspace-key rk_live_... --token at_live_...
agent-relay message list general --workspace-key rk_live_... --token at_live_...
agent-relay message reply msg_123 "Thread reply" --workspace-key rk_live_... --token at_live_...
agent-relay message get_thread msg_123 --workspace-key rk_live_... --token at_live_...
agent-relay channel list --workspace-key rk_live_... --token at_live_...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Keep credentials out of command-line examples.

The messaging examples place workspace keys and agent tokens in process arguments. Use the documented environment variables instead.

  • .agents/skills/using-agent-relay/SKILL.md#L201-L209: Rewrite the examples to use RELAY_WORKSPACE_KEY and RELAY_AGENT_TOKEN.
  • .claude/skills/using-agent-relay/SKILL.md#L201-L209: Apply the same credential-handling correction.
🧰 Tools
🪛 SkillSpector (2.4.4)

[warning] 21: [AS3] Skill Enumeration: Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Remediation: Remove all code or instructions that list or read other skills' files or directories. Skills should operate independently; cross-skill access is a privilege escalation.

(Agent Snooping (AS3))


[warning] 54: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 75: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 91: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 93: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 94: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 95: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 96: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 99: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 248: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 250: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 251: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 252: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 254: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 255: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 256: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))

📍 Affects 2 files
  • .agents/skills/using-agent-relay/SKILL.md#L201-L209 (this comment)
  • .claude/skills/using-agent-relay/SKILL.md#L201-L209
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.agents/skills/using-agent-relay/SKILL.md around lines 201 - 209, Remove
inline workspace keys and agent tokens from the messaging command examples,
replacing them with the documented RELAY_WORKSPACE_KEY and RELAY_AGENT_TOKEN
environment variables in .agents/skills/using-agent-relay/SKILL.md lines 201-209
and .claude/skills/using-agent-relay/SKILL.md lines 201-209; apply the same
correction to every listed command while preserving their arguments and
behavior.

agent-relay message inbox check --workspace-key rk_live_... --token at_live_...
agent-relay message inbox mark_read msg_123 --workspace-key rk_live_... --token at_live_...
agent-relay message dm send Lead "ACK: I am online." --workspace-key rk_live_... --token at_live_...
agent-relay message dm list <conversationId> --workspace-key rk_live_... --token at_live_... # persistent DM history (unlike unread-only inbox check)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Use a shell-safe conversation identifier placeholder.

Bash interprets <conversationId> as redirection syntax.

  • .agents/skills/using-agent-relay/SKILL.md#L204-L204: Replace <conversationId> with CONVERSATION_ID.
  • .claude/skills/using-agent-relay/SKILL.md#L204-L204: Apply the same placeholder correction.
🧰 Tools
🪛 SkillSpector (2.4.4)

[warning] 21: [AS3] Skill Enumeration: Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Remediation: Remove all code or instructions that list or read other skills' files or directories. Skills should operate independently; cross-skill access is a privilege escalation.

(Agent Snooping (AS3))


[warning] 54: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 75: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 91: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 93: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 94: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 95: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 96: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 99: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 248: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 250: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 251: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 252: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 254: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 255: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))


[warning] 256: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.

(Memory Poisoning (MP2))

📍 Affects 2 files
  • .agents/skills/using-agent-relay/SKILL.md#L204-L204 (this comment)
  • .claude/skills/using-agent-relay/SKILL.md#L204-L204
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.agents/skills/using-agent-relay/SKILL.md at line 204, Replace the
angle-bracket conversation identifier placeholder in the `agent-relay message dm
list` example with the shell-safe `CONVERSATION_ID` placeholder at
`.agents/skills/using-agent-relay/SKILL.md:204-204` and apply the same change at
`.claude/skills/using-agent-relay/SKILL.md:204-204`.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants