security: add user confirmation guard to meta-skill installation flow#202
security: add user confirmation guard to meta-skill installation flow#202sjhddh wants to merge 3 commits intoHKUDS:mainfrom
Conversation
When a harness has no README.md, skill_intro is an empty string. The previous code unconditionally appended " - ..." producing malformed output like "Command-line interface for TestApp - ..." in SKILL.md YAML frontmatter. Also fixed: when skill_intro is non-empty but <=100 chars, no ellipsis is appended (previously appended to complete text). Applied the same fix to both: - cli-anything-plugin/skill_generator.py (line 118) - mubu/agent-harness/skill_generator.py (line 238) Updated test_harness_without_readme to assert skill_description is well-formed when intro is empty. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR primarily hardens the CLI hub “meta-skill” installation guidance against prompt-injection by requiring explicit user confirmation before running installation commands fetched from a live catalog, addressing the risk described in #143. It also includes unrelated improvements to the SKILL.md generator logic and tests.
Changes:
- Added prominent “Installation Requires User Confirmation” security preambles and updated agent workflows in hub/meta-skill SKILL docs.
- Updated skill generator logic to avoid trailing
" - ..."when no intro exists and to correctly returnapt-get install ...when detected. - Extended unit tests to cover the empty-intro skill_description behavior.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
cli-hub-meta-skill/SKILL.md |
Adds security preamble + explicit user-confirmation step before installation in the meta-skill flow. |
docs/hub/SKILL.md |
Adds the same security guidance and updates “Quick Install” / “For AI Agents” steps to require confirmation. |
cli-anything-plugin/skill_generator.py |
Adjusts skill_description formatting and improves apt-get extraction behavior. |
cli-anything-plugin/tests/test_skill_generator.py |
Adds assertions ensuring skill_description doesn’t include a dangling " - ..." when intro is empty. |
mubu/agent-harness/skill_generator.py |
Mirrors the generator fixes (skill_description formatting + apt-get extraction) in the mubu harness. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if skill_intro: | ||
| intro_snippet = skill_intro[:100] | ||
| suffix = "..." if len(skill_intro) > 100 else "" | ||
| skill_description = f"Command-line interface for {_format_display_name(software_name)} - {intro_snippet}{suffix}" | ||
| else: | ||
| skill_description = f"Command-line interface for {_format_display_name(software_name)}" |
There was a problem hiding this comment.
The PR description states this is a documentation-only change with no code modified, but this PR also updates Python logic in the skill generators (e.g., skill_description formatting and apt-get extraction). Please update the PR description/scope (or split into a separate PR) so reviewers understand the non-doc behavior changes being introduced.
| patterns = [ | ||
| r"`apt install ([\w\-]+)`", | ||
| r"`brew install ([\w\-]+)`", | ||
| r"apt-get install ([\w\-]+)", | ||
| r"`apt-get install ([\w\-]+)`", | ||
| ] | ||
|
|
||
| for pattern in patterns: | ||
| match = re.search(pattern, content) | ||
| if match: | ||
| package = match.group(1) | ||
| if "apt" in pattern: | ||
| if "apt-get" in pattern: | ||
| return f"apt-get install {package}" | ||
| elif "apt" in pattern: |
There was a problem hiding this comment.
The apt-get handling here changes behavior (and fixes the apt-get→apt misclassification), but the test suite currently only covers apt install extraction. Please add a unit test case for a README containing a backticked apt-get install <pkg> line to lock in the intended output (apt-get install <pkg>).
Addresses the prompt injection surface identified in HKUDS#143: agents following the meta-skill could auto-execute `pip install` commands fetched from the live catalog or embedded in third-party CLI SKILL.md files without user approval. - Add a Security preamble to the "How to Use" section in cli-hub-meta-skill/SKILL.md directing agents to always get explicit user confirmation before running any pip install, and never auto-execute installation commands from untrusted catalog content. - Update the numbered install steps to include a "Confirm with user" step before installation. - Apply the same preamble and step update to docs/hub/SKILL.md, including the "For AI Agents" workflow. Documentation-only change; no code modified. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
sjhddh
force-pushed
the
security/meta-skill-injection-guard
branch
from
da5bc8d to
cab5441
Compare
|
@sjhddh Two things need fixing before merge:
Minor: in |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Thanks for narrowing this to the security docs. The remaining blocker is scope: the new guard only says Please generalize the guidance to all install/update commands discovered from the live catalog or downstream skill files, not just pip. |
Summary
Adds user confirmation guardrails to the meta-skill installation workflow to prevent prompt injection attacks (ref #143).
Problem: The meta-skill documentation instructs AI agents to fetch a live catalog and install CLIs via
pip installwithout requiring explicit user confirmation. A malicious contributor's CLISKILL.mdcould embed adversarial commands that the agent would execute automatically.Fix: Documentation-only change — adds explicit "ask user for confirmation before installing" directives to both
cli-hub-meta-skill/SKILL.mdanddocs/hub/SKILL.md.Note: Code changes (
extract_system_packageandskill_descriptionfixes) have been split into their own PRs (#204 and #203) as requested.