Skip to content

Add safety scanner for skill submissions#81

Merged
timctfl merged 1 commit intomainfrom
safety-scanner
Apr 8, 2026
Merged

Add safety scanner for skill submissions#81
timctfl merged 1 commit intomainfrom
safety-scanner

Conversation

@timctfl
Copy link
Copy Markdown
Owner

@timctfl timctfl commented Apr 8, 2026

Summary

  • Adds scripts/safety_scanner.py with rule-based checks (Python AST, JS/shell regex) and Claude Opus 4.6 LLM review
  • Catches eval/exec, reverse shells, prompt injection, obfuscated payloads, privilege escalation, zero-width char hiding, and more
  • Only scans skills changed in the PR to keep API costs low
  • Gracefully skips LLM checks when ANTHROPIC_API_KEY is not set (fork PRs)
  • Includes 32 unit tests and 8 test fixtures covering every rule category
  • All 19 existing skills pass with zero BLOCK findings

Test plan

  • All 32 unit tests pass (python scripts/test_safety_scanner.py)
  • Integration smoke test: all 19 existing skills produce zero BLOCK findings
  • Live LLM test: ran with real Opus API key, all 19 skills pass
  • import os in product-attribute-dictionary produces WARN (not BLOCK)
  • console-script.js (browser IIFE) produces zero findings
  • After merge: add ANTHROPIC_API_KEY as repo secret (already done)

🤖 Generated with Claude Code

Rule-based checks (AST, regex) for Python, JavaScript, and shell scripts
plus Claude Opus 4.6 LLM review of all skill files. Catches eval/exec,
reverse shells, prompt injection, obfuscated payloads, and more.

Only changed skills are scanned in CI to keep API costs low. If
ANTHROPIC_API_KEY is not set (fork PRs), LLM checks are skipped
gracefully and rule-based checks still run.

Includes 32 unit tests and 8 test fixtures covering every rule category.
All 19 existing skills pass with zero BLOCK findings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@timctfl timctfl merged commit 23e681b into main Apr 8, 2026
1 check passed
@timctfl timctfl deleted the safety-scanner branch April 8, 2026 01:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants