ci(monitor): bump scan budget + enable Anthropic deep-dive on the ambiguous band#22
Merged
Conversation
Enable targeted escalation in the scheduled sweep now that the pipeline is reliably committing: - --max-scans 3000 -> 4000 (rules-only is fetch-bound at ~2 req/s; 4000 still fits the window and shortens the ~66k-skill baseline build). - --escalate-backend anthropic --escalate-budget 200: the ~150 skills/run the rules flag short of a confident verdict (the ambiguous band) get an LLM deep-dive second opinion; confident-clean and confident-malicious are never escalated, so spend tracks the uncertain middle. - AnthropicBackend no longer live-crawls URLs during escalation (use_urls off): removes external-fetch latency/flakiness at escalation scale; the deep dive's value is the LLM's read of intent (rules + threat intel still run). HF classifier stays validated (protectai/deberta-v3-base-prompt-injection-v2, apache-2.0, ungated) and available for local "--escalate-backend hf|tiered", but is kept off the CI critical path to avoid a ~2GB torch install on the run that must reliably commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DNoTXU8k3pfSBzR7aJubqL
✅ Malwar Scan Results
Scanned by Malwar — malware detection for agentic AI skills. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Turns on targeted escalation in the scheduled sweep now that the pipeline reliably commits, and validates the HF model.
--max-scans3000 → 4000 (rules-only is fetch-bound at ~2 req/s; 4000 still fits the CI window and shortens the ~66k-skill baseline build).--escalate-backend anthropic --escalate-budget 200. The ~150 skills/run the rules flag short of a confident verdict (the ambiguous band — SUSPICIOUS/CAUTION) get an LLM second opinion that can raise a sneaky skill or clear a false positive. Confident-clean and confident-malicious are never escalated, so LLM spend tracks the small uncertain middle — not the whole 66k registry.use_urlsoff inAnthropicBackend): removes external-fetch latency/flakiness at escalation scale; the deep dive's value is the LLM's read of intent (rules + threat intel still run).HF model — validated ✅
Confirmed on the Hub:
protectai/deberta-v3-base-prompt-injection-v2— apache-2.0, ungated, 5.6M downloads, ships ONNX weights. That's the right free-tier pick (permissive license, no HF token needed). It stays wired as the default for local--escalate-backend hf|tiered, but is kept out of CI to avoid a ~2GBtorchinstall on the run that must reliably commit. Enabling it in CI (or a decoupled escalation job) is a follow-up if we want the free tier automated.Why Anthropic-direct rather than tiered-in-CI
The ambiguous band is only ~150 skills/run, so Anthropic's cost is already small; putting HF (torch) in the daily job would add fragility to the pipeline we just got committing, for marginal savings. Anthropic-direct is the reliable "deep dive on the outliers."
Type of Change
Checklist
ruff+ tests pass (65 monitor/escalation)docs/crawl.md)🤖 Generated with Claude Code
https://claude.ai/code/session_01DNoTXU8k3pfSBzR7aJubqL
Generated by Claude Code