Skip to content

Add security-control measurement and experimentation - #2

Merged
VinayK88 merged 5 commits into
mainfrom
agent/security-measurement
Aug 19, 2026
Merged

Add security-control measurement and experimentation#2
VinayK88 merged 5 commits into
mainfrom
agent/security-measurement

Conversation

@VinayK88

Copy link
Copy Markdown
Owner

What changed

  • adds a first-class security-control measurement contract that separates risky behavior, intervention, task success, and prevented outcomes
  • measures prevented-risk rate, benign false positives, legitimate task completion, approval rate/latency, detection latency, recovery success, and transparent net security utility
  • adds a deterministic strict-vs-adaptive safeguard experiment to expose the risk-reduction vs. user-friction tradeoff
  • adds a dependency-free bootstrap interval for candidate-minus-baseline security utility
  • integrates the experiment into the normal AgentShield report
  • adds unit tests and a methodology note with production caveats

Why

AgentShield already evaluates runtime authorization and unusual trajectories. This change makes it stronger as a security data-science project by asking whether a safeguard improves real security outcomes enough to justify the friction it introduces, rather than treating block rate as success.

Validation

The new tests cover strict-control effectiveness, adaptive-control friction reduction, deterministic bootstrap behavior, and invalid empty input. Existing CI should run the full unit-test suite and project smoke checks on this branch.

Scope

All experiment data remain synthetic. The utility weights are explicitly illustrative and the documentation states that production use requires sensitivity analysis, assignment-aware experimentation, and rare-severity guardrails.

@VinayK88
VinayK88 marked this pull request as ready for review August 19, 2026 02:29
@VinayK88
VinayK88 merged commit 106f773 into main Aug 19, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant