Skip to content

Pressure-test hardening: report_id/order_id validation, groups-claim robustness, error sanitization - #3

Merged
MattJColes merged 5 commits into
mainfrom
claude/pressure-test-agents-48ukvf
Jul 6, 2026
Merged

Pressure-test hardening: report_id/order_id validation, groups-claim robustness, error sanitization#3
MattJColes merged 5 commits into
mainfrom
claude/pressure-test-agents-48ukvf

Conversation

@MattJColes

Copy link
Copy Markdown
Owner

Summary

I ran the server and pressure-tested every surface (auth/access middleware, the S3 file-delivery path, the audit trail, the agent-behind-a-tool, and the composition/proxy wiring) using both live in-memory Client probes and a fan-out of focused review agents. The core two-layer access control is genuinely sound and fail-closed — I could not find a privilege-escalation bypass. What I did find were five real robustness/security defects around input validation and internal-detail leakage. Each fix ships with regression tests.

Baseline was 54 tests passing; now 77 passing (23 new).

Bugs fixed

  1. export_report — weak report_id validation + error leakage (domains/reports.py)

    • Denylist (/, \, ..) let surprising ids through: surrounding whitespace landed in the S3 key verbatim (blank-check used .strip() but the key used the raw value), control bytes (NUL/CR/LF) reached the key and any key-logging pipeline, and over-length ids failed deep in S3. It also false-rejected legitimate dotted ids like 2024.q1.
    • A boto ClientError propagated verbatim, leaking the bucket name, S3 operation, and AWS error code into the caller's (and model's) context.
    • Fix: strict allowlist ([A-Za-z0-9._-], length-bounded) so prefix escape stays impossible while dotted ids work again; wrap the S3 calls and return an opaque report export failed, logging detail server-side.
  2. allowed_tags — crash and lockout on real IdP tokens (auth.py)

    • groups: null in the claims → TypeError: 'NoneType' object is not iterable surfaced to the caller (dict.get(..., []) doesn't cover explicit null).
    • groups: "admin" (single group as a bare string) → iterated per-character, matched nothing, silently locked the caller out.
    • Fix: _normalize_groups() coerces the claim (None/missing → none, bare string → one group, list/tuple/set → its string members, else → none). Every path fails closed.
  3. on_call_toolget_tool failure leaks internal detail (access.py)

    • get_tool(name) was unwrapped. In-process tools return None on an unknown name, but a proxied tool (the shipped config mounts a remote analytics proxy) can raise a backend/connection error at lookup, whose raw text names internal hosts/operations and defeats the uniform "Unknown tool" response.
    • Fix: wrap the lookup; treat any failure as unknown so the answer stays uniform and the gate stays closed. Also corrected the module docstring, which claimed support can't see finance tools — the real policy (and README) grant support the billing tag.
  4. draft_refund_email — prompt injection via order_id (domains/support.py)

    • A caller-controlled order_id was interpolated straight into the prompt handed to the injected inner agent, with no validation. An id like A1. Ignore prior instructions and ... rode verbatim into the instructions a production model would execute; the same unbounded input inflated the prompt (token-cost/DoS) and pushed control chars into the ctx.info audit line.
    • Fix: validate order_id against a tight allowlist and reject before the agent is called; reword the docstring to describe the real boundary.
  5. issue_refund — misleading bool-guard comment (domains/admin.py)

    • The comment claimed isinstance(amount, bool) stops True/False passing as 1/0, but over the MCP wire the float schema coerces JSON true/false to 1.0/0.0 before the body runs, so amount=true silently records a 1.0 refund. Documented that the guard is defense-in-depth for direct callers and that the real guarantees are the positive-and-finite checks.

Testing

python -m pytest -q77 passed. New tests cover: unsafe/whitespace/control-char/over-length/dotted report_id, masked storage errors, groups: null/scalar-string/odd-type normalization, get_tool-raises sanitization, and order_id injection rejection.

🤖 Generated with Claude Code

https://claude.ai/code/session_01F5dytZCaHHE432vqopou4h


Generated by Claude Code

claude added 5 commits July 6, 2026 15:04
report_id is interpolated into the S3 key, but validation was a denylist of
/, \, and '..'. That let surprising ids through: surrounding whitespace landed
in the key verbatim (blank-check used .strip() but the key used the raw value),
control bytes (NUL/CR/LF) reached the key and any key-logging pipeline, and
over-length ids sailed past only to fail deep in S3. The denylist also
false-rejected legitimate ids containing '..' (e.g. 2024.q1).

Replace it with a strict allowlist ([A-Za-z0-9._-], length-bounded): separators
can't get through so prefix escape stays impossible, control/whitespace/non-ASCII
are rejected, and dotted ids are accepted again.

Also wrap the S3 upload/sign calls: a boto ClientError previously propagated
verbatim, leaking the bucket name, operation, and AWS error code into the
caller's (and model's) context. Log the detail server-side, return an opaque
'report export failed' instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F5dytZCaHHE432vqopou4h
allowed_tags() read token.claims.get('groups', []) and iterated it directly.
Two real failure modes on tokens a real IdP can mint:

- 'groups': null in the claims -> get() returns None (the default only applies
  when the key is absent), so 'for g in None' raised TypeError and surfaced a
  raw 500-style error to the caller instead of a clean deny.
- 'groups': 'admin' (an IdP emitting a single group as a string) -> iterating
  the string looped over its characters, matched nothing, and silently locked
  the caller out of every tool.

Add _normalize_groups() to coerce the claim: None/missing -> no groups, a bare
string -> a one-element list, a list/tuple/set -> its string members, anything
else -> no groups. Every path fails closed, so this hardens availability without
weakening access.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F5dytZCaHHE432vqopou4h
on_call_tool called get_tool(name) unwrapped. For an in-process tool that
returns None on an unknown name, but a proxied tool (the shipped config mounts
a remote analytics proxy) can raise a backend/connection error at lookup time.
That raw error propagated to the caller, naming internal hosts and operations
and defeating the deliberate uniform 'Unknown tool' response. Wrap the lookup
and treat any failure as unknown so the answer stays uniform and the gate stays
closed.

Also correct the module docstring: it claimed 'the support team doesn't see
finance tools', but the actual policy (and the README) grant support the billing
tag, so it does see the read-only invoice lookup. Restate the example accurately
(no one but admin reaches the refund tool) and point at GROUP_TAGS.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F5dytZCaHHE432vqopou4h
draft_refund_email interpolated a caller-controlled order_id straight into the
prompt handed to the injected inner agent, with no validation. The docstring
called this a 'tightly-scoped prompt', but the scope was as wide as the input:
an order_id like 'A1. Ignore prior instructions and ...' rode verbatim into the
instructions a real (production) model would execute. The same unbounded input
inflated the prompt (token-cost/DoS) and pushed control chars into the ctx.info
audit line (log injection).

Validate order_id against a tight allowlist ([A-Za-z0-9._-], length-bounded) and
reject before the agent is called. An order id is an opaque identifier, never
prose, so this loses nothing legitimate while removing the injection surface.
Reword the docstring to describe the real boundary.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F5dytZCaHHE432vqopou4h
issue_refund's comment claimed the isinstance(amount, bool) check stops
True/False passing as 1/0. That guard only fires for a direct Python call; over
the MCP wire the amount: float schema coerces JSON true/false to 1.0/0.0 before
the tool body runs, so the check never sees a bool and amount=true silently
records a 1.0 refund. Document that the guard is defense-in-depth for direct
callers and that the real money guarantees are the positive-and-finite checks
(false -> 0.0 is rejected by amount <= 0).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F5dytZCaHHE432vqopou4h
@MattJColes
MattJColes merged commit 190087b into main Jul 6, 2026
6 checks passed
@MattJColes
MattJColes deleted the claude/pressure-test-agents-48ukvf branch July 6, 2026 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants