Repository navigation
gateway: close mutation-test gaps; deep mutants in 32 two-stage shards - #118
Merged
Merged
Conversation
…aker threshold() Real gaps, each test verified to fail under its mutant by hand: - reset_before_reading (D80): each conjunct alone false keeps the request unresent. - plan_stream_usage_injection: a body not opening with an object is never planned. - ModelScanner::feed: a comma disarms a root `message` that was not an object. - escaped_at: an escaped `"type"` opening quote (odd run) is not a member. - UsageTail: explicit exact-capacity and wrap-straddling usage event test (BIL-15). Equivalent: UsageTail::push's `if rest > 0` only skipped an empty copy; removed. Dead: CircuitBreakerConfig::threshold had no callers; deleted. Five listed survivors (scan_buffered guard, root_open, escaped_at ==, push rest<0, push head*len) were already killed by existing lib tests: the run shared one CARGO_TARGET_DIR across --jobs 3, so builds raced and tested other jobs' mutants. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
…against the full suite) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
…ds filled disks and slowed every mutant) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
…her jobs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
…or diff --re over mutant names skipped most stage-1 survivors (26 of 41 in one shard), so stage 2 never re-tested them against the full suite. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
…he full mutants run
53 listed survivors (translate.rs left to a parallel pass), every kill verified by
hand-applying the mutant and running the targeted test.
Real gaps (new tests):
- catalog_error_relay: a same-endpoint vendor's foreign error (DeepSeek) is put in the
client's envelope; a sub-resource error (count_tokens) is relayed byte for byte. The
D100 test had drifted off this path when grok moved to Responses.
- catalog_chat_relay: an OpenAI Chat stream is a byte relay (no identity bridge).
- request_filter's up-front body reservations: a declared body that fits the budget
once but not twice is refused before a byte is read (headerless and header-won
walks), and a buffered /{provider} body is refused before the upstream sees it.
- walk_reads_body / walk_reads_tools / candidate_path_is_responses unit tests.
- breaker: a generation-0 probe success closes it; Debug prints the config.
- value_end: a separator is no value; disable_openrouter_compression shapes;
empty model span rewrite; a cancelled full-body attempt reaps as Finished.
Dead or simplified: redundant STATE_OPEN arms (breaker), splice_out's no-op guard,
InputTally::run's unreachable (None, Some) arm, estimate_stream_output's event
counter (deltas == 0 is the whole guard), clip_catalog_name -> floor_char_boundary,
catalog_chat_relay's client check (any other client is translated anyway),
request_summary's body moved to a tested summary_line.
Equivalent, excluded in .cargo/mutants.toml with reasons: pack's | vs ^, the TALLY
index | vs ^, close()'s flag clear, remove_items k<l vs <=, rename_max_tokens' comma
guard, force_include_usage s>at vs >=, request_summary delegation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
92 survivors in translate.rs: 88 killed by new tests, 3 removed as dead code, 1 excluded as equivalent. No defects found. Tests (request, response/stream, and pure-table unit tests) pin: the reasoning-value table and both Claude relay rules, assistant refusals in both directions, each legacy function form alone (and the modern field winning), effort/budget tables, tool-result part mapping, strict-schema nesting, json_object both ways, custom grammar flattening, allowed_tools both ways, session-field edge inputs, the FNV-1a vectors, and Chat <-> Messages/Responses stream details (role chunk, summary separator, item_id-only calls, nameless calls, escaped quotes, DONE-only streams). Dead: message_text's Null arm (same as `_`), anthropic_system_text's String arm (only ever called on an array), and the cache_control copy in the Responses instructions helper (only its text was ever read). Equivalent: ArgsEnd::whole's memo `||` (either operator, same answer). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
Without --file, --in-diff falls back to mutants.toml's examine_globs and skips route.rs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
84 survivors of the full mutants run in translate.rs: 67 killed by tests (5 already by existing tests, 2 more by existing tests once the key packing was factored out), 10 removed as dead or equivalent code, 7 excluded as equivalent. No defects found. New tests pin: both bridge overflow bounds one byte past their size (and MAX_STREAM_OUTPUT against the catalog), the 8 MiB thinking cap in HeldThinking and Gather, Gather::close, the raw-JSON keep rules and negative integers in KeepInputs, looks_like_error / in_client_envelope / status_error_type / anthropic_error_type tables, clip at a char boundary, error_info's message sources and the overflow code, the plain-text document mapping, Responses body created/text/content-null rules, OpenRouter reasoning.summary onto Responses, interrupted thinking never folded into a later signature, an unterminated last event, byte-exact relay role cuts (LF, CRLF, unterminated), a repeated call id without a role, nothing after message_stop onto Chat, every Messages stream event shape onto Chat, and Responses refusal deltas. Dead: map_response's and request's `a == b` arms (same as `_`), KeepInputs' visit_string/visit_none/visit_some (serde_json's deserialize_any never calls them), error_info's `is_object` guard on a parsed raw error, chat_finish's tool_calls arm (responses_status reads only length and content_filter), Usage::from_chat's `r > 0` (adds 0), and flush's Responses `!errored` guard (error marks it completed). ChatIdentity's key packing is one fn now. Excluded: KeepInputs expecting/visit_unit, `|` vs `^` in ChatIdentity::key, and assembled's clone-skipping guard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
…-run survivors 78 survivors of the full beyond-ai suite (proxy.rs, usage.rs, peek.rs, circuit_breaker.rs, route.rs). Every kill verified by hand-applying the mutant and running the targeted test. Real gaps (52, new tests): - Large-body walks: every pool key is walked (30 keys) and the attempt bound's last attempt is the client's answer (150 keys, a 429 not a 502); each candidate of a three-candidate walk gets its own refused-stream resend, then fails over. - Catalog walks: a refused stream is resent once; a reused-connection reset is resent on the same candidate; a cooling provider is skipped from any catalog slot; a rewrite ahead of `model` (developer->system, Claude-on-Chat reasoning cut, gateway reasoning cut) is rescanned so the model is re-spelled in place; same-wire relays keep reasoning_details (non-Claude host) and explicit nulls (Anthropic); a Responses stream's stream_options gets no include_usage. - Body budget: bodies within the 64 KiB replay buffer are not charged; a chunked body that outgrows the budget is a 503, not a 413; a body declared at exactly MAX_REQUEST_BODY is read; a translation needing exactly the budget is a 503, not a 413; client_write_timeout_secs = 0 disables the timeout. - Headers and errors: a managed key beside mixed BYO credentials is not ambiguous; a BYO ?key= is forwarded; an allowed anthropic-beta line goes out untouched; a rewritten JSON error is chunked to an HTTP/1.1 client; a non-JSON error keeps its Content-Length; a streaming request's account remedy is neutralized. - Billing: message_start behind a few KiB of pings is still read from the head; a final multi-line SSE event without a blank line bills; skipped strings (escaped quotes, non-letter keys) end at their own closing quote in the input tally. - Units: h2_body_unsent, Redact with no key / a whole body, the unanswered-attempt 502 (Reaped::unanswered), CircuitOpen's Display, str_is on an escaped value, has_typed_member's span end, an escaped stream_options in the injection planner, error_body returns the static table body (REJECT_BODIES is now a static, so the pointer check holds). Dead or simplified (11): ModelScanner's and the planner's depth guards on ':'/',', the planner's whitespace skip and key-skip condition, scan_buffered's key-skip condition, peek_body_model's empty-chunk arm and `complete` guard, the walked-arms fold (now control::Walk::mask), and a stream-only candidate's force_stream usage flag (the usage splice every Chat stream gets already asks for it; ARCHITECTURE.md updated). BODY_PEEK_LIMIT's six comparisons are one past_replay_buffer helper. Equivalent, excluded in .cargo/mutants.toml with reasons (14): str_is's raw fast path, unserved's and the TALLY tables' | vs ^, de_service_tier's Ok(None) visitors, UsageTail::push at exactly the cap, release_body(0), settle_health's undecided arm, retain_managed_client_headers' (None, _) arm, mask_all's at *= (same hits, quadratic), Ctx::deref (DerefMut's supertrait; a panicking body passes the integration suites). Already removed before this pass: 1 (the breaker's STATE_OPEN arm). Found, not fixed: a managed 402 that fails over to the next candidate is never read, so its key is never cooled (D180 cools only a relayed one) and every later request pays a round trip to the unfunded provider first. Cooling needs the body (OpenRouter's "requires more credits" 402 is not unfunded), which the failover discards. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
…r equivalent mutants The replica breaker test opened A's breaker with a 2s reset and then asserted A routed away for three turns: on a loaded host the turns outran the reset and A's half-open probe went to the healed primary. That flake also made cargo-mutants count mutants as caught when nothing else failed. Now two tests: open-phase routing with a reset far past the test, and recovery with a 1s reset, polled. ChatIdentity::key's & -> ^ keeps keys distinct (XOR with a mask is a bijection in range); tally_eager (now its own fn) only picks where the same input estimate is made. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Results from the full mutation run over the gateway's core code.
reset_before_reading(D80): each condition false on its own.plan_stream_usage_injection: a body that doesn't open with an object is never planned.escaped_at: odd vs even backslash runs.ModelScanner: a comma disarms a rootmessagethat wasn't an object.CircuitBreakerConfig::threshold, and a guard that could only skip an empty copy.Remaining survivors from the 32-shard run will be added to this PR.
🤖 Generated with Claude Code
https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk