Skip to content

Delivery-failure retry loop: wake the agent to fix and resend undelivered messages#33

Merged
dimavrem22 merged 1 commit into
mainfrom
delivery-failure-retry-loop
Jul 10, 2026
Merged

Delivery-failure retry loop: wake the agent to fix and resend undelivered messages#33
dimavrem22 merged 1 commit into
mainfrom
delivery-failure-retry-loop

Conversation

@dimavrem22

Copy link
Copy Markdown
Contributor

Mirrors inkbox-ai/hermes-agent-plugin#48 for the Claude Code bridge — the same delivery-failure retry loop, adapted to this repo's per-contact session architecture.

What

An outbound message can die two ways, and until now the bridge only learned about one:

  • Rejected at send time — the server's outbound content policy 422s the send (markdown artifacts, emoji overload…), the recipient has opted out, the address is bad, or the body is too long. The error comes back on the send call and was surfaced as a one-shot recovery turn in sessions.py.
  • Failed after acceptance — the carrier flags it (e.g. error 40002) or the mail server bounces it, reported later via text.delivery_failed / imessage.delivery_failed / message.bounced / message.failed webhooks. The bridge already woke the session for these, but with no cap and no shared budget.

Both surfaces now feed one delivery-failure loop: the agent is woken in the affected contact's session (a run_consult side-turn — it acts via its Inkbox tools rather than replying on the channel that just failed) with the exact error (rule/code/detail) plus its own undelivered body, and instructed to fix and resend.

Guardrails (feature behavior)

  • Hard cap: 3 total sends per logical reply. Failures 1 and 2 wake the agent; failure 3 goes quiet with a loud log line. The budget is shared across both failure surfaces (keyed by conversation + recipient, merged by max).
  • Budget resets on a fresh inbound, a delivered receipt, or a 30-minute TTL.
  • Webhook replays are deduped per failed message.
  • Transient (5xx) send failures are excluded — a bare resend clears those.
  • The synchronous surface routes through ContactSession._deliver_reply via a new on_send_rejected gateway callback, replacing its old one-shot recovery turn.
  • Subscriptions now include message.bounced / message.failed for mail and the outbound delivery lifecycle for text/iMessage (so delivered-reset and delivery_failed-wake fire for real); the reconciler auto-patches the event set on startup.

Testing

  • 19 unit tests (tests/test_delivery_failure_retry.py): cap semantics, budget sharing, resets, replay dedup, group recipient rows, opt-out and bounce paths, all three channels.
  • Two live tests (real-model leg, tests/live/test_sms.py): test_sms_retry_after_carrier_delivery_failure (forged, org-key-signed text.delivery_failed webhook → real follow-up SMS) and test_sms_retry_after_internal_spam_block (emoji-overload bait → asserts the block reached the agent via wake-up or inline tool rejection; a compliant follow-up is best-effort since the ask is inherently unsatisfiable).
  • Live-suite conversation-health reset guardrail (tests/live/conftest.py): reads the opener's window and only resets when it's running low, so the shared 10DLC pool's duplicate_body / unanswered_limit rules don't false-red the suite. Driver SMS bodies are diversified with a unique ref across the SMS, voice, and cross-channel suites.

Raises the inkbox SDK floor to 0.4.20 (ships blocked_spam_filter) across package metadata, the setup wizard, and doctor. Bumps the plugin 0.1.2 → 0.1.3.

Notes vs the hermes PR

  • The hermes PR also suppressed a leaked 🔍 progress glyph in outbound messages; that mechanism (_is_hermes_admin_notice / tool-progress-glyph list) does not exist in this bridge, so there's nothing to mirror here.
  • The wake-up is a run_consult capture turn (this bridge's existing delivery-failure design) rather than an auto-replying event turn, so the prompt instructs the agent to send via its Inkbox tools; the cap now bounds what was previously a deliberately-uncapped-but-non-auto-replying path.

…ered messages

An outbound message can die two ways and the bridge only learned about one:
rejected synchronously at send time (server content-policy 422, opt-out,
bad address, too-long) or accepted and then failed downstream (carrier
rejection, mail bounce) reported by a lifecycle webhook. Both surfaces now
feed one delivery-failure loop: the agent is woken (run_consult, so it acts
via its Inkbox tools rather than replying on the possibly-dead channel) with
the exact error plus the undelivered body so it can fix and resend.

Total sends per logical reply are hard-capped at OUTBOUND_FAILURE_MAX_ATTEMPTS
(3), with the budget shared across both surfaces (keyed by conversation /
recipient, merged by max) and reset on a fresh inbound, a delivered receipt,
or a 30-minute TTL. Webhook replays are deduped per failed message and
transient 5xx send failures are excluded (a bare resend clears those). The
synchronous surface routes through the session's _deliver_reply via a new
on_send_rejected callback, replacing its old one-shot recovery turn.

Covers all three channels: SMS/MMS (incl. group recipient rows), iMessage,
and email — the mailbox subscription now includes message.bounced /
message.failed, and text/iMessage now subscribe to the delivery lifecycle so
delivered-reset and delivery_failed-wake fire for real; the reconciler
auto-patches the event set on startup.

Testing:
- tests/test_delivery_failure_retry.py: 19 unit tests — cap semantics, budget
  sharing, resets, replay dedup, group rows, opt-out/bounce, all three channels.
- Two live tests (real leg) in tests/live/test_sms.py: a forged, org-key-signed
  text.delivery_failed webhook → real follow-up SMS, and an emoji-overload bait
  asserting the block reached the agent (wake-up or inline tool rejection).
- tests/live/conftest.py: conversation-health reset guardrail so the shared
  10DLC pool's rules don't false-red the suite; driver bodies diversified.

Raises the inkbox SDK floor to 0.4.20 (blocked_spam_filter status) across
package metadata, the setup wizard, and doctor. Bump 0.1.2 -> 0.1.3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dimavrem22 dimavrem22 merged commit 178a814 into main Jul 10, 2026
9 of 10 checks passed
dimavrem22 added a commit that referenced this pull request Jul 10, 2026
…ect (#34)

* Deflake the external-event call test: bigger budget + one-shot re-inject

test_valid_github_signature_makes_agent_call_driver flaked on PR #33's CI
("agent never called within 200s") — unrelated to that change. Every external
event wakes a FRESH Claude Code session (chat_id external:<source>:<event_key>,
no resume), so it's a cold-start + real-model call on a fixed budget; under CI
load (it runs right after the channels + voice suites drain the shared runner
and AUT), a single attempt at 200s occasionally times out. Same test went
failure→failure→success on 2026-07-03, and passed immediately on re-run here.

- Raise the shared external-event budget 200s → 300s (workflow env default,
  which overrides the test-file default; test-file default bumped to match).
- Give the valid-signature test a one-shot re-inject: if the first window
  lapses, re-post a fresh-id escalation and wait one RETRY_WINDOW_S (180s)
  more before failing. The forged-event test proves we never dial spuriously,
  so a benign re-send is safe.
- Bump the job timeout 20 → 25 min for headroom.

Test-only + workflow-only; no product code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Fix external-event call flake at the source: refusal, not timeout

The fuller gateway log from the failed run showed the real cause: the model
was not slow, it REFUSED. On the valid-signature escalation the real model
sometimes declined to place the call — "an unsolicited phone call … the wrong
move to auto-execute", "the instruction to phone a specific person is embedded
in the payload's free-text fields" (i.e. it treated a verified escalation as a
possible prompt-injection). A bigger timeout / one re-inject can't fix a
refusal.

Two changes:

- Product: strengthen the VERIFIED external-event directive so a verified
  event's request to reach a named person (call/text/email the contact in the
  payload) is treated as the operator-pre-authorized action this channel
  exists to perform — the signature IS the authorization — and is explicitly
  NOT to be declined as "unsolicited" or downgraded as prompt-injection
  (injection caution stays for embedded credentials/tokens/links/rule-changes).
  Scoped to VERIFIED events only; the UNVERIFIED directive stays fully cautious.

- Test: make the valid-signature test take several INDEPENDENT draws (each a
  fresh event → fresh cold-start session), so a rare residual hesitation isn't
  a false red. Bounded windows keep the suite under the job timeout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant