Skip to content

fix: stop reconsidering a payment already handed to a human - #21

Merged
iamsiddhesh-dev merged 5 commits into
mainfrom
escalation-cap
Sep 4, 2026
Merged

iamsiddhesh-dev merged 5 commits into
mainfrom
escalation-cap

Conversation

@iamsiddhesh-dev

Copy link
Copy Markdown
Owner

Found while reading a case brief for the explainer: one payment listed
ESCALATE_HUMAN twelve times. Pulling the distribution showed 48 escalations
across four payments — twelve each — against a run cap of fifty.
₹5,280 spent
handing over cases that had already been handed over, and two more repeats on any
one of them would have exhausted the pool for every payment that genuinely
warranted a human.

The root cause

Escalation hands a case to a person. The executor records it, writes STOPPED,
and returns succeeded=False, because no money came back. The runner's rule was:

if not executed.succeeded and when < horizon:
    reschedule(RECONSIDER)

So the payment came back a minute later. Nothing about it had changed, so the
policy proposed escalation again, the gate allowed it, and the loop ran until the
per-payment decision cap of twelve stopped it.

Execution now carries terminal alongside succeeded. They answer different
questions — handing a case to a human did not recover the money and is also not a
failed attempt worth retrying — and collapsing them into one boolean is what
created the loop.

And a rule that should have existed anyway

The compliance gate now separately refuses a second escalation on the same
payment. This is the more important half. "One payment, one human" is a
policy, and a policy that holds only because of the shape of a scheduling loop
somewhere else is not enforced — it is coincidental. The mandate bug already in
FAILURES.md was a compliance rule that looked present and did nothing; this is the
same lesson from the other direction.

That rule does not fire on the committed seed, because with the runner fixed a
second escalation is never proposed. A rule that never fires is exactly the shape
of the bug it guards against, so it is covered by unit tests that construct the
condition directly rather than trusted because the run looks clean.

Why nothing caught it

Every individual piece was correct. The executor did stop touching the payment.
The gate did enforce its run cap — 48 is under 50, so it never fired. The policy
priced each escalation correctly. actions_by_kind showed ESCALATE_HUMAN: 48,
and 48 looks like forty-eight escalated payments, which would be reasonable. The
failure was visible only in the distribution, and nothing summarised that.

It also cost nothing in recovery, because escalation never recovers money in this
simulator — so the totals moved by ₹5,280 of cost and not one rupee of revenue. A
bug that only makes you slightly poorer is much harder to notice than one that
breaks something.

What moved

Before After
Recovered ₹5,41,724 ₹5,41,724
Contacts 926 926
Refusals 903 903
Escalations 48 4
Cost ₹8,336 ₹3,056
Net margin ₹1,43,347 ₹1,48,627

Incremental recovery, the coverage/judgment split and the LLM ablation are
unchanged to the rupee, because all three are measured on recovery.

The full sweep re-ran and reports/sensitivity.json came back byte-identical
so this is cost-only across all fifteen points in the parameter space, not just at
the base case. Twenty minutes to prove a negative, which is what the sweep is for.

reproduce flagged exactly the six figures that should have changed — cost and
digest for the three escalating arms — and nothing else. First time it has caught
a real change rather than confirming a null one.

Also

summarise() said "The agent acted 1 times" and then "stopped when no option was
worth its cost" on a case that was in fact escalated. It now names the handover and
says the outcome of that review is outside this record, which is the honest
statement: whether a person resolved it is not something this run may claim credit
for.

Verification

pytest — 469 passed, 4 new. ruff check . clean. recoup reproduce green
against regenerated claims. Run, explanations, claims and sweep all regenerated;
FAILURES.md and the README figures updated.

@iamsiddhesh-dev
iamsiddhesh-dev merged commit 5ff5d62 into main Sep 4, 2026
1 check passed
@iamsiddhesh-dev
iamsiddhesh-dev deleted the escalation-cap branch September 4, 2026 06:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant