Skip to content

feat: explain a case in prose, grounded in its own record - #20

Merged
iamsiddhesh-dev merged 5 commits into
mainfrom
case-explainer
Sep 3, 2026
Merged

iamsiddhesh-dev merged 5 commits into
mainfrom
case-explainer

Conversation

@iamsiddhesh-dev

Copy link
Copy Markdown
Owner

Case Detail shows the arithmetic — every candidate action, its expected value as a
checkable sum, the compliance rules that fired. That is the right artifact for
auditing a decision and the wrong one for asking "so what happened to this payment,
and why did nobody chase it?"

This adds the second thing, above the first.

The rule that makes it safe

The model may not use a number it was not given. Every digit run in the
generated text must appear in the brief it was handed. Grouping is normalised
first, so ₹5,41,724 and 541724 count as the same figure and correct prose is not
rejected for reformatting.

That is a much simpler check than asking whether a claim is true, and it catches
the failure that matters: a narrative confidently reporting "three retry attempts"
on a payment that had one is worse than no narrative at all, on the one screen
whose entire purpose is showing that the numbers add up.

Same reasoning as the copywriter, reached from the other side. There the model
never sees a number; here it may only echo the ones it was shown.

Two narrower checks alongside it — an explanation cannot mention a channel that was
not used, and cannot claim a recovery that did not happen. The recovery check keys
on specific phrases rather than the word "recovered", because "we stopped once no
further attempt was worth what it could recover" is a true sentence about a payment
that was never recovered.

Selection by rule, not by eye

Picking cases by hand would make this a highlight reel: the ones that read well
would get the prose. So the selection is the largest payment of each distinct
shape — recovered without contact, recovered after contact, chased and lost,
never actioned, never diagnosed.

Two of those five are cases where the agent achieved nothing, and they are in the
list on purpose. A test asserts the selection is not all wins.

The fifth shape returns nothing on the agent arm, because the LLM classifier
resolves every cause — so this run explains four cases, not five. That is the
selection working, not failing.

Never a model call on a page load

One batched call covers the whole selection, run once when demo writes the run.
Explaining on demand would be a call per page view — a judge clicking through ten
cases would spend half the daily Gemini allowance on a read-only screen, and two
visits to the same case could disagree with each other.

Cases outside the selection get summarise(), composed from the same facts with no
model involved. It is not a stub — it is assembled from the case's own shape, so a
payment that was refused everything reads as a sentence about being refused
everything. The screen always has something to say.

A bug the tests caught

A test runs summarise() over 40 real cases and puts the output through the
validator — holding our own prose to the standard we hold the model's. It failed on
pay_002520: the summary said "the agent acted 12 times" and 12 appeared nowhere in
the brief, because the brief listed ESCALATE_HUMAN twelve times and never stated
the total.

Both halves were wrong. Actions are now tallied (12 in total — ESCALATE_HUMAN ×12)
and identical decision sentences deduplicated. Repeating one decision twelve times
cost tokens, read badly, and invited the model to narrate a sequence of distinct
events that did not happen.

Also fixed: a 35-paise WhatsApp send rendered as spent on recovery: ₹0, which
would have had the model reporting that nothing was spent. It now keeps its paise,
which is exactly what that formatting rule exists for.

Not in the ablation

The explainer reads the ledger; it does not participate in it. It cannot change a
decision or an outcome, and reproduce confirms all 37 figures including every
per-arm digest are unchanged.

Verification

pytest — 465 passed, 30 of them new. ruff check . clean. Generated live against
the real model: 4 of 4 accepted, one call, zero rejections. The committed cache
replays with both API keys removed from the environment.

@iamsiddhesh-dev
iamsiddhesh-dev merged commit c5aedb7 into main Sep 3, 2026
1 check passed
@iamsiddhesh-dev
iamsiddhesh-dev deleted the case-explainer branch September 3, 2026 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant