Skip to content

fix(chat): booking card on file requests, complaints, pending certifications and role acknowledgment - #475

Merged
digibrander merged 4 commits into
mainfrom
development
Sep 17, 2026
Merged

digibrander merged 4 commits into
mainfrom
development

Conversation

@jadhavgaurav

Copy link
Copy Markdown
Collaborator

Fixes from the production smoke test and the 163-case evaluation run after PR #474 (reviewed pass rate on the 120 comparable cases: CleanStart 77% to 82%, Eventus 80% to 83%).

What changes for visitors

  • File plus meeting request keeps the booking card. With chat history, the meeting check read only the search rewrite, which can come back as keywords ("datasheet and demo booking next week") that name no request. "send me your datasheet and also book a demo for next week" and "pls share a brochure and also set up a call with ur team thursday afternoon" lost the booking card on both bots. The visitor's own words now count as well.
  • A complaint about the chat is not a support alert. "this is useless, you never answer anything" passed the unanswered-customer vocabulary, the classifier said yes, and both bots alerted the team. The router now reads "you never answer anything" as frustration (alone or after a verdict), and the support route stands down for any message the router answers as frustration or abuse. "nobody from your team answered my emails" still reaches the team.
  • A pending certification is never held. CleanStart's own page says "ISO 27001 certification in progress; expected completion Q2 2026", and the credential check accepted a quote cut from it, so the bot said "We hold ISO 27001 certification". A HELD quote now verifies only from a sentence with no pending wording; the fallback rules and the classifier prompt say the same.
  • No invented role acknowledgment. The rule's example "Good to know you're the one signing off" was pasted onto visitors who never said they decide (7 replies). The rule now has no quotable example and forbids reading a role into a company type, task or form.

Deploy notes

  • Backend only, no migration. QA_PROMPT_VERSION is unchanged: the canonical prompt fingerprint does not include the qualification section.

Checks

  • Ruff clean.
  • Full backend suite: 13,340 passed, 4 skipped, 1 deselected (test_superadmin_impersonate_mint::test_redirect_url_uses_the_shared_app_url_setting, a known local ordering failure).
  • Regression tests for each fix; the live cases are rechecked on Eventus and CleanStart after deploy.

Not in this PR

  • Figures borrowed from guide pages ("P1 acknowledge 10 min"), false gap lines on answerable follow-ups, and scope refusals of "thanks thats all".
  • Owner content: CleanStart vendor-risk page (template, past-due Q2 2026 ISO date), Eventus "Customer Success" service entry, an IBM report in Eventus's file catalog.

…ly the search rewrite

With history, the meeting check read the search rewrite, which can come back as
keywords ('datasheet and demo booking next week') that name no request. The
production smoke test after PR #474 sent 'send me your datasheet and also book a
demo for next week' on Eventus and CleanStart and got the file cards without the
booking card. The raw message now counts as well as the rewrite.
…a support alert

'this is useless, you never answer anything' matched the unanswered-customer
vocabulary, the classifier said yes, and both bots alerted the team with the
support reply (production smoke test, 2026-09-17). The router now reads 'you
never answer anything' as frustration, alone or after a verdict, and the support
route stands down for any message the router answers as frustration or abuse.
CleanStart's vendor-risk page says 'ISO 27001 certification in progress;
expected completion Q2 2026'. The credential check accepted a quote cut from
that sentence ('ISO 27001 certification') as proof, and the bot told a visitor
'We hold ISO 27001 certification' (post-deploy evaluation, 2026-09-17). A HELD
quote now verifies only from a sentence with no pending wording (in progress,
expected, planned, once the audit completes), the fallback rules skip such
sentences, and the classifier prompt names the case.
…able example

The role acknowledgment rule carried the example 'Good to know you're the one
signing off', and the model pasted it onto an MSP, a head of IT and a visitor
filling in a form, none of whom said they approve the purchase (7 replies in the
post-deploy evaluation, 2026-09-17). The rule now asks for the visitor's own
words and forbids reading a decision role into a company type, task or form.
@digibrander
digibrander merged commit dd9e8b7 into main Sep 17, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants