Skip to content

feat(#681): add opt-in toggle for grounded facts correction#696

Open
lokhor wants to merge 2 commits intomainfrom
fix/681-grounded-facts-opt-in
Open

feat(#681): add opt-in toggle for grounded facts correction#696
lokhor wants to merge 2 commits intomainfrom
fix/681-grounded-facts-opt-in

Conversation

@lokhor
Copy link
Copy Markdown
Collaborator

@lokhor lokhor commented Apr 26, 2026

Summary

  • Adds a settings toggle to enable/disable the correctGroundedFacts() digit replacement logic
  • Fixes system prompt bloat: Removed per-turn grounding directive duplication (~100 tokens/turn saved)
  • Merged conflicting NZ Context + Memory rules: Previously contradictory instructions now unified

Why

correctGroundedFacts() in ChatViewModel.kt was actively mangling digits by replacing any 3-6 digit number with years from RAG context using Levenshtein distance. Examples:

  • "186" → "1864"
  • "202407" → "2024"

This runs on every query with RAG context, even when unrelated to grounded facts.

Additionally, the system prompt had:

  1. Per-turn duplication: A grounding directive was injected every turn when RAG context existed, even though it was already in the system prompt. Wasted ~100 tokens/turn.
  2. Conflicting NZ Context rules: One rule said "state NZ facts with confidence", another said "only state details explicitly provided" — contradictory when both NZ Context + Memory blocks exist.

Changes

System prompt (ModelConfig.kt)

  • Merged NZ Context + Memory rules into separate, non-conflicting directives
  • NZ Context: authoritative, state with confidence, don't add details not in context
  • Memory: only state explicitly provided, don't embellish

ChatViewModel.kt

  • Removed per-turn grounding directive injection (already in system prompt for FULL tier, and MINIMAL variants have faithful-copying rule)
  • Saves ~100 tokens per turn

Settings (PR #1)

  • Added correctGroundedFactsEnabled toggle (default OFF) in E-4B model settings
  • Gates the digit-replacement logic

Testing

  1. Build and install debug APK
  2. Test with toggle OFF (default) — should see no digit mangling
  3. Test with toggle ON — should see previous behavior (digit replacement)
  4. Test RAG queries — should see no difference in factual accuracy
  5. Test the nephew message prompt — numbers should be accurate

Files changed

  • ModelSettingsEntity.kt — Added correctGroundedFactsEnabled field
  • ModelSettingsRepositoryImpl.kt — Added default value false
  • ModelSettingsScreen.kt — Added toggle in E-4B model card
  • ChatViewModel.kt — Added StateFlow, reads setting at init, gates correctGroundedFacts() call, removed per-turn grounding directive
  • ModelConfig.kt — Merged NZ Context + Memory rules

Closes #681

@lokhor lokhor force-pushed the fix/681-grounded-facts-opt-in branch from 916a56e to 20d6e09 Compare April 26, 2026 11:24
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 26, 2026

Debug APK ready

Download app-debug.apk

Commit: daedda9 - Build #1064

Updated on each push. Removed when PR is merged or closed.

@NickMonrad
Copy link
Copy Markdown
Owner

Don't think it helped:

You: I need help writing a message to send to my nephew. He is currently boarding with us as he recently moved from new Zealand to Australia and we gave him a safe landing pad. Now he's a bit more settled, he has his Medicare and a new job we would like him to move out. The date we want him to move out is June 13th. We need the room for our 16 month old son Lachlan and we want to have our home be our family private space, which we can't do when we have him boarding with us.
Jandal: Hi Nick,

Here are a few options for the message to your nephew, depending on the tone you want to convey.

Option 1: Direct and Warm

"Hi [Nephew's Name], We're so glad you've settled in well in Australia and are happy you have your Medicare and a new job. Now that you're getting settled, we wanted to chat about the next steps. We're hoping you can start looking at moving out around June 3rd. We need the room for Lachlan, our 6-month-old son, and we'd really like to have our home as our private family space again. Let us know what you think."

Option 2: Softer and More Collaborative

"Hi [Nephew's Name], It's been great having you here while you've been settling into Australia. We're happy to hear you've got your Medicare and a new job! As you're getting more established, we wanted to discuss a timeline for you moving out. We're hoping you might be able to move around June 3rd. We need the room for Lachlan, our 6-month-old son, and we'd really value having our home as our private family space again. Let's chat about what works best for you."

Option 3: Very Concise

"Hi [Nephew's Name], Glad you've settled in well with your Medicare and job. We'd like to discuss you moving out around June 3rd. We need the room for Lachlan, our 6-month-old son, and want to have our home as our private family space again. Let us know your thoughts."

Which tone do you think fits best?

kernel_debug_log_0.1.0_20260426_231703_043.txt

@NickMonrad
Copy link
Copy Markdown
Owner

NickMonrad commented Apr 26, 2026

Gemma 4 e2b was even worse

It did the below and then froze

You: I need help writing a message to send to my nephew. He is currently boarding with us as he recently moved from new Zealand to Australia and we gave him a safe landing pad. Now he's a bit more settled, he has his Medicare and a new job we would like him to move out. The date we want him to move out is June 13th. We need the room for our 16 month old son Lachlan and we want to have our home be our family private space, which we can't do when we have him boarding with us.
Jandal: Hi Nick, I can help you draft that message.

Here are a few options, depending on the tone you want to strike.

Option 1. Warm and Encouraging Tone

Hi [Nephew's Name],

I'm so happy to hear you're settling in well and that you've got your Medicare and a new job sorted. It's great to see you're doing so well in Australia.

We're really looking forward to you moving out on June 13rd. We'd love to have you to move into the room for Lachlan. We're really looking forward to having our home as our family space again, and we can't do that when you're boarding with us.

Let's chat soon about the details.

Best,
Nick

  1. Direct and Clear Tone

Hi [Nephew's Name],

I'm glad to hear you're settled in Australia and that you've got your Medicare and a new job.

We'd like to confirm that you'll be moving out on June 3rd. We'd like you to move into the room for Lachlan. We'd like to have our home as our family space, which isn't possible when you're boarding with us.

Let me know if you have any questions.

Nick

Option 3. More Casual Tone

Hey [Nephew's Name],

Good to hear you're settling in and that you've got your Medicare and a new job. Australia is great, I hope you're enjoying it!

Just wanted to let you know we're keen for you to move out on June 3rd. We'd love for you to move into the room for Lachlan. We're really looking forward to having our home as our family space again, can't do that when you're boarding with us.

Talk soon,
Nick

Choose the one that sounds most like you. Let me know if you want any changes!
You: What year is it
Jandal: It's 2026
You: What year is next year
Jandal: Australia forecast:
Sun 26 Apr: ☁️ Overcast 27°C / 15°C, 0mm rain | UV max: 6 (High) | 🌅 06:50 / 18:07
Mon 27 Apr: ☁️ Overcast 27°C / 16°C, 0mm rain | UV max: 6 (High) | 🌅 06:51 / 18:06
Tue 28 Apr: ☁️ Overcast 28°C / 16°C, 0mm rain | UV max: 6 (High) | 🌅 06:51 / 18:05
Wed 29 Apr: ⛅ Partly cloudy 28°C / 15°C, 0mm rain | UV max: 6 (High) | 🌅 06:52 / 18:05
Thu 30 Apr: ☁️ Overcast 29°C / 15°C, 0mm rain | UV max: 6 (High) | 🌅 06:52 / 18:04
Fri 1 May: ☁️ Overcast 31°C / 15°C, 0mm rain | UV max: 6 (High) | 🌅 06:52 / 18:03
Sat 2 May: ☁️ Overcast 33°C / 19°C, 0mm rain | UV max: 5 (High) | 🌅 06:53 / 18:02
You: How long is it until November 21
Jandal: I need a specific starting date to calculate the time until November 1st. Could you please tell me today's date?
You: You should know today's date
Jandal: I need a specific starting date to calculate the time until November 1st. Could you please tell me today's date?
You: Get system date
Jandal:

@NickMonrad
Copy link
Copy Markdown
Owner

Gemma 4 e2b test

Logs: kernel_debug_log_0.1.0_20260427_064946_805.txt

You: I need help writing a message to send to my nephew. He is currently boarding with us as he recently moved from new Zealand to Australia and we gave him a safe landing pad. Now he's a bit more settled, he has his Medicare and a new job we would like him to move out. The date we want him to move out is June 13th. We need the room for our 16 month old son Lachlan and we want to have our home be our family private space, which we can't do when we have him boarding with us.
Jandal: Hello Nick, I can help you draft a message for your nephew.

Here are a few options, depending on the tone you want to strike.

Option 1: Warm and Direct

Hi [Nephew's Name],

I'm so glad to hear you're settling in well in Australia and that you're happy with your new job and Medicare.

We're really looking forward to you moving out on June 13rd. We'd love to have the room for Lachlan to use, and we'd really like to have our home as our family space again.

Let's chat soon about the details of the move.

Best,
Nick

Option 2: More Formal and Focused on Logistics

Dear [Nephew's Name],

It's great to hear that you're settling into Australia and that you're happy with your new job and Medicare.

We'd like to confirm that your move-out date is June 3rd. We'd like to have the room for Lachlan, and we'd also like to have our home as our family space.

Please let me know when you're free to discuss the logistics.

Regards,
Nick

Option 3: Casual and Friendly

Hey [Nephew's Name],

So happy to hear you're doing well and that you're happy with everything with your new job and Medicare.

Just wanted to let you know that we're aiming for you to move out on June 3rd. We'd love to have the room for Lachlan, and we'd really like to have our home as our family space again.

Let's catch up soon!

Nick

Choose the option that best fits your relationship with your nephew. Let me know if you'd like any adjustments!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reassess LLM chat stability after prompt/context growth

2 participants