Skip to content

feat(security): isolate and scan untrusted context against prompt injection (#94) - #95

Open
Meet-hybrid wants to merge 1 commit into
Deen-Bridge:devfrom
Meet-hybrid:feat/prompt-injection-hardening
Open

feat(security): isolate and scan untrusted context against prompt injection (#94)#95
Meet-hybrid wants to merge 1 commit into
Deen-Bridge:devfrom
Meet-hybrid:feat/prompt-injection-hardening

Conversation

@Meet-hybrid

Copy link
Copy Markdown

Summary

Closes #94 — closes the gap between the assistant's content-safety policy (which already exists and is enforced on the user prompt) and its prompt-injection / jailbreak resistance (which did not exist for retrieved or injected content). Today only the bare user prompt is screened; everything the RAG and personalization layers glue into the prompt — tafsir, zakat/purchase blocks, the user memory profile, and client-supplied context — previously reached Gemini as trusted text with no instruction-isolation and no injection scanning.

Changes

  • Instruction isolation of untrusted context (safety/untrusted.py)
    Every retrieved/client-supplied block is wrapped in distinctive <untrusted_data label="…"> delimiters, paired with a trusted system directive that content inside them is reference material — never instructions. Applied uniformly in /chat and /chat/stream.

  • Injection/jailbreak filter as a policy stage (safety/policy.yaml)
    New DB-SAFE-005 category for override/role-escape attempts. injected_block() scans each untrusted block (and the input gate covers the bare prompt); a hit refuses the turn deny-by-default before the model is ever called.

  • Scan persisted memory at write and read (memory/)
    Injected facts are rejected before storage; facts and summaries are neutralized both when persisted and when re-rendered, so an instruction persisted by an older build replays harmlessly.

  • Adversarial red-team corpus (tests/redteam/)
    Injection payloads delivered via client context, tafsir, and memory; delimiters/directive presence asserted; benign academic near-misses still pass.

Acceptance criteria

  • Injection in request.context, retrieved tafsir, or a memory fact cannot override system policy (/chat and /chat/stream).
  • Untrusted blocks are delimited and the model is told they're data (test asserts delimiters and directive in the assembled prompt).
  • A bare-prompt jailbreak is refused/neutralized; benign near-misses pass.
  • A persisted remembered_fact carrying an injection is neutralized on write and on render.
  • pytest -q tests/redteam offline and green in CI; ruff/mypy pass.

Demo (test output)

$ pytest -q tests/redteam
70 passed, 1 skipped   # skip = opt-in live audit (SAFETY_LIVE_TESTS)

$ ruff check . && ruff format --check .
All checks passed!
65 files already formatted

$ mypy main.py safety memory tafsir.py
Success: no issues found in 12 source files

Out of scope

PII redaction (#31), sensitive-topic disclaimers (handled by DB-SAFE-001), moderation-queue tie-in (#32).

…e#94)

Retrieved and client-supplied context (tafsir, zakat/purchase, user memory,
client context) previously reached Gemini as trusted text, so an injected
instruction in any of them could override the safety policy. Wrap each block in
instruction-isolation delimiters with a trusted data-only directive, add a
DB-SAFE-005 prompt-injection category that refuses deny-by-default, and
reject/neutralize injected instructions in persisted memory at write and read.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
@coderabbitai

coderabbitai Bot commented Aug 16, 2026

Copy link
Copy Markdown

Important

Review available on request

  • 🔍 Trigger review

Reviews should be triggered manually for repositories with fewer than 10 stars. Select Trigger review above or comment @coderabbitai review to review the latest changes. For a full review, comment @coderabbitai full review.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: bdc1a831-6e35-47f8-b776-ce05684f20cc


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@zeemscript

Copy link
Copy Markdown
Contributor

Strict review blocker: this branch conflicts with the base branch. Please rebase and resolve conflicts before requesting merge.

1 similar comment
@zeemscript

Copy link
Copy Markdown
Contributor

Strict review blocker: this branch conflicts with the base branch. Please rebase and resolve conflicts before requesting merge.

@zeemscript

Copy link
Copy Markdown
Contributor

Strict review blocker: this branch conflicts with the base branch and/or changes have been requested. Please rebase, resolve conflicts, and address requested changes before requesting merge.

@zeemscript

Copy link
Copy Markdown
Contributor

@Meet-hybrid this PR has merge conflicts with the main branch. Please resolve the conflicts (merge main in or rebase) and push the fix so it can be merged. Thanks!

@zeemscript

Copy link
Copy Markdown
Contributor

@TS-mfon you are assigned to the issue that PR #95 addresses, which currently has merge conflicts with the main branch. Please resolve the conflicts (merge main in or rebase) and push the fix so it can be merged. Thanks!

@zeemscript

Copy link
Copy Markdown
Contributor

@Dayz-tech-co you are assigned to the issue that PR #95 addresses, which currently has merge conflicts with the main branch. Please resolve the conflicts (merge main in or rebase) and push the fix so it can be merged. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants