Skip to content

[FIX] Strip Llama special tokens from batch prompt content - #563

Open
oussamahansal wants to merge 1 commit into
mainfrom
fix/llama-batch-prompt-special-token-injection
Open

oussamahansal wants to merge 1 commit into
mainfrom
fix/llama-batch-prompt-special-token-injection

Conversation

@oussamahansal

@oussamahansal oussamahansal commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Description

The Meta Llama batch path renders chat messages into Llama's raw instruct template, where turn boundaries are text rather than structure. Message content was interpolated verbatim, so untrusted chunk text could close its turn and open a forged one. This strips Llama special tokens from content before formatting.

Changes

  • Add _strip_llama_special_tokens
  • _format_llama_prompt hoists system messages to the front, so a later turn can't displace the instruction block.

Problem

Related issue (if any): #

Testing

  • Unit tests added/updated
  • Integration tests added (as appropriate)
  • Existing tests pass (pytest)
  • Tested manually (describe below)

Checklist

  • Code follows existing style and conventions
  • License headers present on new files
  • Documentation updated (if applicable)
  • No breaking changes (or clearly documented)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@github-actions

Copy link
Copy Markdown

Lexical Graph Coverage Report: The coverage is at 67.16% (target: 80%). Download the HTML report here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant