Skip to content

System messages render with role "developer", breaking chat templates #451

Description

@GregoryBolshakov

Problem

The chat template renderer emits system messages with the role developer instead of system. MessageType::SystemPrompt is annotated #[serde(rename = "developer")], and HuggingFaceChatTemplate::format builds the Jinja context with context!{ role, content }, which serializes the role through serde. So any template that branches on role == 'system' never matches the system message.

Depending on the template, the result is one of:

  • the system prompt is silently replaced by the model's built-in default (e.g. Qwen2.5 emits "You are Qwen, created by Alibaba Cloud..." instead of the system message you passed),
  • the system message is dropped entirely,
  • the render fails with "conversation roles must alternate" (Falcon 7B, Gemma 3, Mistral 7B).

Separately, trim_blocks/lstrip_blocks aren't set on the environment (transformers sets both), and the tojson filter doesn't match Python's json.dumps, so some templates drift on whitespace and on serialized tool/JSON output.

Steps To Reproduce

  • Load a chat model whose template special-cases the system role (Qwen2.5, Falcon 7B, Gemma 3, Mistral 7B, ...).
  • Build a chat with a system message followed by user/assistant turns.
  • Apply the chat template.
  • Observe the system content is missing or replaced by a default; for Falcon 7B / Gemma 3 / Mistral 7B the render errors with "conversation roles must alternate".

Expected behavior

System messages render with role system, and the output matches transformers.apply_chat_template(..., tokenize=False) byte-for-byte.

Environment:

  • Crate version: kalosm main
  • Rust version: stable
  • OS info: Linux

Questionnaire

  • I would like to fix this and I have a solution

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions