Add Idefics3 original and training chat template with generation markers

[aazizyan]

What does this PR do?

Adds a training chat template for Idefics3, wiring it into get_training_chat_template.

The new template patches idefics3.jinja with a single modification: split the assistant message into its own branch so the {% generation %} / {% endgeneration %} markers wrap the assistant content (everything after the Assistant: prompt cue, up to and including the trailing <end_of_utterance>\n). The vision-content rendering inside the assistant branch is duplicated from the original non-assistant branch unchanged. This enables return_assistant_tokens_mask=True to produce correct masks for SFT assistant-only loss.

Idefics3 does not ship tool-calling support, so no prefix-preservation changes are needed.

Changes

trl/chat_templates/:

• idefics3.jinja – new original template
• idefics3_training.jinja – new training template

trl/chat_template_utils.py:

• Module-level template variable: idefics3_chat_template, used for identity comparison.
• One new branch in get_training_chat_template() mapping the source template to its training variant.

tests/test_chat_template_utils.py — one new entry in TestGetTrainingChatTemplate's parametrize block (idefics3).

Docs — Idefics3 added to the supported-families list, training-template section added in:

• docs/source/chat_templates.md
• trl/chat_templates/README.md

Part of #5471

cc: @qgallouedec

---

Note

Low Risk
Template-only addition following existing VLM patterns; no auth, data, or core training logic changes beyond automatic template swap for matching tokenizers.

Overview
Adds Idefics3 to TRL’s bundled chat-template support so SFT with assistant_only_loss=True can use correct assistant token masks on multimodal Idefics3 checkpoints.

New idefics3.jinja and idefics3_training.jinja mirror the shipped format (role lines, text/image blocks, <end_of_utterance>). The training template gives assistant turns their own branch and wraps assistant output in {% generation %} / {% endgeneration %} (after the Assistant: cue). No tool-calling or prefix-preservation changes.

get_training_chat_template maps the reference template to the training variant; docs list Idefics3 as supported; TestGetTrainingChatTemplate gains an idefics3 parametrize case.

^{Reviewed by Cursor Bugbot for commit 311f0a8. Bugbot is set up for automated code reviews on this repo. Configure here.}

[qgallouedec]

thanks, can you ensure we keep an alphabetical ordering?

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit b91d52d. Configure here.}

[aazizyan]

done