Skip to content

fix: shorten generated tool-call ids to 9 alphanumeric chars (fixes #1375)#1399

Open
u7k4rs6 wants to merge 1 commit into
ml-explore:mainfrom
u7k4rs6:fix/tool-call-id-length
Open

fix: shorten generated tool-call ids to 9 alphanumeric chars (fixes #1375)#1399
u7k4rs6 wants to merge 1 commit into
ml-explore:mainfrom
u7k4rs6:fix/tool-call-id-length

Conversation

@u7k4rs6

@u7k4rs6 u7k4rs6 commented Jun 12, 2026

Copy link
Copy Markdown

Summary

  • ToolCallFormatter._format in server.py falls back to str(uuid.uuid4()) when a tool call has no id field
  • str(uuid.uuid4()) is 36 characters with hyphens; Mistral v3 chat templates enforce a 9-character alphanumeric constraint and raise TemplateError when this is violated
  • This causes apply_chat_template to raise on the second turn of any multi-turn tool exchange with affected models, which the server surfaces as HTTP 404

Change

# before
tc_id = tc.pop("id", None) or str(uuid.uuid4())
# after
tc_id = tc.pop("id", None) or uuid.uuid4().hex[:9]

uuid.uuid4().hex[:9] yields 9 lowercase hex characters (e.g. 0819261ae) — valid under the Mistral v3 constraint and accepted by all other templates tested.

Affected models

mlx-community/Ministral-8B-Instruct-2410-4bit, Mistral-7B-Instruct-v0.3, Mixtral, Mistral-Nemo (any checkpoint sharing the v3 template family). Devstral-Small-2-...-2512 is unaffected as its template has no id-length constraint.

Closes #1375

Mistral v3 chat templates enforce a 9-character alphanumeric id constraint
on tool calls. str(uuid.uuid4()) produces a 36-char hyphenated string that
fails this check, causing apply_chat_template to raise on multi-turn
tool exchanges and the server to return HTTP 404.

Fixes ml-explore#1375
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mlx_lm.server assigns 36-char UUID tool-call ids, which Mistral v3 templates reject (length-9 constraint)

1 participant