Target language ignored when using strict JSON-schema structured output with smaller models (e.g. DeepSeek V4 Flash)

[engels74]

Environment

• Lingarr image: ghcr.io/lingarr-translate/lingarr:latest
• Service type: LocalAI (OpenAI-compatible /v1/chat/completions)
• Endpoint: https://nano-gpt.com/api/v1/chat/completions
• Model: deepseek/deepseek-v4-flash

Summary

When TranslateBatchWithStructuredOutput is used, the batch_translation_response schema's line field has no language constraint. With smaller models, strict schema-constrained decoding dominates attention and the target-language directive in the system prompt is effectively ignored — every line is returned in an arbitrary non-target language (Hindi, Indonesian, Chinese, or pass-through English).

Reproduce

1. In Lingarr → Settings → Services → LocalAI, configure:
   • Endpoint: https://nano-gpt.com/api/v1/chat/completions
   • Model: deepseek/deepseek-v4-flash
   • API key: your Nano-GPT key
2. Leave AI Prompt at its default (Translate from {sourceLanguage} to {targetLanguage}, …).
3. Set source language = English, target language = Danish.
4. From Bazarr or directly in Lingarr, trigger translation of any English subtitle file (~100+ lines is enough; the issue is visible on the first batch).
5. Wait for status = Completed.
6. Open the produced .da.srt (or inspect translation_request_lines.target in the DB).

Expected: lines translated to Danish.
Actual: lines returned in Hindi (Devanagari), Indonesian, Chinese, or unchanged English — the language varies between runs but is never Danish.

A 1471-line subtitle I tested completed cleanly (en → da, status = Completed), but 0 of 1471 stored target lines contained any Danish letter (æ/ø/å); 300+ contained Devanagari.

Evidence

I intercepted Lingarr's outgoing request with a logging proxy and replayed it with only the model and response_format varied:

Model	response_format	Output
deepseek-v4-flash	absent	Danish ✅
deepseek-v4-flash	strict schema (as Lingarr sends)	Hindi / Indonesian / Chinese / pass-through English ❌
deepseek-v4-pro	strict schema	Danish ✅

Stronger system prompts, temperature: 0, and moving the directive to the user role did not help. The only mitigation that worked on Flash was adding the target language to the line field's description inside the schema:

"line": {
  "type": "string",
  "description": "Subtitle text translated into Danish (da). Must be Danish, never any other language."
}

With that single change, output was Danish across all trials.

Captured request/response from Lingarr (one batch, 100 lines)

• Request body Lingarr sends: https://logs.notifiarr.com?17f60a245fc00176#Gs4ox9X1dUhwiYT5MLYTgE89XkeAJ1pQwyUswGZsLZv7
• Upstream response (all 100 line values returned in Hindi): https://logs.notifiarr.com?cfafa656e719fa6b#68bYNPhx2zZjn5CzhEZ8hrP8yqL3RBzzzHN7iRqmdTcj

Suggested fix

In TranslateBatchWithStructuredOutput, include the target language in the JSON-schema field descriptions, e.g.:

properties: new {
    position = new { type = "integer", description = "Position number of the subtitle item" },
    line     = new { type = "string",  description = $"Subtitle text translated into {targetLanguageName} ({targetLanguageCode}). Must be {targetLanguageName} only." }
}

This anchors the language directive inside the schema, where attention is concentrated during constrained decoding. It's a no-op for strong models (they already follow the system prompt) and a fix for smaller, faster models like DeepSeek V4 Flash.

[rowanfuchs]

Good point, cheers, I will introduce it in the next version.

[engels74]

@rowanfuchs One extra implementation note after looking closer:

The captured request/response seems to show that the model is not failing JSON generation. It returns valid schema-shaped JSON with all positions preserved, but every line is Hindi despite the system prompt saying English → Danish. So the failure appears specifically to be target-language anchoring, not parsing, truncation, or invalid JSON.

My wording about “strict schema-constrained decoding dominates attention” should probably be treated as an inference rather than proven DeepSeek/NanoGPT internals. The reproducible part is:

• without response_format, the same model translates to Danish
• with Lingarr’s strict json_schema, it returns valid JSON but in the wrong language
• adding the target language directly to line.description fixes it in my tests

DeepSeek’s JSON Output docs may also be relevant here: they document response_format: {"type":"json_object"} and recommend including “json” plus an example in the prompt, rather than OpenAI-style json_schema.

So for compatibility, the safest fix may be to add source/target language constraints directly into the batch schema descriptions, especially line, and longer term possibly make batch response_format provider-aware/configurable.

Also worth noting: batch mode currently bypasses the AI Request Body Template, so users cannot fully work around this from the UI until batch requests become configurable.

[rowanfuchs]

While I do see the addition of defining language in the structured output I sadly do not see the advantage of DeepSeek's experimental feature. DeepSeek sady doesn't support proper structured output. That experimental feature will only introduce more failures as also mentioned in their docs.

[engels74]

While I do see the addition of defining language in the structured output I sadly do not see the advantage of DeepSeek's experimental feature. DeepSeek sady doesn't support proper structured output. That experimental feature will only introduce more failures as also mentioned in their docs.

Totally understand :). Thanks for taking the issue into consideration.