fix: support large input texts with chunking by RSKKSOFFICIAL · Pull Request #100 · nextcloud/translate2

RSKKSOFFICIAL · 2026-09-05T17:04:10Z

Fixes #71

What this does

Large input texts (>250 words) were being silently truncated by the model
because the output hit the decoding length limit mid-document. This adds a
chunking layer that splits the input into smaller pieces before translation,
then joins the results back into a single output.

Changes

lib/Service.py

Added _chunk_text(): splits input at sentence boundaries into chunks of
max 80 words. Hard-splits any single sentence that exceeds the limit.
Added _join_chunks(): joins translated chunks in document order. Uses an
empty string separator for no-space languages (zh, ja, th, etc.) and a
single space for all others. Chunk order is always preserved regardless of
source/target script direction — each chunk is already translated correctly
by the model independently.
Updated translate(): applies chunking when input exceeds the threshold,
caps max_decoding_length proportionally per chunk to prevent runaway
repetition loops, and enforces a minimum repetition_penalty per chunk.

config.json

Added chunking section with four configurable parameters:
- chunk_threshold (250): word count above which input is chunked
- chunk_size (80): max words per chunk
- min_repetition_penalty (1.5): lower bound for repetition penalty per chunk, prevents output loops on dense scripts like Devanagari
- max_decoding_multiplier (3): output token cap as a multiple of input tokens per chunk

Testing

Tested with 350+ word inputs across 7 language pairs:

English → German, French, Hindi, Arabic
Arabic → English, Persian
Persian → English

All pairs now produce complete output covering the full input. Before this
change every pair was truncated at roughly Section 03/04 of a 5-section test
document.

RTL languages (Arabic, Persian) are handled correctly, chunks are always
joined in forward document order since each chunk is translated independently.

Signed-off-by: RSKKSOFFICIAL <rsksofficial02@gmail.com>

marcelklehr · 2026-09-17T11:34:29Z

+        current_count = 0
+
+        for sentence in sentences:
+            word_count = len(sentence.split())


For Chinese/Japanese/Thai source text, split() returns 1 regardless of length, I think

marcelklehr · 2026-09-17T11:35:49Z

+                        self.config["inference"].get("repetition_penalty", 1.0), min_repetition_penalty
+                    ),
+                }
+                results = self.translator.translate_batch(


Would be nice to use one batch for all chunks, instead of running one batch per chunk in sequence

marcelklehr · 2026-09-17T11:36:17Z

 logger = logging.getLogger(os.environ["APP_ID"] + __name__)

+# Languages that do not use spaces between words — join chunks without a space separator
+_NO_SPACE_LANGUAGES = {"zh", "ja", "th", "my", "km", "lo", "bo"}


What about yue (Cantonese), dz (Dzongkha — same script as the included bo), shn (Shan)?

Good point. yue, dz, and shn also use writing systems where spaces shouldn't be inserted between chunks. I'll add them to _NO_SPACE_LANGUAGES.

btw, are their any more languages which uses no space?

Signed-off-by: RSKKSOFFICIAL <rsksofficial02@gmail.com>

RSKKSOFFICIAL · 2026-09-17T15:16:41Z

Hi @marcelklehr, I’ve addressed the review comments and tested the changes locally. Everything is working as expected on my side. Could you please take another look and let me know if there’s anything else you’d like me to adjust?

Signed-off-by: RSKKSOFFICIAL <rsksofficial02@gmail.com>

github-actions · 2026-09-20T06:30:27Z

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

(If you believe you should not receive this message, you can add yourself to the blocklist.)

kyteinsky

thanks for the PR, looks good overall but posted some comments maybe we can discuss about.

did you try out translate_iterable? It was mentioned in the original issue: https://opennmt.net/CTranslate2/python/ctranslate2.Translator.html#ctranslate2.Translator.translate_iterable
it seems to have a few upsides like parallel translation (which we don't do here I think) and parallel prefetching, but should simplify the implementation here at least.

kyteinsky · 2026-09-20T13:09:54Z

+    "__comment::chunking": "Options controlling how large inputs are split before translation",
+    "__comment::chunking::chunk_threshold": "Word count above which the input is split into chunks before translation",
+    "__comment::chunking::chunk_size": "Maximum number of words per chunk when splitting large inputs",
+    "__comment::chunking::min_repetition_penalty": "Lower bound for repetition_penalty applied per chunk; overrides inference.repetition_penalty if that value is lower, to suppress runaway output loops on dense scripts such as Devanagari",


why add min_repetition_penalty when repetition_penalty is in the same file configurable by the admin?

kyteinsky · 2026-09-20T14:38:53Z

+            # split() always returns 1 for these scripts regardless of actual length.
+            text_size = len(cleaned) if is_no_space_source else len(cleaned.split())
+            chunks = (
+                self._chunk_text(cleaned, chunk_size, data.get("origin_language", ""))


the origin_language here is always set to "Detect Language" since the model used here does not need it. We could try to add the target languages to the origin language enum but the "Detect Language" option could be selected by the user and we'd have the same sitation when the origin language is not known.

translate2/lib/main.py

Line 198 in daf679d

"origin_language": [DETECT_LANGUAGE],

It would be better if we do the detection of CJK languages with the existence of space.

kyteinsky · 2026-09-21T08:53:56Z

+            # Devanagari ~1.5x) may need a higher value. Floor of 64 handles very
+            # short inputs.
+            max_input_tokens = max(len(t) for t in all_input_tokens)
+            batch_max_decoding = max(max_input_tokens * max_decoding_multiplier, 64)


I was a bit wrong in pushing the max_decoding_length to 10k before, it should be aligned with what the model is trained for, which seems to be 256 for the 3B and the 7B models: (page 34 in https://arxiv.org/pdf/2309.04662)
It should work even with the large lengths but in the previous change we didn't change the input length which is 1024 by default: https://opennmt.net/CTranslate2/python/ctranslate2.Translator.html#ctranslate2.Translator.translate_batch
we can set the max decoding length to 200 to be conservative but even if we cross the 256 limit, the output would "just" be subpar but would work.
the values are in tokens so we would need to think in terms of characters, or maybe translate_iterable can handle that for us.

Co-authored-by: Anupam Kumar <kyteinsky@gmail.com> Signed-off-by: Ravi Shankar Kumar <154051646+RSKKSOFFICIAL@users.noreply.github.com>

Signed-off-by: RSKKSOFFICIAL <rsksofficial02@gmail.com>

for more information, see https://pre-commit.ci

RSKSOFFICIAL added 2 commits September 5, 2026 16:54

fix: support large input texts with chunking

1a17c3b

Signed-off-by: RSKKSOFFICIAL <rsksofficial02@gmail.com>

fix: Replaced single quotes with double quotes

ffd2e3a

Signed-off-by: RSKKSOFFICIAL <rsksofficial02@gmail.com>

RSKKSOFFICIAL mentioned this pull request Sep 5, 2026

Support large input texts with more than 250 words #71

Open

fix: resolve ruff linting errors in Service.py

c135e8b

Signed-off-by: RSKKSOFFICIAL <rsksofficial02@gmail.com>

marcelklehr reviewed Sep 17, 2026

View reviewed changes

fix: improve chunk batching and language handling

e42be61

Signed-off-by: RSKKSOFFICIAL <rsksofficial02@gmail.com>

fix: add blank line after docstring summary in _chunk_text

5b870ea

Signed-off-by: RSKKSOFFICIAL <rsksofficial02@gmail.com>

github-actions Bot added the feedback-requested label Sep 20, 2026

kyteinsky reviewed Sep 21, 2026

View reviewed changes

RSKKSOFFICIAL and others added 3 commits September 21, 2026 22:15

Apply suggestion from @kyteinsky

38021cf

Co-authored-by: Anupam Kumar <kyteinsky@gmail.com> Signed-off-by: Ravi Shankar Kumar <154051646+RSKKSOFFICIAL@users.noreply.github.com>

refactor: improved chunk translation and used translate_iterable

2b94777

Signed-off-by: RSKKSOFFICIAL <rsksofficial02@gmail.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

621322f

for more information, see https://pre-commit.ci

Navigation Menu

fix: support large input texts with chunking - #100

fix: support large input texts with chunking #100
RSKKSOFFICIAL wants to merge 8 commits into
nextcloud:mainfrom
RSKKSOFFICIAL:fix/chunk-long-text

RSKKSOFFICIAL commented Sep 5, 2026

Uh oh!

marcelklehr Sep 17, 2026

Uh oh!

marcelklehr Sep 17, 2026

Uh oh!

marcelklehr Sep 17, 2026

Uh oh!

RSKKSOFFICIAL Sep 17, 2026

Uh oh!

RSKKSOFFICIAL commented Sep 17, 2026

Uh oh!

github-actions Bot commented Sep 20, 2026

Uh oh!

kyteinsky left a comment

Uh oh!

kyteinsky Sep 20, 2026

Uh oh!

Uh oh!

kyteinsky Sep 20, 2026

Uh oh!

kyteinsky Sep 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

RSKKSOFFICIAL commented Sep 5, 2026

What this does

Changes

Testing

Uh oh!

marcelklehr Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

marcelklehr Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

marcelklehr Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

RSKKSOFFICIAL Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

RSKKSOFFICIAL commented Sep 17, 2026

Uh oh!

github-actions Bot commented Sep 20, 2026

Uh oh!

kyteinsky left a comment

Choose a reason for hiding this comment

Uh oh!

kyteinsky Sep 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kyteinsky Sep 20, 2026

Choose a reason for hiding this comment

Uh oh!

kyteinsky Sep 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants