fix(engine): truncate section and document titles to the Data column limit

[Rl0007]

Fixes #4. Reported as Helpdesk #0700 — IMP-2026-00001 (CMC Vellore / Nephrology, 398 pages) on wikify.m.frappe.cloud.

The failure

The import died at stage_label: "Building section tree", stage_progress: 100:

frappe.exceptions.CharacterLengthExceededError: Source Section 8t7tco15qn:
'Title' (5. The transplant coordinator also has the responsibility to maintain up to
date records and reports as required for the Renal Transplant license.)
will get truncated, as max characters allowed is 140

Page 72 has a bold numbered clause. The cleanup remediation pass promoted it to ## 5. The transplant coordinator also has…, sectionize() believed it and opened a section titled with that whole 146-character sentence, and Source Section.title is a Data field.

Parsing itself was fine — Parsed, 398 pages, mean score 0.961, canonical mean 0.943. Only the tree write failed, and because the rebuild rolled back, the site kept the older parse-time tree and rendered stale content.

The change

store.fit() — trims any content-derived string to what its column actually holds, mirroring how _validate_length picks the limit. Applied to Source Section.title and Source Document.title. Text out of a PDF or an LLM is unbounded by definition, and insert() throws rather than truncating; one runaway heading should not be able to fail a 400-page import at the last step. The full text always survives in the section markdown.

_looks_like_prose() — a heading over 120 chars that ends in terminal punctuation is body text the parser (or the cleanup pass) mis-promoted; it stays in the section it was found in rather than opening one. The embedded ToC still wins where it lists the title. Without this, the guard above would only replace a hard failure with a truncated junk section — the import would pass but the tree would be worse.

Tests

Three added to test_sectionize.py:

• test_long_sentence_heading_stays_body_text — the exact page-72 line does not open a section, and lands in the previous section's markdown.
• test_long_heading_listed_in_the_toc_is_kept — a genuinely long chapter title in the embedded ToC is still a heading.
• test_overlong_section_title_is_stored_truncated — a 300-char title survives a real replace_sections insert, stored at 140 chars with an ellipsis, body intact.

$ bench --site wikify.localhost run-tests --app wikify
Ran 185 tests in 53.417s
FAILED (failures=1)

The one failure is test_wiki_preview.test_renders_with_wiki_renderer (mermaid <pre class="mermaid">), which fails identically on main with this branch stashed — pre-existing, in the installed wiki app's renderer, unrelated to this change. test_sectionize is fully green (20/20).

Recovering IMP-2026-00001

No re-parse needed. Once this is deployed, re-running the sectionize/remediate tail rebuilds the tree off the canonical pages already stored — no re-reading of 398 pages, no new LLM spend.

🤖 Generated with Claude Code

https://claude.ai/code/session_0126A9jCM6LTxnmCPH3hWbTR

[greptile-apps]

Greptile Summary

The PR prevents unbounded document and section titles from exceeding their 140-character persistence limit.

• Truncates Source Document.title and Source Section.title before insertion.
• Adds an integration test confirming an overlong section title is truncated while its markdown remains intact.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
wikify/engine/store.py	Truncates content-derived document and section titles to the Data-field limit before persistence; no eligible follow-up defect remains.
wikify/tests/test_sectionize.py	Adds persistence coverage showing overlong section titles are truncated without losing section markdown.

_{Reviews (2): Last reviewed commit: "fix(engine): truncate section and docume..." | Re-trigger Greptile}

[greptile-apps]

Leading prose content is discarded

When a prose-like heading appears before the first accepted section, this branch buffers it while current remains unset; the next heading performs a no-op flush() and clears the buffer, causing that source content to disappear from the section tree and generated wiki.

Knowledge Base Used: Section structure and taxonomy

Prompt To Fix With AI

This is a comment left during a code review.
Path: wikify/engine/loader/sectionizer.py
Line: 94-98

Comment:
**Leading prose content is discarded**

When a prose-like heading appears before the first accepted section, this branch buffers it while `current` remains unset; the next heading performs a no-op `flush()` and clears the buffer, causing that source content to disappear from the section tree and generated wiki.

**Knowledge Base Used:** [Section structure and taxonomy](https://app.greptile.com/bwh-tech/-/custom-context/knowledge-base/bwhtech/wikify/-/docs/section-structure-and-taxonomy.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[greptile-apps]

Valid long headings lose boundaries

When a legitimate heading longer than 120 characters ends in ., ;, or ? and has no matching embedded ToC entry, this condition demotes it to body text, causing the section boundary, hierarchy, page span, and downstream generated structure to be wrong.

Knowledge Base Used: Section structure and taxonomy

Prompt To Fix With AI

This is a comment left during a code review.
Path: wikify/engine/loader/sectionizer.py
Line: 94

Comment:
**Valid long headings lose boundaries**

When a legitimate heading longer than 120 characters ends in `.`, `;`, or `?` and has no matching embedded ToC entry, this condition demotes it to body text, causing the section boundary, hierarchy, page span, and downstream generated structure to be wrong.

**Knowledge Base Used:** [Section structure and taxonomy](https://app.greptile.com/bwh-tech/-/custom-context/knowledge-base/bwhtech/wikify/-/docs/section-structure-and-taxonomy.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[NagariaHussain]

Let's increase the limit in WIKI? This means losing data.