Skip to content

fix: keep stage and subagent text on UTF-8 character boundaries#121

Merged
wujiangang merged 2 commits into
espressif:masterfrom
BigUncleUs:fix/utf8-boundary-truncation
Jun 11, 2026
Merged

fix: keep stage and subagent text on UTF-8 character boundaries#121
wujiangang merged 2 commits into
espressif:masterfrom
BigUncleUs:fix/utf8-boundary-truncation

Conversation

@BigUncleUs

Copy link
Copy Markdown
Contributor

Summary

Fixes two byte-level text truncations in the agent stage / notification path that can split a multi-byte UTF-8 character and emit an invalid byte sequence to downstream consumers. In non-English (CJK / emoji) deployments this surfaces as replacement characters () and, depending on the consumer's UTF-8 handling, can corrupt or break the message (e.g. a strict WebSocket text-frame validator rejecting the frame).

Both issues are present on current master (4f9e2c0).

Changes

  • claw_core: claw_core_publish_stage_tool_calls() truncated tool-call arguments_json with "%.40s", cutting at a raw byte offset. The file already has a utf8_prefix_len() helper (used for the reasoning snippet just above), so the fix reuses it and the cut now lands on a character boundary.
  • claw_manager: claw_agent_mgr_completion_observer() splices a [truncated] suffix into the <subagent_completed> buffer at a fixed byte offset on overflow, which can leave an orphaned partial UTF-8 character in the kept prefix. The splice offset is now backed off to a character boundary before the suffix is written.

Implementation

  • claw_core: compute prefix_len = utf8_prefix_len(args, 40) and print with "%.*s"; the ellipsis is shown when strlen(args) > prefix_len. This mirrors the existing reasoning-snippet path, so no new helper is introduced.
  • claw_manager: before writing the suffix, decrement the splice offset while the byte at that position is a UTF-8 continuation byte ((byte & 0xC0) == 0x80), so the retained prefix always ends on a character boundary. The strlcpy size is widened to the real remaining buffer.
  • No new public API; both changes are local to existing functions.

Files

File Change
components/claw_modules/claw_core/src/claw_core_events.c Reuse utf8_prefix_len() for tool-call args truncation instead of "%.40s"
components/claw_modules/claw_manager/src/claw_agent_mgr.c Back the [truncated] splice offset off to a UTF-8 boundary

Test plan

  • Symptom reproduced end-to-end in a downstream integration: CJK tool args and long sub-agent final_text produced / invalid UTF-8 in the stage and <subagent_completed> text; gone after the fix.
  • Built in a clean ESP-IDF environment (pure logic change, no toolchain locally; relying on CI for the component build)
  • Unit test added (none added; behavior verified manually)

The stage tool-call summary truncated arguments_json with "%.40s",

which cuts at a byte offset and can split a multi-byte UTF-8

character, emitting an invalid byte sequence downstream.

Reuse the existing utf8_prefix_len() helper (already used for the

reasoning snippet) so the cut lands on a character boundary.
When the <subagent_completed> notification overflows the buffer, the

truncated suffix was spliced in at a fixed byte offset, which can

leave an orphaned partial multi-byte UTF-8 character in the prefix.

Back the splice offset off to a character boundary before writing

the suffix so no invalid byte sequence is emitted.
@CLAassistant

CLAassistant commented Jun 11, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@BigUncleUs BigUncleUs changed the title Fix/utf8 boundary truncation fix: keep stage and subagent text on UTF-8 character boundaries Jun 11, 2026
@laride laride requested a review from Copilot June 11, 2026 02:42

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes two truncation paths that previously operated on raw byte offsets and could split multi-byte UTF-8 sequences, resulting in invalid UTF-8 being emitted in stage/notification text.

Changes:

  • claw_core: truncate tool-call arguments_json using utf8_prefix_len() + "%.*s" so the cut lands on a UTF-8 character boundary.
  • claw_manager: when inserting the "[truncated]" suffix into <subagent_completed>, back the splice position off continuation bytes to avoid cutting into a multi-byte UTF-8 sequence.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
components/claw_modules/claw_manager/src/claw_agent_mgr.c Adjusts truncation suffix splice point to preserve UTF-8 boundaries in <subagent_completed> notifications.
components/claw_modules/claw_core/src/claw_core_events.c Reuses existing utf8_prefix_len() to safely truncate tool-call argument previews on character boundaries.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@laride

laride commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

sha=61ea5660818f34cabb2269d7d1685884e43ef64b

@laride

laride commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Thank you for your PR! We are currently going through our internal review process.

@wujiangang wujiangang merged commit 61ea566 into espressif:master Jun 11, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants