[ENHANCEMENT] Investigation: TOON token-efficient context serialization — plugin vs CLI vs external

[jordan-carson]

Problem (one or two sentences)

When Roo Code serializes structured data (file lists, tool results, memory entries) into LLM prompt context, it uses standard JSON — a verbose format that consumes significantly more tokens than necessary. There is currently no mechanism to substitute a token-efficient encoding format, resulting in avoidable cost and context window pressure at scale.

Context (who is affected and when)

As context windows grow and structured data is injected into prompts, token consumption becomes a cost and performance concern at scale — affecting every agent invocation across potentially hundreds of concurrent jobs.
This investigation is relevant to any Roo Code deployment where structured data is serialized into LLM prompts programmatically

Desired behavior (conceptual, not technical)

Roo Code (or its CLI) should be capable of serializing structured context data in a token-efficient format before it is injected into the LLM prompt — specifically for uniform arrays of objects (file lists, memory entries, task records, tool results) where TOON's tabular encoding can reduce token usage by ~40% versus standard JSON without loss of LLM comprehension accuracy.
The goal is not to change what the agent receives conceptually, but to reduce the token cost of how that structured data is represented in the prompt.

Constraints / preferences (optional)

No response

Request checklist

• I've searched existing Issues and Discussions for duplicates
• This describes a specific problem with clear context and impact

Roo Code Task Links (optional)

Identify where in the Roo Code source structured data is serialized into prompt context today (e.g. system prompt construction, context window management, tool result formatting)
Determine if JSON serialization of structured arrays is happening inline in the plugin, in the CLI layer, or assembled externally
Note any existing abstraction points (serializers, context builders, formatters) where a pluggable encoding strategy could be introduced without deep surgery

Acceptance criteria (optional)

No response

Proposed approach (optional)

No response

Trade-offs / risks (optional)

No response

[roomote-v0]

I have analyzed the codebase to identify where structured data is serialized into LLM prompt context and evaluate where TOON-style tabular encoding could provide token savings. Here are my findings and a proposed plan:

Investigation Findings

Where structured data is serialized into prompt context today

1. System prompt construction (src/core/prompts/system.ts, src/core/prompts/sections/)

• The system prompt is assembled from text template sections (capabilities, rules, modes, skills, objectives, etc.)
• This is overwhelmingly static prose/markdown, not structured arrays of objects. TOON would not apply here.

2. Environment details (src/core/environment/getEnvironmentDetails.ts)

• Generates <environment_details> blocks appended to user messages containing: visible files, open tabs, active terminals, git status, workspace file lists, cost info, and reminder/todo sections.
• File lists are already newline-delimited plain text (one path per line), not JSON arrays. Terminal output is also plain text.
• The file list is produced by formatResponse.formatFilesList() in src/core/prompts/responses.ts, which sorts and joins paths with \n. This is already compact.

3. Tool result formatting (src/core/prompts/responses.ts)

• formatResponse.toolResult() returns text or text+image blocks.
• Status responses (toolDenied, toolError, rooIgnoreError, etc.) use JSON.stringify but these are small single objects (3-5 keys), not arrays. TOON targets uniform arrays, so these would not benefit.

4. Individual tool outputs (across src/core/tools/)

• ListFilesTool: Returns newline-delimited file paths (already compact).
• ReadFileTool: Returns file content as plain text.
• ExecuteCommandTool: Returns command output as plain text.
• SearchFilesTool, CodebaseSearchTool: Return search results as text.
• UseMcpToolTool / accessMcpResourceTool: MCP server results are passed through as-is from the server. These could theoretically contain JSON arrays, but the format is controlled by external MCP servers, not Roo Code.

5. API message formatting (src/api/transform/)

• Tool call arguments are serialized with JSON.stringify for the API wire format (OpenAI, Anthropic, etc.), but these are protocol requirements, not prompt context.

6. Condense/summarization (src/core/condense/index.ts)

• Uses JSON.stringify for metadata formatting during context condensation, but again these are small objects, not uniform arrays.

Key takeaway

The Roo Code codebase already avoids verbose JSON for most prompt-injected structured data. File lists use newline-delimited text, tool results are plain text, and the system prompt is prose. The JSON.stringify calls that exist fall into a few categories:

• Small status objects (tool denied/approved/error responses) -- 3-5 keys, not arrays
• API wire protocol (tool call arguments for OpenAI/Anthropic formats) -- required by the API
• Logging/debugging -- not sent to the LLM
• Disk persistence -- not sent to the LLM

There are no current code paths that serialize large uniform arrays of objects (file metadata arrays, task record arrays, memory entry arrays) into prompt context using JSON. The workspace file list, which could be the largest structured dataset, is already a flat newline-delimited list of paths.

Where TOON could potentially help (if/when these features exist)

TOON-style encoding would become relevant if Roo Code adds features that inject structured tabular data into prompts, such as:

• A structured task history/memory system that injects arrays of past task records
• Structured file metadata (path + size + modified date + type) instead of plain file paths
• Batch MCP tool results that return arrays of uniform objects
• Structured search results with multiple fields per result

Proposed plan

Given the findings, I see two paths forward:

Option A: No-op for now (recommended)
Document the investigation results -- TOON does not currently have a high-impact application point because Roo Code already uses compact text formats for prompt-injected data. Revisit when structured array data is introduced into prompt context.

Option B: Create a pluggable serialization abstraction
Introduce a lightweight ContextSerializer interface at the abstraction points identified above (formatResponse, getEnvironmentDetails, tool result formatting) so that a TOON encoder could be swapped in later without refactoring. This is forward-looking infrastructure.

I would recommend Option A unless there are specific upcoming features that will introduce structured arrays into prompt context. Would appreciate guidance on which direction makes sense.

Please mention @roomote in a comment saying that you approve this plan, and I will proceed with the implementation.

[roomote-v0]

Hey @roomote-agent, I independently investigated this and can confirm the previous analysis is accurate. Here is a summary of my findings:

Where structured data enters LLM prompts today

I traced every JSON.stringify call in src/core/ (237 hits) and categorized them by whether the output ends up in LLM prompt context:

1. System prompt construction (src/core/prompts/sections/) -- All prose/markdown. Modes are bullet-point text, skills use XML, rules are plaintext. No JSON arrays.

2. Environment details (src/core/environment/getEnvironmentDetails.ts) -- File lists use formatResponse.formatFilesList() which produces newline-delimited paths. Todo/reminders use a markdown table via formatReminderSection(). Terminal output is plain text. No JSON.

3. Tool result responses (src/core/prompts/responses.ts) -- The formatResponse methods (toolDenied, toolError, rooIgnoreError, etc.) do use JSON.stringify, but each produces a small single object with 3-5 keys. These are not uniform arrays, so TOON tabular encoding would not apply.

4. MCP tool/resource results (UseMcpToolTool.ts, accessMcpResourceTool.ts) -- Text content is extracted from MCP results and joined as plain text before being pushed as tool results. The structured format is controlled by external MCP servers, not Roo Code.

5. Tool definitions -- These ARE arrays of structured objects (via getNativeMcpTools() in src/core/prompts/tools/native-tools/mcp_server.ts), but they are sent through the API native tools parameter, not serialized into the prompt text. The wire format is dictated by the API protocol.

6. UI messaging -- Many JSON.stringify calls in tool files (e.g., task.ask("tool", JSON.stringify({...}))) are for webview display, not LLM prompt injection.

Conclusion

There are currently no code paths that serialize large uniform arrays of objects into prompt context using JSON. The codebase already uses compact text formats (newline-delimited lists, markdown tables, plain text) for structured data that enters prompts.

Recommendation

I agree with the previous analysis -- Option A (no-op for now) is the right call. TOON does not have a high-impact application point today.

If a pluggable serializer is desired for future-proofing, the cleanest abstraction points would be:

• formatResponse.formatFilesList() in src/core/prompts/responses.ts (if file metadata becomes richer than just paths)
• formatReminderSection() in src/core/environment/reminder.ts (if todo items gain more structured fields)
• The MCP result processing in UseMcpToolTool.processToolContent() (if MCP servers start returning tabular data)

These are the places where a ContextSerializer interface could be introduced later without deep surgery.