Skip to content

feat(documents): add native document analysis - #108

Open
binsarjr wants to merge 4 commits into
anvie:devfrom
binsarjr:feat/native-document-analysis
Open

feat(documents): add native document analysis#108
binsarjr wants to merge 4 commits into
anvie:devfrom
binsarjr:feat/native-document-analysis

Conversation

@binsarjr

@binsarjr binsarjr commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

What this adds

Evonic can now send a supported document to a compatible model through analyze_document and ask a focused question about it. The original file is passed through the provider's native document input, while only the useful answer returns to the main agent conversation.

The tool works with files uploaded from Web, Telegram, Discord, and WhatsApp, as well as filesystem paths visible to the agent. Document analysis can be enabled per agent, while the primary model and two fallbacks are configured once in System Settings.

Model routing

Document model routing

Per-agent control

Document Analysis toggle

Model capabilities

Native document capabilities

How files are routed

Native analysis is used for PDFs, Office and presentation files, RTF, and non-text spreadsheets such as XLS/XLSX. Evonic selects only models with the matching capability and follows the configured fallback order.

Text, code, CSV, TSV, and IIF stay on exact readers instead: uploaded files use read_attachment, while local paths use read_file. This keeps the tool choice predictable for smaller models and avoids spending a native document call on content that can already be read exactly.

Anthropic models handle PDF input. OpenAI-compatible and Codex models handle the native document categories enabled for that model.

This uses provider-native file inputs. It does not add OCR, document parsers, converters, or new runtime dependencies.

Why not just use read_attachment?

read_attachment is the right tool when the agent needs exact text from an uploaded file. For local text and code, read_file provides the same exact-reading behavior.

Binary documents are different. Extracting their text into the main agent context can consume a lot of space and flatten useful structure such as page layout, tables, slides, and relationships between visual elements. analyze_document sends the original file to a native document model instead, then returns only the answer the agent asked for.

The tools therefore have separate jobs: exact readers for text-based content, and analyze_document for understanding structured non-text documents.

Safety

Attachments remain metadata-only until the agent chooses a tool. Before a document is sent, Evonic validates ownership, session access, file type, signature, size, path scope, and query length. Document contents are treated as untrusted data, so instructions embedded inside a file cannot override the agent request.

Verification

  • Focused document, model, and channel tests: 123 passed
  • Local full suite: 2,181 passed and 86 skipped; the same 3 unrelated baseline failures remain
  • GitHub Python 3.11 suite: 2,173 passed and 95 skipped; the 2 failures are existing dev failures in the SFTP mock and artifact prompt token budget
  • Go tests passed
  • Python compilation, JSON validation, JavaScript syntax checks, and git diff --check passed

@binsarjr
binsarjr force-pushed the feat/native-document-analysis branch from 45d94d1 to bc55e5b Compare August 13, 2026 10:45
@binsarjr
binsarjr marked this pull request as ready for review August 13, 2026 10:57
@anvie

anvie commented Aug 15, 2026

Copy link
Copy Markdown
Owner

This is nice to have, can you please group the fallback models UI into one group?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants