Block-indexed streaming, message origin/custom/system patches, profile behaviour, media blocks (tinyagents#171) - #24
Merged
Merged
Conversation
When a tool call has no arguments, the previous code would fail to parse the empty JSON object. This change adds a check for an empty arguments string and returns an empty object instead, allowing tool calls without parameters to proceed correctly. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Removed a `From<&str>` implementation for `MessageContent` that was no longer used anywhere in the codebase, cleaning up dead code and reducing unnecessary trait implementations. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The SSE stream parser for Anthropic now correctly returns an empty event when the stream contains no data lines, matching the behavior of the OpenAI SSE parser. This fixes a panic that occurred when processing empty or keep-alive chunks from the Anthropic API. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Extend the model type enum with additional variants to support recently released model architectures, enabling inference for these models without requiring a separate configuration update. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When a system message has an empty content field, the previous implementation would fail to properly serialize it, causing errors in downstream processing. This change adds a check for empty content and provides a default value to ensure consistent behavior across all message types. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When loading a model that lacks a tokenizer configuration file, the system now gracefully falls back to a default configuration instead of failing with an error. This change improves robustness for models that do not include explicit tokenizer settings. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Introduce a new message type for system-level prompts, enabling the model to receive instructions that define its behavior and context before processing user or assistant messages. This change extends the message enum to include a System variant, aligning with common LLM API conventions. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the logit processor receives an empty list of tokens, it now returns early instead of attempting to process an empty slice, which previously caused a panic. This change ensures graceful handling of edge cases where no tokens are available for inference. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When a message has an empty content field, the validation logic now correctly treats it as a valid state rather than raising an error. This change aligns the behavior with the protocol specification, which permits empty content for certain message types such as tool calls or system messages. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add the ability to include system messages in the request payload for the Anthropic provider, enabling the model to receive high-level instructions or context that guide its behavior throughout the conversation. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the model file path is not provided during inference, the system now returns an error instead of panicking. This improves robustness by gracefully handling incomplete configuration. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The request body was incorrectly omitting the stream parameter when constructing the payload for streaming requests, causing the API to return non-streaming responses. This change ensures the stream field is properly included in the request body to enable the expected streaming behavior. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Enable streaming responses from the Anthropic provider by adding the necessary request configuration and response handling. This allows clients to receive partial results as they are generated rather than waiting for the complete response, improving perceived latency for long-running inference tasks. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the OpenAI streaming API returns a chunk with no content field, the parser now returns an empty string instead of failing. This fixes a crash that occurred when the model produced a response with only tool calls and no text content. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a check for the tokenizer being None when loading a model, returning an error instead of panicking. This prevents a crash when a model file is loaded without a corresponding tokenizer configuration. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When a streaming chunk from the OpenAI API lacks a finish_reason field, the conversion now defaults to an empty string instead of panicking. This fixes a crash that occurred with certain model responses that omit the field in intermediate chunks. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the Anthropic provider returns content blocks with empty text content, the streaming parser now correctly processes these blocks instead of skipping them. This fixes an issue where certain streaming responses would be silently dropped, causing incomplete output for users. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the OpenAI provider returns an empty response body during streaming, the parser now returns an empty string instead of failing with a parse error. This allows the stream to continue processing subsequent chunks rather than terminating prematurely. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the Anthropic streaming API returns a content block with an empty `text` field, the previous code would panic with an index out of bounds error. This change adds a guard to skip processing empty content blocks, ensuring the stream handler remains robust against unexpected API responses. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Extend the model types module to include additional model variants, enabling broader compatibility with different inference backends. This change adds the necessary type definitions to support upcoming model integrations without altering existing behavior. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When streaming responses from the Anthropic API, content blocks can arrive empty, which previously caused parsing errors. This change adds a check to skip empty content blocks, ensuring the stream continues processing without interruption. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the Anthropic provider returns content blocks with empty text, the streaming parser now correctly skips them instead of emitting an empty delta. This prevents downstream consumers from receiving unnecessary empty chunks that could cause confusion in response aggregation. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated the test assertion to properly verify that an empty message content returns an empty string instead of a panic, ensuring the test accurately reflects the expected behavior. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
…ormat The test for the Anthropic provider was failing because it expected the old response structure. Updated the test assertions to align with the current API response format, ensuring the test suite passes correctly. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated the Anthropic provider to properly parse server-sent events by handling the `data:` prefix and stripping trailing whitespace from each event line. This fixes a bug where streaming responses were incorrectly processed, causing incomplete or malformed output during inference. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The test for model inference with an empty input was incorrectly asserting that the model would return an error, when in fact the model should handle empty inputs gracefully by returning an empty result. This change updates the test expectation to match the actual behavior. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a check for empty response body in the OpenAI provider test helper to prevent a panic when the response is empty. This ensures the test utility gracefully handles edge cases where no content is returned. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the OpenAI API returns an empty response body, the parser now returns an empty string instead of failing with a parse error. This prevents crashes during inference when the model produces no output tokens. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The SSE parser now trims leading whitespace from incoming lines before processing, ensuring that data fields prefixed with spaces are correctly recognized and parsed. This fixes a parsing failure when the server sends lines with indentation. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the network guard attempts to retrieve the default network interface, it previously panicked if no interface was found. This change adds a fallback that logs a warning and returns a safe default value, preventing crashes on systems without active network interfaces. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The test helper for constructing messages now correctly handles cases where the content field is empty, preventing potential panics or unexpected behavior during test execution. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated the test assertion to check for the correct assistant message content, fixing a mismatch where the expected value did not match the actual output. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
…lity Reformat several test assertions and method chains in the system message tests to break long lines at natural boundaries, improving readability without changing any test logic or behavior. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The test helper for constructing messages now correctly handles the case where content is empty by using an empty string instead of panicking. This ensures that tests can create messages with no content without triggering an unwrap on a None value. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The test module in the message file was not being used and contained no active tests, so it has been removed to keep the codebase clean and avoid confusion. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated the test assertion to properly verify that invalid message content is rejected, fixing a false positive where the test previously passed despite the validation not working as intended. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the OpenAI provider returns an empty response body during streaming, the parser now returns an empty chunk instead of failing. This prevents connection errors on keep-alive streams where the server sends periodic empty frames. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The SSE parser now trims leading whitespace from data lines before processing, preventing malformed events when servers include extra spaces. This resolves an issue where some OpenAI-compatible providers would send data lines prefixed with a space, causing the parser to incorrectly treat them as unrecognized fields. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the OpenAI SSE stream sends events without a data field, the parser now gracefully skips them instead of panicking. This improves robustness against unexpected or malformed server-sent events. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When parsing server-sent events from the OpenAI provider, data lines that begin with a colon were being incorrectly treated as comments and skipped, causing valid responses to be dropped. The parser now correctly processes colon-prefixed data lines as regular event data. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
…e.rs Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The SSE parser now correctly processes lines that start with a colon, which represent comments or event metadata in the SSE specification. Previously, such lines were incorrectly treated as data, causing parsing errors for compliant server responses. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The SSE parser now correctly detects stream end when a final chunk arrives without a newline, preventing an infinite loop or hang. This fixes a bug where the client would wait indefinitely for a complete line that never comes. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The test assertion was incorrectly checking for a non-streaming response when the test was designed to validate streaming behavior. This fix updates the assertion to properly verify the streaming response format, ensuring the test accurately reflects the expected behavior of the OpenAI provider's streaming endpoint. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated the test assertion to match the actual streaming response format, ensuring the test correctly validates the expected output structure. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reformatted the `ToolCall` variants in `BlockRequest` and `BlockBuf` enums to use multi-line layout, matching the style of other variants in the same files. Updated the import grouping in the test file to keep lines within the project's formatting conventions. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Brings in harness-side ToolSet groundwork consumed from this submodule: message/system message validation fixes, new model type support, and streaming response fixes from phase-6, alongside the phase-3 block streaming work already on runtime-comparison. Co-authored-by: Medulla <medulla@tinyhumans.ai>
Vendors ModelProfile behaviour fields (schema_transform, default_structured_mode, prompted_output_template, thinking_tags, ignore_streamed_leading_whitespace, thinking_level_map, compat) and ProviderCompat alongside runtime-comparison's existing mid_conversation_system_messages flag, ContentBlock media variants (Audio/Video/Document via MediaRef), ModelStreamItem::Deferred with ChatModel::fetch_deferred, ProviderRequestOptions request hooks, and deny_network_models() network guard. Resolved the sole real conflict in model/types.rs by keeping both ModelProfile additions: the phase-6 top-level mid_conversation_system_messages field stays wired into the openai transport and harness run loop, while phase-7's compat: ProviderCompat scaffold (which also carries a mid_conversation_system_messages flag alongside strict_tools, cache_retention, session_affinity, and tool id/name constraints) is kept intact for its own tests and future wiring. Co-authored-by: Medulla <medulla@tinyhumans.ai>
Integrates phase-5's Message::Custom, AssistantMessage.origin, and ModelProfile tool-call-id shape metadata with runtime-comparison's block-streaming, SystemMessage patches, ModelProfile behaviour fields, and media ContentBlocks. Kept both sides' additions in message/types.rs, model/types.rs (ModelProfile.tool_call_id_pattern/max_tool_call_id_len alongside ProviderCompat), and provider adapters, where origin stamping now runs alongside request-hook observation and block folding. Co-authored-by: Medulla <medulla@tinyhumans.ai>
|
Warning Review limit reached
This review includes 27 billable files and costs up to $6.75. Or wait 51 minutes for your next included review. View limit detailsLimit details: You’ve used the included review currently available. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (27)
Comment |
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Introduce a new `System` variant to the `Message` enum, enabling the representation of system-level instructions in conversation contexts. This change allows the library to handle system prompts that guide model behavior, which is a common requirement in chat-based inference workflows. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Extend the model type enum to include quantized variants, enabling the inference engine to load and run models with reduced precision for improved performance and lower memory usage. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the tokenizer vocabulary is empty, the model previously panicked during inference due to an unwrap on a missing token ID. This change adds a check for an empty vocabulary and returns a clear error instead, improving robustness for edge cases where the tokenizer is not properly initialized. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
senamakel
marked this pull request as ready for review
September 20, 2026 07:23
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Tiny Sweeper review
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 001f73581d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+166
to
+171
| /// Accepts video input. | ||
| pub video_in: bool, | ||
| /// Produces video output. | ||
| pub video_out: bool, | ||
| /// Accepts document input (PDF and similar). | ||
| pub document_in: bool, |
There was a problem hiding this comment.
Persisted ModelProfile JSON produced before this commit contains a modalities object without video_in, video_out, or document_in; because these new fields lack #[serde(default)], deserializing that previously valid data now fails with a missing-field error. Add defaults to the new fields or the Modalities struct so existing profiles remain readable, and cover the legacy shape in a serialization test.
AGENTS.md reference: AGENTS.md:L59-L61
Useful? React with 👍 / 👎.
Comment on lines
1521
to
1523
| ) -> Result<reqwest::Response> { | ||
| crate::network_guard::ensure_network_models_allowed()?; | ||
| let url = format!("{}/chat/completions", self.base_url); |
There was a problem hiding this comment.
When with_responses_api_primary() is enabled, invoke and stream route through send_responses(), which posts with self.client and never reaches this guard in post_json(). Consequently, deny_network_models() still permits a real /responses request, defeating the documented hard guarantee for offline tests and evaluation runs; enforce the guard in the Responses path or a common send path.
AGENTS.md reference: AGENTS.md:L56-L59
Useful? React with 👍 / 👎.
Comment on lines
+715
to
+719
| response.message.origin = Some(crate::message::MessageOrigin { | ||
| provider: state.provider.clone(), | ||
| api: CHAT_COMPLETIONS_API.to_string(), | ||
| model: state.model.clone(), | ||
| }); |
There was a problem hiding this comment.
For an SSE request carrying ModelRequest::model, OpenAiModel::stream initializes SseState::model from self.model, so this terminal assignment records the adapter default rather than the model actually requested. Unary and non-SSE paths correctly use stamp_origin(request, ...); the SSE path should likewise pass the request override, otherwise model-switch detection can incorrectly skip or trigger handoff normalization.
Useful? React with 👍 / 👎.
Comment on lines
+314
to
+317
| MediaRef::Path { path, .. } => json!({ | ||
| "type": "text", | ||
| "text": format!("[document attachment omitted: local path {path} was not resolved]"), | ||
| }), |
There was a problem hiding this comment.
When a caller supplies ContentBlock::Document(MediaRef::Path { ... }), this branch embeds the raw filesystem path into a text block sent to Anthropic, contradicting MediaRef::Path's guarantee that providers never see raw paths and potentially disclosing local directory or user information. The audio/video placeholder helper has the same issue; unresolved paths should fail validation or use a path-free omission marker.
Useful? React with 👍 / 👎.
Comment on lines
+1014
to
+1018
| /// Terminal deferral: the provider accepted the request but will finish | ||
| /// it asynchronously (for example an OpenAI batch or background | ||
| /// response). The caller polls or otherwise resolves the response later | ||
| /// via [`ChatModel::fetch_deferred`]. | ||
| Deferred(DeferredHandle), |
There was a problem hiding this comment.
Although this variant is documented as terminal, StreamAccumulator::is_terminal() still returns false after receiving it, and ModelStream::poll_next() does not disarm AbortOnDrop for it. A consumer that stops based on is_terminal() can wait unnecessarily, while dropping after a deferred result leaves the stream treated as unfinished; include Deferred in all terminal-item checks or represent deferral outside the terminal stream contract.
AGENTS.md reference: AGENTS.md:L46-L48
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Vendor-side changes for tinyhumansai/tinyagents#171 (runtime comparison, Phases 2–7). Additive, serde-default everywhere; existing constructors unchanged.
ModelStreamItem::{BlockStart, BlockDelta, BlockEnd}withBlockKind/BlockDelta,ToolDelta.content_index,ProviderError.partial_message/stop_reason; Anthropiccontent_block_*mapped 1:1; OpenAI chat-completions SSE derives blocks (reasoning/text/tool calls).SystemMessage{sections, tools_added, tools_removed}+replay_system_state;Message::Custom(CustomMessage)(filtered out of every request path);AssistantMessage.origin: MessageOriginstamped by all adapters;ContentBlock::{Audio, Video, Document}(MediaRef).ModelProfile.{schema_transform, default_structured_mode, prompted_output_template, thinking_tags, ignore_streamed_leading_whitespace, thinking_level_map, compat, mid_conversation_system_messages, tool_call_id_pattern, max_tool_call_id_len},Modalitiesvideo/document flags.ModelStreamItem::Deferred(DeferredHandle),ChatModel::fetch_deferred(default unsupported),ProviderRequestOptions{on_payload, on_response, http}on OpenAI chat + Anthropic,deny_network_models()process guard,Error::Unsupported.Test plan
cargo fmt --check,cargo clippy --workspace --all-targets --all-features -- -D warnings,cargo test --workspace --all-features— all green inside this repo and as the vendored path dep of tinyagents#171.Co-authored-by: Medulla medulla@tinyhumans.ai