Tool dialect unify - #11
Merged
Merged
Conversation
When the repair JSON response is missing optional fields like `explanation` or `confidence`, the parser now returns default values instead of failing. This makes the agent more resilient to variations in LLM output format. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When parsing bare JSON objects, the grammar previously failed to recognize top-level JSON values without surrounding braces. This change adds support for parsing standalone JSON objects by adjusting the grammar rules to accept JSON values at the top level, enabling the parser to handle bare JSON inputs correctly. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The bare JSON parser now skips leading whitespace before attempting to parse a JSON value, ensuring that inputs with indentation or spacing are correctly recognized as valid bare JSON rather than being rejected. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the agent output contains only a JSON object with no surrounding text, the parser now correctly extracts it instead of failing. Previously, the parser required non-JSON text around the JSON block, which caused valid bare JSON responses to be rejected. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When parsing invoke XML, the grammar now correctly handles cases where optional attributes are absent, preventing parse failures that occurred when attributes like `target` or `timeout` were omitted from the invocation element. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the `name` attribute is absent from an invoke XML element, the parser now returns a clear error instead of panicking. This improves robustness against malformed input. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When parsing invoke XML, the grammar now correctly handles cases where optional attributes are absent, preventing parse failures that occurred when expected attributes were not present in the input. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Fix the test assertion to properly verify that the stream terminates after processing all items, rather than checking for an incorrect condition that would never be met. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The sentinel grammar parser now correctly processes a previously unhandled edge case where certain input patterns caused unexpected termination. This fix ensures robust parsing behavior in the sentinel module. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Fix a parsing issue in the sentinel grammar where certain edge cases caused incorrect token recognition. The change ensures that sentinel boundaries are properly respected during parsing, preventing false positives in token classification. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The test for stream termination was using an incorrect assertion that would never trigger, causing the test to pass even when the stream failed to terminate properly. Updated the assertion to correctly verify that the stream ends after the expected number of items. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reorder the fields in the struct initialization to match the definition order, preventing potential compilation errors or mismatches in field assignment. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When a tool response is received without a `tool_call_id` field, the agent now gracefully handles the missing value instead of panicking. This change improves robustness when interacting with models that may omit the identifier in certain edge cases. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a test module for parsing bare JSON input in the tinytools-agent crate, covering the case where input is a raw JSON value without any tool call structure. This ensures the parser correctly handles standalone JSON payloads. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When a tagged expression lacks a closing delimiter, the parser now returns an appropriate error instead of panicking. This improves robustness when processing malformed input. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When a tagged expression lacks a closing delimiter, the parser now returns a clear error instead of panicking. This improves robustness for malformed input in the tagged grammar parser. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The tagged parser now returns an error when a closing tag is missing, rather than silently consuming input. This prevents incorrect parsing results and ensures malformed tagged content is properly reported to the caller. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When repairing malformed JSON, the agent now gracefully handles cases where expected fields are absent, preventing panics or incomplete repairs. This improves robustness when processing partial or corrupted data. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Changed the test assertion in the JSON repair test to expect an empty array instead of a null value when no repairs are needed, aligning the test with the actual serialization behaviour of the system. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The grammar parser now correctly accepts harmony statements that omit trailing semicolons, which are optional in the specification. Previously, the parser would reject valid input that followed the relaxed syntax rules. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a fallback to return an empty grammar when the harmony grammar file is not found, preventing a panic during parsing. This ensures the parser can continue operating even when the optional harmony grammar resource is absent. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The harmony grammar rule previously rejected a trailing comma after the last element in a sequence, which caused valid inputs to fail parsing. The change now allows an optional comma at the end, aligning the parser with common formatting conventions and user expectations. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The harmony mistral parser now returns an empty result when given an empty input string, preventing a panic that occurred when trying to parse zero-length content. This ensures the parser behaves consistently with other parsers in the codebase. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When a tool call has no arguments, the parser now correctly returns an empty object instead of failing. This fixes a crash that occurred when the model produced a tool call with an empty arguments field. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When a Mistral-formatted tool call has an empty arguments field, the parser now correctly produces an empty JSON object instead of failing. This fixes a crash that occurred when the model returned a tool call with no arguments. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When a Mistral-style prompt lacks the `[INST]` tag, the parser now falls back to treating the entire input as a single user message instead of failing. This improves robustness for malformed or incomplete prompts. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reformat several expressions that exceeded the project's preferred line length, breaking them across multiple lines to improve readability without changing any behaviour. The affected areas include a return expression in the Harmony grammar parser, a conditional in `absorb_start_prefix`, a test assertion in `bare_json.rs`, a template marker search in the JSON repair module, and two test assertions in `stream/test.rs`. One line in `harmony_mistral.rs` is shortened to fit within the line length limit. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The `std::fmt` import was no longer needed after a previous refactor removed the code that relied on it, so it has been cleaned up to avoid compiler warnings. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
|
Warning Review limit reached
This review includes 13 billable files and costs up to $3.25. Or wait 37 minutes for your next included review. View limit detailsLimit details: You’ve used the included review currently available. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (13)
Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d2d0a89e2c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+216
to
+219
| if !in_string | ||
| && let Some(marker) = TEMPLATE_MARKERS | ||
| .iter() | ||
| .find(|marker| rest.starts_with(*marker)) |
There was a problem hiding this comment.
When an object combines the already-supported mismatched-key quoting with a leaked trailing marker—for example {"name":"get_weather","parameters':{'city':"Paris"}}<tool_call|>—this pre-repair quote tracker becomes desynchronized and treats the marker as string data, so it is not removed. The later key-quoting repair cannot parse the trailing marker and recover_object returns None, causing the call's arguments to be dropped or replaced with {}; previously the unconditional marker removal allowed this cumulative repair to succeed. Preserve a fallback marker-stripping pass for candidates that remain invalid after quote-aware stripping.
Useful? React with 👍 / 👎.
Tiny Sweeper review
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Related issue
API or behavior changes
Validation
Commands actually run, with their outcome:
cargo fmt --all -- --checkcargo clippy --all-targets --all-features -- -D warningscargo build --all-targets --all-featurescargo test --all-featuresTests
Documentation
Checklist
#[allow(...)],#[ignore], or relaxed lints.envcontents in the diff or the description