feat(harness): single-owner tool dialects via tinytools-agent #164
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,316 @@ | ||
| //! Host-side selection and application of the tool dialect for one run. | ||
| //! | ||
| //! The protocol itself — how a call is rendered, parsed, repaired, and | ||
| //! scrubbed from a stream — is owned by `tinytools-agent`. What is decided | ||
| //! *here* is the host's part: which dialect a run speaks | ||
| //! ([`RunPolicy::tool_dialect`]), the rewrite of a request onto a text | ||
| //! protocol when one is forced, the minting of call ids for calls recovered | ||
| //! from text, and the fallback that reads a native model's narrated call out | ||
| //! of its visible text. | ||
| //! | ||
| //! [`RunPolicy::tool_dialect`]: crate::runtime::RunPolicy::tool_dialect | ||
|
|
||
| use std::sync::Arc; | ||
|
|
||
| use tinyinference_llm::message::ContentBlock; | ||
| use tinyinference_llm::model::{ModelRequest, ModelResponse, ToolChoice}; | ||
| use tinyinference_llm::tool::{ToolCall, ToolSchema}; | ||
| use tinytools_agent::dialect::PFormatDialect; | ||
| use tinytools_agent::types::{ParseOptions, ParsedToolCall}; | ||
| use tinytools_agent::{PFormatRegistry, StreamScrubber}; | ||
|
|
||
| use crate::config::ToolDispatcher; | ||
| use crate::ids::CallId; | ||
|
|
||
| /// The dialect a run speaks, resolved once from policy. | ||
| #[derive(Debug, Clone)] | ||
| pub(super) enum RunDialect { | ||
| /// Schemas on the wire; the provider adapter owns any text fallback. | ||
| Native, | ||
| /// JSON-in-tag, rendered into the system prompt by the host. | ||
| Xml, | ||
| /// Positional P-Format, rendered into the system prompt by the host. | ||
| PFormat(Arc<PFormatRegistry>), | ||
| } | ||
|
|
||
| impl RunDialect { | ||
| /// Resolves the policy against the tools this run offers. | ||
| pub(super) fn resolve(dispatcher: ToolDispatcher, tools: &[ToolSchema]) -> Self { | ||
| match dispatcher { | ||
| ToolDispatcher::Auto | ToolDispatcher::Native => Self::Native, | ||
|
senamakel marked this conversation as resolved.
There was a problem hiding this comment.
Honor Auto's documented text fallback
[RULE] dispatcher-fallback · There was a problem hiding this comment.
Gate native capability after adding synthetic tools The native dialect is selected without accounting for per-turn synthetic tools, such as the structured-output fallback tool. If those tools are added after model resolution, a model lacking native tool support can still receive them on the wire and enter the native path. Recompute or enforce the required native capability after the final tool set is assembled, before dispatch. [RULE] capability-gating · |
||
| ToolDispatcher::Xml => Self::Xml, | ||
| ToolDispatcher::Pformat => Self::PFormat(Arc::new(tinytools_agent::build_registry( | ||
| tools | ||
| .iter() | ||
| .map(|schema| (schema.name.clone(), schema.parameters.clone())), | ||
| ))), | ||
| } | ||
| } | ||
|
|
||
| /// Whether the host renders the protocol and parses the answer itself. | ||
| pub(super) fn is_text(&self) -> bool { | ||
| !matches!(self, Self::Native) | ||
| } | ||
|
|
||
| /// The P-Format registry for one call, extended with any tool in `tools` | ||
| /// beyond the run-level set the registry was built from. | ||
| /// | ||
| /// The run-level registry is built once from the schemas offered at the | ||
| /// start of the run (see [`Self::resolve`]); a synthetic per-turn tool — | ||
| /// the structured-output fallback schema pushed onto `request.tools` | ||
| /// after that — is advertised in the P-Format catalogue (rendered fresh | ||
| /// from the final tool list on every call) but would otherwise have no | ||
| /// positional layout to decode a call against. Extending here, rather | ||
| /// than rebuilding from scratch every call, keeps the common case (no | ||
| /// new tool this turn) a cheap `Arc::clone`. | ||
| pub(super) fn registry_for(&self, tools: &[ToolSchema]) -> Option<Arc<PFormatRegistry>> { | ||
| match self { | ||
| Self::PFormat(registry) => { | ||
| let extra: Vec<&ToolSchema> = tools | ||
| .iter() | ||
| .filter(|schema| !registry.contains_key(&schema.name)) | ||
| .collect(); | ||
|
senamakel marked this conversation as resolved.
|
||
| if extra.is_empty() { | ||
| return Some(Arc::clone(registry)); | ||
| } | ||
| let mut merged = (**registry).clone(); | ||
| merged.extend(tinytools_agent::build_registry( | ||
| extra | ||
| .into_iter() | ||
| .map(|schema| (schema.name.clone(), schema.parameters.clone())), | ||
| )); | ||
| Some(Arc::new(merged)) | ||
| } | ||
| _ => None, | ||
| } | ||
| } | ||
|
|
||
| /// Rewrites `request` onto this dialect's text protocol: the transcript | ||
| /// is folded into forms a prompt-guided model can read, the protocol block | ||
| /// and catalogue go into the system prompt, and no schema goes on the | ||
| /// wire. A no-op for [`Self::Native`] or when no tools are offered. | ||
| pub(super) fn apply_to_request(&self, request: &mut ModelRequest) { | ||
| if !self.is_text() || request.tools.is_empty() || request.tool_choice == ToolChoice::None { | ||
| return; | ||
| } | ||
| use tinyinference_llm::prompt_tools; | ||
|
|
||
| let tools = std::mem::take(&mut request.tools); | ||
| let messages = prompt_tools::coalesce_tool_results(&request.messages); | ||
| let messages = prompt_tools::ensure_resolvable_user_turn(&messages); | ||
| request.messages = match self { | ||
| Self::Xml | Self::Native => { | ||
| prompt_tools::with_tool_instructions(&messages, &tools, &request.tool_choice) | ||
| } | ||
| Self::PFormat(_) => { | ||
| let specs: Vec<tinytools_agent::tinytools::ToolSpec> = tools | ||
| .iter() | ||
| .map(|schema| tinytools_agent::tinytools::ToolSpec { | ||
| name: schema.name.clone(), | ||
| description: schema.description.clone(), | ||
| parameters: schema.parameters.clone(), | ||
| }) | ||
| .collect(); | ||
| let mut block = PFormatDialect::instructions(); | ||
| block.push_str(&tinytools_agent::render::render_pformat_catalogue(&specs)); | ||
| // The XML branch renders `tool_choice` into its instructions | ||
| // via `prompt_tools::tool_instructions`; P-Format has no | ||
| // schema on the wire either (the wire choice is reset to | ||
| // `Auto` below), so a forced choice has to be said in plain | ||
| // English here too or `Required`/`Tool(name)` silently loses | ||
| // its meaning — in particular the sole synthetic | ||
| // structured-output tool would no longer be forced, and a | ||
| // plain-text response would make extraction fail. | ||
| match &request.tool_choice { | ||
| ToolChoice::Required => { | ||
| block.push_str("\nYou must emit at least one tool call.\n"); | ||
| } | ||
| ToolChoice::Tool(name) => { | ||
| block.push_str(&format!("\nYou must call the `{name}` tool.\n")); | ||
| } | ||
| ToolChoice::Auto | ToolChoice::None => {} | ||
| } | ||
| prompt_tools::append_system_block(&messages, &block) | ||
|
senamakel marked this conversation as resolved.
|
||
| } | ||
| }; | ||
| request.tool_choice = ToolChoice::Auto; | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| } | ||
| } | ||
|
|
||
| /// What a model call needs in order to recover text-dialect calls: the | ||
| /// tools that were offered (which a text-dialect request no longer carries | ||
| /// on the wire) and the P-Format registry, when there is one. | ||
| #[derive(Debug, Clone, Default)] | ||
| pub(super) struct TextRecovery { | ||
| /// The tools offered this turn, before any dialect rewrite. | ||
| pub(super) offered: Arc<Vec<ToolSchema>>, | ||
| /// The P-Format layouts, for [`RunDialect::PFormat`]. | ||
| pub(super) registry: Option<Arc<PFormatRegistry>>, | ||
| } | ||
|
|
||
| impl TextRecovery { | ||
| /// A scrubber for one streamed model call, or `None` when no tools were | ||
| /// offered and there is nothing to recover. | ||
| pub(super) fn scrubber(&self, model_call_id: &CallId) -> Option<DeltaScrubber> { | ||
| (!self.offered.is_empty()).then(|| { | ||
| DeltaScrubber::new(model_call_id.clone(), &self.offered, self.registry.clone()) | ||
| }) | ||
| } | ||
| } | ||
|
|
||
| /// How one model call is made: streamed or unary, and what it needs to | ||
| /// recover text-dialect calls from the answer. | ||
| #[derive(Debug, Clone, Default)] | ||
| pub(super) struct CallShape { | ||
| /// Whether the provider's streaming path is used. | ||
| pub(super) streaming: bool, | ||
| /// Offered tools and P-Format registry for text recovery. | ||
| pub(super) recovery: TextRecovery, | ||
| } | ||
|
|
||
| /// Converts a recovered call into the harness's [`ToolCall`], minting an id | ||
| /// scoped to the model call it came from. | ||
| /// | ||
| /// `{model_call_id}-tool-{n}` is unique per run by construction — model call | ||
| /// ids already are — and visibly distinct from any provider's, so a | ||
| /// recovered call can never be confused with a native one in a transcript. | ||
| fn to_tool_call(call: ParsedToolCall, model_call_id: &CallId, slot: usize) -> ToolCall { | ||
| // `call.id` is intentionally never used, even when a grammar or a future | ||
| // change to `tinytools-agent` happens to populate one: this function's | ||
| // whole contract (see its doc comment) is that a text-recovered call's id | ||
| // is always host-minted and unique per run, so it can never collide with | ||
| // another recovered call or be confused with a native provider one. A | ||
| // parser-supplied id would be model-controlled input; trusting it here | ||
| // would let two calls collide on an id the model chose, or let a | ||
| // narrated call impersonate a specific native one. | ||
| let id = format!("{model_call_id}-tool-{slot}"); | ||
| ToolCall::new(id, call.name, call.arguments) | ||
| } | ||
|
|
||
| /// Reads text-dialect calls out of a response that carries no structured | ||
| /// ones, through every grammar `tinytools-agent` knows, with the offered | ||
| /// tools enabling name repair. Non-text content blocks (reasoning) survive. | ||
| pub(super) fn recover_text_calls( | ||
| response: &mut ModelResponse, | ||
| model_call_id: &CallId, | ||
| offered: &[ToolSchema], | ||
| registry: Option<&PFormatRegistry>, | ||
| ) { | ||
| if offered.is_empty() { | ||
| return; | ||
| } | ||
| let known: Vec<String> = offered.iter().map(|tool| tool.name.clone()).collect(); | ||
| let mut options = ParseOptions::new().with_known_tools(&known); | ||
| if let Some(registry) = registry { | ||
| options = options.with_registry(registry); | ||
| } | ||
| let text = response.text(); | ||
| let outcome = tinytools_agent::parse_text(&text, &options); | ||
| if outcome.calls.is_empty() { | ||
| return; | ||
| } | ||
| for diagnostic in &outcome.diagnostics { | ||
| tinyagents_tracing::debug!(?diagnostic, "[agent_loop] text-dialect recovery"); | ||
| } | ||
| // Appended, not assigned: a provider can legitimately return one native | ||
| // structured call *and* narrate a second one as text in the same | ||
| // response (this is deliberately parsed even when `tool_calls` was | ||
| // already non-empty — see above), and overwriting the collection here | ||
| // used to silently drop whichever set ran second. | ||
| let recovered = outcome | ||
| .calls | ||
| .into_iter() | ||
| .enumerate() | ||
| .map(|(index, call)| to_tool_call(call, model_call_id, index + 1)); | ||
| response.message.tool_calls.extend(recovered); | ||
| response.message.content = | ||
| replace_text_blocks(std::mem::take(&mut response.message.content), outcome.text); | ||
| } | ||
|
|
||
| /// Keeps every non-text block in place and substitutes one cleaned text at | ||
| /// the first text block's position; an empty `cleaned` emits no text block. | ||
| fn replace_text_blocks(content: Vec<ContentBlock>, cleaned: String) -> Vec<ContentBlock> { | ||
| let mut out = Vec::with_capacity(content.len()); | ||
| let mut inserted = false; | ||
| for block in content { | ||
| match block { | ||
| ContentBlock::Text(_) => { | ||
| if !inserted { | ||
| if !cleaned.is_empty() { | ||
| out.push(ContentBlock::Text(cleaned.clone())); | ||
| } | ||
| inserted = true; | ||
| } | ||
| } | ||
| other => out.push(other), | ||
| } | ||
| } | ||
| if !inserted && !cleaned.is_empty() { | ||
| out.push(ContentBlock::Text(cleaned)); | ||
| } | ||
| out | ||
| } | ||
|
|
||
| /// Scrubs tool-call markup from streamed visible text and collects the | ||
| /// calls it completes, minting harness ids for them. | ||
| /// | ||
| /// Consumers of [`AgentEvent::ModelDelta`](crate::events::AgentEvent::ModelDelta) | ||
| /// never see a partial `<tool_call>`; the calls surface on the terminal | ||
| /// response instead, exactly once. | ||
| pub(super) struct DeltaScrubber { | ||
| inner: StreamScrubber, | ||
| model_call_id: CallId, | ||
| calls: Vec<ToolCall>, | ||
| } | ||
|
|
||
| impl DeltaScrubber { | ||
| /// A scrubber for one model call, knowing the tools it offered. | ||
| pub(super) fn new( | ||
| model_call_id: CallId, | ||
| offered: &[ToolSchema], | ||
| registry: Option<Arc<PFormatRegistry>>, | ||
| ) -> Self { | ||
| let known = offered.iter().map(|tool| tool.name.clone()).collect(); | ||
|
There was a problem hiding this comment.
Pass the declared tool schemas to the stream scrubber The scrubber is given only the names of the offered tools, so streamed calls are parsed and recovered without the declared parameter schemas. A streamed model can therefore supply arguments that do not conform to the tool contract, while the equivalent terminal recovery has access to [RULE] schema-validation · |
||
| let mut inner = StreamScrubber::new().with_known_tools(known); | ||
|
There was a problem hiding this comment.
Pass declared tool schemas to the stream scrubber The recovery path receives the full [RULE] schema-preservation · |
||
| if let Some(registry) = registry { | ||
| inner = inner.with_registry(registry); | ||
| } | ||
| Self { | ||
| inner, | ||
| model_call_id, | ||
| calls: Vec::new(), | ||
| } | ||
| } | ||
|
|
||
| /// Feeds one text delta; returns the text safe to forward. | ||
| pub(super) fn feed(&mut self, text: &str) -> String { | ||
| let step = self.inner.feed(text); | ||
| self.collect(step.calls); | ||
| step.text | ||
| } | ||
|
|
||
| /// Drains the remainder at end of stream. | ||
| pub(super) fn flush(&mut self) -> String { | ||
| let step = self.inner.flush(); | ||
| self.collect(step.calls); | ||
| step.text | ||
| } | ||
|
|
||
| fn collect(&mut self, calls: Vec<ParsedToolCall>) { | ||
| for call in calls { | ||
| let slot = self.calls.len() + 1; | ||
| self.calls | ||
| .push(to_tool_call(call, &self.model_call_id, slot)); | ||
| } | ||
| } | ||
|
|
||
| /// The calls completed during the stream, in order. | ||
| pub(super) fn into_calls(self) -> Vec<ToolCall> { | ||
| self.calls | ||
| } | ||
|
|
||
| /// Whether any call was completed during the stream. | ||
| pub(super) fn has_calls(&self) -> bool { | ||
| !self.calls.is_empty() | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.