fix(google-genai): localize JSON wrapping of tool responses to Gemini…#1468
Draft
alexheifetz wants to merge 1 commit intomainfrom
Draft
fix(google-genai): localize JSON wrapping of tool responses to Gemini…#1468alexheifetz wants to merge 1 commit intomainfrom
alexheifetz wants to merge 1 commit intomainfrom
Conversation
… provider Google GenAI requires tool responses (FunctionResponse.response) to be valid JSON objects. Plain text causes JsonParseException or silent data loss resulting in hallucinations (#1391). Introduce ToolResponseContentAdapter strategy interface — following the same per-provider pattern as OptionsConverter — so each provider can adapt tool response content at the Spring AI boundary: - PASSTHROUGH (default): OpenAI, Anthropic, Bedrock — unchanged - JsonWrappingToolResponseContentAdapter: Gemini — wraps non-JSON content as {"result": "..."}, passes JSON through Threaded through SpringAiLlmService → SpringAiLlmMessageSender → toSpringAiMessage(), wired in GoogleGenAiModelsConfig. Fixes #1391
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a provider-specific strategy for adapting tool response content before sending it to LLMs, addressing format requirements such as mandatory JSON for Google GenAI. The main change is the addition of a pluggable
ToolResponseContentAdapterinterface, allowing each LLM provider to specify how tool responses are formatted (e.g., plain text passthrough for OpenAI, JSON wrapping for Google GenAI). This refactoring keeps provider-specific logic out of the shared message conversion infrastructure and is thoroughly tested.Provider-specific tool response adaptation
ToolResponseContentAdapterinterface and a defaultPASSTHROUGHimplementation for providers that accept plain text, as well as aJsonWrappingToolResponseContentAdapterfor providers like Google GenAI that require tool responses to be valid JSON objects.SpringAiLlmServiceandSpringAiLlmMessageSenderto accept and use a configurabletoolResponseContentAdapter, defaulting toPASSTHROUGH. This adapter is now used during message conversion for tool responses. [1] [2] [3] [4] [5] [6] [7]Google GenAI integration
JsonWrappingToolResponseContentAdapter, ensuring all tool responses sent to Gemini are valid JSON and avoiding data loss or errors. [1] [2] [3]Testing and validation
PASSTHROUGHandJsonWrappingToolResponseContentAdapter) covering plain text, JSON, whitespace, and edge cases.These changes ensure that tool responses are correctly formatted for each provider, improving reliability and maintainability of LLM integrations.