feat: Add Ollama environment variable support and fix compatibility - #12
Open
CodeNKasch wants to merge 7 commits into
Open
feat: Add Ollama environment variable support and fix compatibility#12CodeNKasch wants to merge 7 commits into
CodeNKasch wants to merge 7 commits into
Conversation
…guration Introduce Ollama integration for local AI model execution. Enhance model selection logic to prioritize Ollama when OLLAMA_MODEL is set. Update documentation to reflect new Ollama setup and configuration options. Add environment variables for Ollama base URL and model. Improve error handling and type clarity in Ollama client. Add comprehensive tests for model selection priority and Ollama URL normalization.
CodeNKasch
marked this pull request as ready for review
October 13, 2025 20:21
This commit renames the OLLAMA_MODEL environment variable to GIA_DEFAULT_MODEL. This change aims to unify the default model configuration across different AI providers, making the environment variable naming more consistent and intuitive. The GIA_DEFAULT_MODEL can now be used to specify either a Gemini model or an Ollama model by prefixing the Ollama model name with ollama::. Documentation and code examples have been updated to reflect this change. The priority of model selection remains the same: -m flag > GIA_DEFAULT_MODEL > default Gemini model.
The created_at field is no longer needed in the ConversationSummary struct and has been removed.
created with Ollama
This commit introduces several improvements to the Ollama integration: Enhanced Message Content Handling: The MessageContentWrapper now correctly identifies and handles ContentPartWrapper::Image and ContentPartWrapper::Audio variants. This ensures that when these media types are present, the message content is not unnecessarily merged into a single text part, which could lead to compatibility issues with Ollama. Streamlined Ollama Client Setup: The OllamaClient setup has been refined for better readability and maintainability. The custom base URL configuration now uses a more concise ServiceTargetResolver closure, making the client initialization cleaner.
This commit refactors how MessageContentWrapper and OllamaClient handle message parts to ensure compatibility with both Gemini and Ollama. Previously, multiple text parts in MessageContentWrapper were merged into a single text for Ollama, which caused issues with Gemini's caching mechanism. This change introduces the following: content_part_wrapper.rs: The to_genai_message_content function in MessageContentWrapper is simplified to always convert parts to their genai::chat::ContentPart equivalents. This ensures that Gemini receives the distinct parts necessary for its internal caching and processing. Tests are added to verify that multiple parts (text, file, clipboard, image) are preserved and correctly represented in the genai::chat::MessageContent. ollama.rs: A new private helper function merge_text_parts_if_needed is added to OllamaClient. This function checks if a ChatMessage contains multiple ContentPart::Text variants and, if so, merges them into a single MessageContent::Text. This addresses the Ollama compatibility issue where multiple text parts are not handled correctly by its OpenAI compatibility layer. This merging logic is applied only to messages being sent to Ollama. Tests are added to verify the merging behavior for single text, multiple text, and single text within a Parts structure. By separating the concerns of data representation for Gemini and compatibility for Ollama, this change improves the robustness and flexibility of the application's AI interaction layer.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR enhances Ollama integration with environment variable support and fixes compatibility issues, enabling seamless use of local LLMs without requiring Gemini API keys.
Key Features
🔧 Environment Variables
GIA_DEFAULT_MODEL: Set your preferred model (Ollama or Gemini)export GIA_DEFAULT_MODEL="ollama::llama3.2"-mflag >GIA_DEFAULT_MODEL>gemini-2.5-flash-liteOLLAMA_BASE_URL: Connect to remote Ollama serversexport OLLAMA_BASE_URL="http://192.168.1.100:11434"http://localhost:11434/v1/endpoint🐛 Ollama Compatibility Fix
Fixed critical issue where Ollama's OpenAI compatibility layer couldn't handle multiple text parts in a single message:
📚 Documentation
Updated
README.mdandCLAUDE.mdwith:🧪 Testing
Usage Examples
Technical Details
OLLAMA_MODELtoGIA_DEFAULT_MODELfor consistencycreated_atfield fromConversationSummarynotify_rustimport/v1/endpoint format