Implement LLM-based entity extractor#118
Conversation
…ADME instructions for running with Docker
…rocessing capabilities
|
the major issues i can see even tho i have just glanced over the pr like i haven't had an in depth look it is priortising the i'll review the code when i get free from work |
|
also compose file shd have a network and shs host all the 4 things
|
| elif "base_url_override" in config: | ||
| final_base_url = config["base_url_override"] | ||
| elif config.get("base_url_env"): | ||
| final_base_url = os.getenv(config["base_url_env"]) |
There was a problem hiding this comment.
make the base url configurale for all
- allow it to be setup via ui - priority 1
- allow then by get settings - pydantic settings
- then .env
| @@ -204,12 +211,18 @@ def summarize_text(self, text: str) -> str: | |||
| "openrouter": "openrouter_api_key", | |||
| } | |||
|
|
|||
There was a problem hiding this comment.
modify the config structure for cloud llms also have an option for these base url especially for openai
beacuse most llm apis areopenai compliant
|
|
||
| logger = get_logger(__name__) | ||
|
|
||
| _EMBEDDINGS = GoogleGenerativeAIEmbeddings( |
There was a problem hiding this comment.
make this also configurable in the ui then .env using pydantic settings please
| def __init__(self) -> None: | ||
| self._llm = llm | ||
|
|
||
| def _get_embeddings(self) -> Any: |
There was a problem hiding this comment.
not the best implemnentation
see llm providers configration and implement that
|
|
||
| # ── Google GenAI path (cloud File API) ────────────────────────────────────── | ||
|
|
||
| async def _handle_file_google( |
There was a problem hiding this comment.
this is the expensive. way to handle this use
base64 incoded images send over apis please
for this we would require google file storage
y complicate it this much just use
base64 images for image / multimodal support
use audio imputs the same way only
| │ ollama │ image │ base64 HumanMessage (vision model) │ | ||
| │ ollama │ text/pdf │ Extract text → inject into prompt │ | ||
| │ other (oai…) │ image │ base64 HumanMessage (LangChain vision) │ | ||
| │ other (oai…) │ text/pdf │ Extract text → inject into prompt │ |
There was a problem hiding this comment.
use pymupdf4llm or docling to analyse and convert the docuement to markdown then send it in the case for ollama we do not require cloud dependecy for ollama as we need keep shit local offline accessible
| networks: | ||
| agentic_net: | ||
| driver: bridge | ||
| driver: bridge No newline at end of file |
There was a problem hiding this comment.
why do we need docker if we do package everything frontend and all also so that 1 command starts everythinf
| base_url_param_name = config["param_map"].get("base_url", "base_url") | ||
| params[base_url_param_name] = final_base_url | ||
| elif config.get("base_url_env") and not final_base_url: | ||
| # For providers that *need* a base URL, raise; but for ollama we |
There was a problem hiding this comment.
we shd suppoert vllm also not just ollama
| # have a sensible default in Settings so we should never reach here. | ||
| raise ValueError( | ||
| f"Base URL for '{self.provider}' not found. " | ||
| f"Please provide it directly or set the '{config['base_url_env']}' environment variable." |
There was a problem hiding this comment.
support mlx model as well rn this only supports llama cpp models which are very slow on macs we need mlx for macos local model testing
2 participants
This pull request introduces comprehensive improvements to Docker support, multi-provider LLM configuration, and file handling in the backend. It adds a production-ready
Dockerfile,.dockerignore, and detailed Docker usage instructions. The LLM configuration is now more flexible, supporting both cloud and self-hosted providers (like Ollama) with improved secret/base URL resolution. File handling in the agent service is refactored to support images, PDFs, and text/code files across Google, Ollama, and other providers, with robust routing and error handling. Embedding extraction is now provider-aware, supporting both Google and Ollama embeddings.Dockerization and Deployment:
Dockerfilefor the Python backend, supporting reproducible builds, layer caching, and health checks. A.dockerignorefile is also included to optimize build context. [1] [2]README.mdwith clear, step-by-step Docker usage instructions, including environment setup, container management, and extension integration.LLM Provider Configuration and Flexibility:
core/llm.pyto support dynamic resolution of API keys and base URLs for both cloud (e.g., Google, OpenAI) and self-hosted (e.g., Ollama) providers, using secrets service and environment variables. This allows seamless switching and configuration of LLM providers. [1] [2] [3] [4].invoke()method, ensuring compatibility with LangChain v0.2+ and future-proofing provider integrations.File and Embedding Handling Enhancements:
services/react_agent_service.pyto route file processing based on provider and file type (image, PDF, text/code), supporting Google File API uploads, Ollama vision models, and text extraction/injection for other providers. Includes robust error handling and logging.memory/ingestion/extractor.py, dynamically selecting Google or Ollama embeddings based on the active LLM provider. [1] [2]These changes significantly improve the backend's flexibility, maintainability, and deployment readiness for both cloud and self-hosted environments.