113 kubernetes integration #130
Conversation
- Introduced a new backend option for Kubernetes in the Backends enum. - Updated segmentation algorithms to accept optional keyword arguments for Kubernetes backend, enhancing flexibility for users.
…thm execution - Added a new section outlining how to configure and use the Kubernetes backend for running algorithms. - Provided examples for setting the KUBECONFIG environment variable and specifying backend options in the inference method.
- Integrated Kubernetes backend option into the BraTSAlgorithm class. - Updated methods to accept optional keyword arguments for Kubernetes, allowing for enhanced configuration during inference. - Added error handling to ensure Kubernetes kwargs are only used with the Kubernetes backend.
- Updated pyproject.toml to include the Kubernetes package with a minimum version of 34.1.0, enabling support for Kubernetes features in the project.
…Kubernetes configuration - Changed the parameter name from `mount_path` to `data_mount_path` in the documentation to accurately describe its function in the `infer_single` method for Kubernetes backend usage.
- Implemented functions for creating and managing Kubernetes jobs, including PVC creation, job execution, and output handling. - Added methods for downloading additional files from Zenodo and checking file presence in pods. - Enhanced logging for better traceability during job execution and output verification. - Integrated command execution within pods to facilitate file uploads and downloads, ensuring smooth operation of the Kubernetes backend for algorithm inference.
…_from_pod function in kubernetes.py
…rnetes.py`. Tests cover command argument building, file handling, job creation, and PVC management for different algorithm configurations.
…`local_base_dir` parameter
…hanced Kubernetes backend configuration
SimoneBendazzoli93
requested review from
Copilot
and removed request for
Copilot
|
🤖 Code Formatting Reminder Hello there! 👋 It looks like the code in this pull request might benefit from some formatting improvements. |
|
/format |
|
🤖 I will now format your code with black. Check the status here. |
There was a problem hiding this comment.
Pull Request Overview
This PR adds Kubernetes backend support to the BraTS orchestrator, enabling remote algorithm execution via Kubernetes Jobs as an alternative to local Docker/Singularity containers. Key changes include:
- New Kubernetes backend implementation with job orchestration, file transfer, and PVC management
- Integration of Kubernetes backend into the existing algorithm inference pipeline
- Support for configurable Kubernetes resources via
kubernetes_kwargsparameter
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 14 comments.
Show a summary per file
| File | Description |
|---|---|
| brats/constants.py | Added KUBERNETES enum value to Backends |
| brats/core/kubernetes.py | New module implementing Kubernetes job execution with PVC management, file transfers, and pod lifecycle management |
| brats/core/brats_algorithm.py | Updated _infer_single and _infer_batch to support kubernetes_kwargs parameter and dispatch to Kubernetes backend |
| brats/core/segmentation_algorithms.py | Added kubernetes_kwargs parameter to infer_single and infer_batch methods for both Adult and Pediatric classes |
| tests/core/test_kubernetes.py | Comprehensive test suite for Kubernetes backend functionality |
| README.md | Added documentation for Kubernetes backend usage with configuration examples |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…new packages: `cachetools`, `durationpy`, `google-auth`, `kubernetes`, `oauthlib`, `pyasn1`, `pyasn1-modules`, `requests-oauthlib`, `rsa`, and `websocket-client`. Adjust version constraints and add optional dependencies for improved functionality.
…Lesion/BraTS into 113-kubernetes-integration
… path handling based on algorithm year, and streamline command execution logging.
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 7 out of 8 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
… and modify type hints in `kubernetes.py` for clarity. Enhance logging in `_download_folder_from_pod` and add TODO comments for future security context implementation. Remove commented-out code in `run_job` and adjust test cases in `test_kubernetes.py` to reflect changes.
|
🤖 Code Formatting Reminder Hello there! 👋 It looks like the code in this pull request might benefit from some formatting improvements. |
|
/format |
|
🤖 Code Formatting Reminder Hello there! 👋 It looks like the code in this pull request might benefit from some formatting improvements. |
|
🤖 I will now format your code with black. Check the status here. |
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 7 out of 8 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…t_path, enhancing flexibility. Improve documentation for argument types and clarify logging messages related to job pod completion.
Updated the condition for validating Kubernetes kwargs in the BraTSAlgorithm class to ensure they are only used with the Kubernetes backend. Enhanced the logging message for successful file uploads to indicate multiple files are uploaded. Additionally, refactored variable names in the volume mount loop for better clarity.
|
🤖 Code Formatting Reminder Hello there! 👋 It looks like the code in this pull request might benefit from some formatting improvements. |
|
/format |
|
🤖 I will now format your code with black. Check the status here. |
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 7 out of 8 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…un_job Updated the condition for validating Kubernetes kwargs in the BraTSAlgorithm class to use '!=' for clarity. Modified the run_job function to ensure output_path and data_path are wrapped in Path objects for consistent path handling.
…Lesion/BraTS into 113-kubernetes-integration
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 7 out of 8 changed files in this pull request and generated 11 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…ity function Updated the return type of _create_finalizer_job and _create_namespaced_job functions to return job names for better usability. Introduced a new utility function _check_pod_terminal_or_running to streamline pod status checks, enhancing code readability and maintainability.
…tional checks Updated the _build_command_args function to ensure additional file paths are only appended if they exist. Enhanced the documentation for _observe_job_output and _create_namespaced_job functions to clarify return values and error handling, improving overall code clarity and usability.
|
🤖 Code Formatting Reminder Hello there! 👋 It looks like the code in this pull request might benefit from some formatting improvements. |
|
/format |
|
🤖 I will now format your code with black. Check the status here. |
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 7 out of 8 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
fixes #113