Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 116 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,125 @@
# Python Project Template
# CIAO: Contextual Importance Assessment via Obfuscation

This project template serves as a robust foundation for Python projects, promoting best practices and streamlining development workflows. It comes pre-configured with essential tools and features to enhance the development experience.
An implementation of explainable AI techniques for image classification. CIAO identifies influential image regions by systematically segmenting images, obfuscating segments, and using search algorithms to find important regions (hyperpixels).

## Tools Included
## Overview

- [uv](https://docs.astral.sh/uv/) for efficient dependency management.
- [Ruff](https://docs.astral.sh/ruff) for comprehensive linting and code formatting.
- [Pytest](https://docs.pytest.org) for running tests and ensuring code reliability.
- [GitLab CI/CD](https://docs.gitlab.com/ee/ci) for continuous integration.
- [Pydocstyle](https://www.pydocstyle.org) for validating docstring styles, also following the [Google style](https://google.github.io/styleguide/pyguide.html#s3.8-comments-and-docstrings).
- [Mypy](https://mypy-lang.org) for static type checking.
CIAO explains what regions of an image contribute to a neural network's classification decisions. The method:

1. Segments the image into small regions
2. Obfuscates each segment and measures impact on model predictions
3. Uses search algorithms to group adjacent important segments into hyperpixels
4. Generates explanations showing which regions influenced the prediction

## Usage
## Quick Start

Key commands for effective project management:
### Installation

- `uv sync` - Installs all project dependencies.
- `uv add <package>` - Adds a new dependency to the project.
- `uv run ruff check` - Runs linting.
- `uv run ruff format` - Runs formatting
- `uv run mypy .` - Runs mypy.
- `uv run pytest tests` - Executes tests located in the tests directory.
- `uv run <command>` - Runs the specified command within the virtual environment.
```bash
# Clone the repository
git clone https://github.com/RationAI/ciao.git
cd ciao

## CI/CD
# Install dependencies using uv
uv sync
```

The project uses our [GitLab CI/CD templates](https://gitlab.ics.muni.cz/rationai/digital-pathology/templates/ci-templates) to automate the linting and testing processes. The pipeline is triggered on every merge request and push to the default branch.
### Basic Usage

Explain a single image with default settings:

```bash
uv run ciao
```

Customize the explanation using Hydra configuration overrides:

```bash
uv run ciao data.image_path=./my_image.jpg explanation.method=mcts explanation.segment_size=8
```

Alternatively, run as a module:

```bash
uv run python -m ciao
```

### Development Commands

- `uv sync` - Install all dependencies
- `uv add <package>` - Add a new dependency
- `uv run ruff check` - Run linting
- `uv run ruff format` - Format code
- `uv run mypy .` - Run type checking
- `uv run ciao` - Run CIAO with default configuration
- `uv run pytest tests` - Execute tests

## Method Details

### How CIAO Works

1. **Segmentation**: The input image is divided into small regions (segments) using hexagonal or square grids
2. **Score Calculation**: Each segment is obfuscated (replaced) and the model is queried to measure how much that segment affects the prediction. This gives an importance score to each segment
3. **Hyperpixel Search**: A search algorithm finds groups of adjacent segments with high importance scores, creating "hyperpixels" that represent influential image regions
4. **Explanation**: The top hyperpixels are visualized to show which regions most influenced the model's prediction

### Search Algorithms

- **MCTS (Monte Carlo Tree Search)**: Tree-based search with UCB exploration
- **MC-RAVE**: MCTS with Rapid Action Value Estimation
- **MCGS (Monte Carlo Graph Search)**: Graph-based variant allowing revisiting of states
- **MCGS-RAVE**: MCGS with RAVE enhancements
- **Lookahead**: Greedy search with lookahead using efficient bitset operations
- **Potential**: Potential field-guided sequential search

### Segmentation Methods

- **Hexagonal Grid**: Divides image into hexagonal cells for better spatial coverage
- **Square Grid**: Simple square grid segmentation

### Replacement Methods

- **Mean Color**: Replace masked regions with the image's mean color (normalized)
- **Blur**: Gaussian blur applied to masked regions
- **Interlacing**: Interlaced pattern replacement
- **Solid Color**: Replace with a specified solid color (RGB)

## Proposed project Structure

```
ciao/
├── ciao/ # Main package
│ ├── algorithm/ # Search algorithms and data structures
│ │ ├── mcts.py # Monte Carlo Tree Search
│ │ ├── mcgs.py # Monte Carlo Graph Search
│ │ ├── lookahead_bitset.py # Greedy lookahead with bitsets
│ │ ├── potential.py # Potential-based search
│ │ ├── bitmask_graph.py # Bitset operations for hyperpixels
│ │ ├── nodes.py # Node classes for tree/graph search
│ │ └── search_helpers.py # Shared MCTS/MCGS helper functions
│ ├── data/ # Data loading and preprocessing
│ │ ├── loader.py # Image loaders
│ │ ├── preprocessing.py # Image preprocessing utilities
│ │ └── segmentation.py # Segmentation utilities (hex/square grids)
│ ├── evaluation/ # Scoring and evaluation
│ │ ├── surrogate.py # Surrogate dataset creation and segment scoring
│ │ └── hyperpixel.py # Hyperpixel evaluation and selection
│ ├── explainer/ # Core explainer implementation
│ │ └── ciao_explainer.py # Main CIAO explainer class
│ ├── model/ # Model inference and predictions
│ │ └── predictor.py # ModelPredictor class for inference
│ ├── visualization/ # Visualization tools
│ │ ├── visualization.py # Interactive visualizations
│ │ └── visualize_tree.py # Tree/graph visualization utilities
│ └── __main__.py # CLI entry point
├── configs/ # Hydra configuration files
│ ├── ciao.yaml # Main entry point
│ ├── base.yaml # Base configuration
│ ├── data/ # Data configurations
│ │ └── default.yaml
│ ├── explanation/ # Explanation method configs
│ │ └── ciao_default.yaml # Default CIAO parameters
│ ├── hydra/ # Hydra settings
│ └── logger/ # Logger configurations
└── pyproject.toml # Project metadata and dependencies
```
46 changes: 42 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,13 +1,51 @@
[project]
name = "rationai-"
name = "rationai-ciao"
version = "0.1.0"
description = ""
authors = []
description = "CIAO: Contextual Importance Assessment via Obfuscation - An XAI method for identifying influential image regions"
authors = [{ name = "David Halmazňa", email = "david.halmazna@mail.muni.cz" }]
requires-python = ">=3.11"
readme = "README.md"
license = { file = "LICENSE" }
dependencies = []
dependencies = [
# Core ML/DL frameworks
"torch>=2.0.0",
"torchvision>=0.15.0",

# Configuration and experiment tracking
"hydra-core>=1.3.0",
"mlflow>=3.0",
"omegaconf>=2.3.0",

# XAI and visualization
"matplotlib>=3.5.0",
"plotly>=5.0.0",
"ipywidgets>=7.0.0",

# Image processing and segmentation
"scikit-image>=0.19.0",
"pillow>=9.0.0",

# Scientific computing
"numpy>=1.21.0",
"networkx>=2.6.0",

# Others
"tqdm>=4.0.0",
]

[project.scripts]
ciao = "ciao.__main__:main"

[dependency-groups]
dev = ["mypy", "ruff"]
test = ["pytest", "pytest-cov"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.wheel]
packages = ["ciao"]

[tool.uv]
package = true
Loading