CrackSeg 🔍

Semantic segmentation for surface crack detection using fine-tuned CLIPSeg.

---

Overview

CrackSeg fine-tunes CLIPSeg — a vision-language segmentation model — on a 14,000+ image crack dataset to produce pixel-wise binary masks for surface crack detection. The model takes an image as input and returns a segmentation mask highlighting crack regions.

Metric	Score
Dice Score	0.612
mIoU	0.716

---

Architecture

• Base model: CLIPSeg (CIDAS/clipseg-rd64-refined)
• Fine-tuning strategy: Partial — decoder fully unfrozen, last 2 layers of CLIP vision encoder, last 1 layer of CLIP text encoder
• Loss: Focal Loss (α=0.75, γ=2.0) to handle severe class imbalance from thin crack regions
• Optimizer: AdamW with differential learning rates
  • Decoder: 5e-5
  • Vision encoder layers: 2e-6
  • Text encoder layers: 1e-6
• Scheduler: CosineAnnealingLR
• Gradient clipping: max norm 1.0
• Early stopping: patience = 5

---

Dataset

• Source: Roboflow (COCO format)
• Size: 14,000+ images
• Splits: Train / Validation / Test
• Annotation pipeline: COCO JSON → binary mask generation via polygon rasterization with bbox fallback

---

Results

The model correctly identifies thin surface cracks under varied lighting conditions. Known failure cases include shadowed regions being misidentified as cracks — a known limitation of texture-based segmentation.

---

Project Structure

crackseg/
├── notebook/
│   └── Cracks.ipynb
├── app.py
├── requirements.txt
├── .gitignore
├── LICENSE.md
└── README.md

---

Run Locally

python -m venv venv

# Windows
venv\Scripts\activate

# Mac/Linux
source venv/bin/activate

pip install -r requirements.txt
python app.py

Weights are loaded automatically from HuggingFace Hub (primus29/crackseg).

---

Live Demo

Try it on HuggingFace Spaces — upload any image of a surface and get back a segmentation mask and red overlay.