Mask R-CNN, Finetuned for the FireAssist project

This project marks the first POC of the FireAssist project, which does not reflect the current status of the project, which is confidential in nature. The aim of the project was to automate the score and accuracy calculation from a soldier's firearm usage on a 🎯 "bullseye" target. This repository serves as a proof-of-concept for the image segmentation part of the pipeline (see ASCII diagram below for more details).

The project was split into two main parts:

1. Mask segmentation - This repository
2. Bullet hole detection and decision inference (BulletsAPI) - CONFIDENTIAL

The main pipeline was structured in this format:

                                                  ┌───────────────────────┐   ┌───────────────────────┐
 ┌───────────────────────┐                        │   Firing score and    │   │Verdict on performance │
 │     Raw Image of      │                        │       accuracy        │   │and possible issues for│
 │ Target + Environment  │                        │                       │   │     future action     │
 └───────────┬───────────┘                        └───────────▲───────────┘   └───────────▲───────────┘
             │                                                │                           │            
             │Mask R-CNN inference                            │                           │            
             │                                                └─────────────┬─────────────┘            
 ┌───────────▼───────────┐                                                  │                          
 │    Segmented Image    │                                Use the provided  │  Use a multi-class       
 │                       │                                formula to        │  classifier ML model     
 │(Skewed circle target) │                                calculate score   │  to get verdict on       
 └───────────┬───────────┘                                                  │  firer (e.g. need        
             │                                                              │  training, possible      
             │Image processing and                                          │  injury, etc.)           
             │co-ordinate math                                              │                          
             │                                                              │                          
┌────────────▼────────────┐                                   ┌─────────────┴───────────┐              
│      Square Image       │  (Confidential part starts here)  │Bullet hole co-ordinates │              
│                         ├───────────────────────────────────▶     on square image     │              
│(perfect circular target)│   Bullets API scans the image,    └─────────────────────────┘              
└─────────────────────────┘   and runs object detection to                                             
                              find bullet holes                                                        

How to navigate the repository

The repository itself is mostly code by matterport, but the files of concern are:

• Main iteration and testing - scratchpad notebook
• Final POC of how the segmentation model would work
• Simple FastAPI server to act as a microservice

Please keep in mind that at the time of the last changes done to the file, the training set was too small, so the data may be prone to overfit. It served well as a proof-of-concept nonetheless. The project was likely put through more iteration and refined for the stakeholders' use by now, but I do not have visibility of that.

Below, the source project by matterport is added for quick reference: https://github.com/matterport/Mask_RCNN