uncbiag · jiaoyining · Apr 25, 2024 · Apr 25, 2024 · Apr 25, 2024 · Apr 25, 2024
diff --git a/.DS_Store b/.DS_Store
diff --git a/README.md b/README.md
@@ -1,5 +1,172 @@
-# NAISR
-NAISR: A 3D Neural Additive Model for Interpretable Shape Representation
+# <img src="NAISRicon.svg" width="35" height="35"> NAISR: A 3D Neural Additive Model for Interpretable Shape Representation
 
+Pytorch implementation for our `NAISR` paper <br>
+[NAISR: A 3D Neural Additive Model for Interpretable Shape Representation](https://arxiv.org/abs/2303.09234), ICLR 2024 **Spotlight**. <br>
+Yining Jiao, Carlton Zdanski, Julia Kimbell, Andrew Prince, Cameron Worden, Samuel Kirse, Christopher Rutter, Benjamin Shields, William Dunn, Jisan Mahmud, Marc Niethammer. <br>
+UNC-Chapel Hill
 
-Codes on the way...
+#### [Paper](https://arxiv.org/abs/2303.09234) | [Project Page](https://uncbiag.github.io/NAISR/) | [Colab Demos](https://colab.research.google.com/drive/1OudGynEydIXpAgfA9lvi5d0L86kK-uwg?usp=sharing)
+
+
+<div align=center><img src="https://github.com/uncbiag/NAISR/blob/main/figs/NAISRoverviewwide.svg?raw=true" width="1000" height="600"/></div>
+
+Please cite as:
+```
+@inproceedings{
+jiao2024naisr,
+title={\texttt{NAISR}: A 3D Neural Additive Model for Interpretable Shape Representation},
+author={Yining Jiao and Carlton Jude ZDANSKI and Julia S Kimbell and Andrew Prince and Cameron P Worden and Samuel Kirse and Christopher Rutter and Benjamin Shields and William Alexander Dunn and Jisan Mahmud and Marc Niethammer},
+booktitle={The Twelfth International Conference on Learning Representations},
+year={2024},
+url={https://openreview.net/forum?id=wg8NPfeMF9}
+}
+```
+
+>[!NOTE]
+>For the three datasets used in `NAISR` paper, we provided this [Colab demo](https://colab.research.google.com/drive/1OudGynEydIXpAgfA9lvi5d0L86kK-uwg) to investigate the shapes learned with `NAISR`. <br>
+We also provide the instructions on this page to apply `NAISR` to your customized shape analysis questions. 
+
+
+## Installation
+The code is tested with ``python=3.9``, ``torch=2.1.0``, ``torchvision=0.15.2``.
+```
+git clone https://github.com/uncbiag/NAISR
+cd NAISR
+```
+Now, create a new conda environment and install required packages accordingly.
+```
+conda create -n naisr python=3.9
+conda activate naisr
+pip install -r requirements.txt
+```
+
+
+## Data and Model Weights
+We train and test our method on three datasets: Starman, ADNI Hippocampus, and Pediatric Airways.
+
+| Dataset   |                      Description             |           Dataset Link              |   Model Link              |
+|-----------|----------------------------------------------|:------------------------------------:|:------------------------------------:|
+|Starman      |  simulated 2D Starman shapes      |  [simulation code)][StarmanData]          | [weights][StarmanModel]    | 
+|ADNI Hippocampus  |  1632 hippocampus from ADNI     |  [official site][ADNIData]              | [weights][ADNIModel]  |
+|Pedeatric Airway |  357 pediatric airway shapes      |  NA | [weights][AirwayModel]               |
+
+
+
+[ADNIData]: https://ida.loni.usc.edu/login.jsp?project=ADNI
+[StarmanData]: 
+
+[ADNIModel]: https://github.com/uncbiag/NAISR/releases/download/naisr_weights_v0/naisr_weights_adni.pth
+[StarmanModel]: https://github.com/uncbiag/NAISR/releases/download/naisr_weights_v0/naisr_weights_starman.pth
+[AirwayModel]: https://github.com/uncbiag/NAISR/releases/download/naisr_weights_v0/naisr_weights_pediatric_airway.pth
+
+
+## Visualization of Existing Shape Space
+To get the shape space like the following figures for the three datasets used in the paper, one just need to apply the model weights with the following command, 
+
+First, download the `NAISR model weights`.  These weights will be automatically saved to the ``checkpoints`` folder.
+```
+python download.py
+```
+
+Run interactive GUI with the downloaded weights. The ``assets`` contains images for demo.
+```
+python evolution_shapematrix.py -examples/hippocampus/naigsr_0920_base.json
+```
+
+
+
+
+## Customize
+### Data Preprocessing
+
+#### Alignment with Rigid Transformation
+The shapes to explore need to be registered with a rigid transformation (translation + rotation). 
+If paired point clouds are available, we recommend to use ; otherwise, we recommend to use ICP to register the point clouds. 
+In our case, we use airway landmarks to learn the rigid transformation; and ICP algorithms to register the point clouds of the hippocampi.
+
+#### SDF Extraction
+
+###
+```json
+ {
+  "Description" : [ "This experiment learns a shape representation for starman dataset." ],
+  "Device":0, 
+  "DataSource": {"train": "/home/jyn/NAISR/examples/starman/2dshape_train_with_temp.csv",
+                 "test": "/home/jyn/NAISR/examples/starman/2dshape_test_with_temp.csv"},
+  "Split": null,
+  "Network": "DeepNAIGSR",
+  "NumEpochs": 300,
+  "LoggingRoot": "/playpen-raid/jyn/NAISR/log",
+  "ExperimentName": "DeepNAIGSR_STARMAN3D_0222_256_base",
+
+  "EpochsTilCkpt": 10,
+  "StepsTilSummary": 1000,
+  "UseLBFGS": false,
+  "DoublePrecision": false,
+  "CheckpointPath": "",
+  "CodeLength": 256,
+
+  "AdditionalSnapshots" : [ 50, 100, 200, 300, 400, 500 ],
+  "LearningRateSchedule" : [
+    {
+      "Type": "Step",
+      "Initial": 0.00005,
+      "Interval": 1000,
+      "Factor": 0.5
+    },
+    {
+      "Type": "Step",
+      "Initial":  0.001,
+      "Interval": 1000,
+      "Factor": 0.5
+    }],
+  "SamplesPerScene" : 750,
+  "BatchSize": 64,
+  "DataLoaderThreads": 4,
+  "ClampingDistance": 1,
+
+  "Articulation": true,
+  "NumAtcParts": 1,
+  "TrainWithParts": false,
+  "Class": "starman",
+  "Attributes": ["cov_1", "cov_2"],
+   "TemplateAttributes":  {"cov_1": 0, "cov_2": 0},
+  "Backbone": "siren",
+  "PosEnc": false,
+  "InFeatures": 2,
+  "HiddenFeatures": 256,
+  "HidenLayers": 6,
+  "OutFeatures": 1,
+  "Loss": {
+    "whether_sdf": true, 
+    "whether_normal_constraint": true, 
+    "whether_inter_constraint": true, 
+    "whether_eikonal": true,
+    "whether_code_regularization": true}
+```
+### Training
+
+### Testing
+
+### Application - Visualization of Your Shape Space
+
+
+
+
+Instructions on the way...
+
+
+
+
+
+## Cite this work
+```
+@inproceedings{
+jiao2024naisr,
+title={\texttt{NAISR}: A 3D Neural Additive Model for Interpretable Shape Representation},
+author={Yining Jiao and Carlton Jude ZDANSKI and Julia S Kimbell and Andrew Prince and Cameron P Worden and Samuel Kirse and Christopher Rutter and Benjamin Shields and William Alexander Dunn and Jisan Mahmud and Marc Niethammer},
+booktitle={The Twelfth International Conference on Learning Representations},
+year={2024},
+url={https://openreview.net/forum?id=wg8NPfeMF9}
+}
+```
diff --git a/docs/NAISR_icon.png b/docs/NAISR_icon.png
diff --git a/docs/NAISR_icon.svg b/docs/NAISR_icon.svg
diff --git a/docs/NAISR poster (2).png → docs/NAISR_poster.png b/docs/NAISR poster (2).png → docs/NAISR_poster.png
diff --git a/docs/index.html b/docs/index.html
@@ -6,6 +6,7 @@
         content="Deformable Neural Radiance Fields creates free-viewpoint portraits (nerfies) from casually captured videos.">
   <meta name="keywords" content="Nerfies, D-NeRF, NeRF">
   <meta name="viewport" content="width=device-width, initial-scale=1">
+
   <title>NAISR: A 3D Neural Additive Model for Interpretable Shape Representation
 </title>
 
@@ -104,6 +105,16 @@ <h2 class="title is-2 publication-title"><tt>NAISR</tt>: A 3D Neural Additive Mo
                   </a>
               </span>
               <!-- video Link. -->
+              <span class="link-block">
+                <a href="https://colab.research.google.com/drive/1OudGynEydIXpAgfA9lvi5d0L86kK-uwg?usp=sharing"
+                   class="external-link button is-normal is-rounded is-dark">
+                  <span class="icon">
+                      <i class="fab fa-google"></i>
+                  </span>
+                  <span>Colab</span>
+                  </a>
+              </span>
+              <!-- video Link. -->
               <span class="link-block">
                 <a href="https://www.youtube.com/watch?v=8wSIIPAvlLI&t=43s"
                    class="external-link button is-normal is-rounded is-dark">

diff --git a/figs/.DS_Store b/figs/.DS_Store
diff --git a/figs/Manifold.png b/figs/Manifold.png
diff --git a/figs/NAISR_fig.png b/figs/NAISR_fig.png
diff --git a/figs/NAISRoverviewwide.pdf b/figs/NAISRoverviewwide.pdf
diff --git a/figs/NAISRoverviewwide.svg b/figs/NAISRoverviewwide.svg
diff --git a/figs/Presentation2.pptx b/figs/Presentation2.pptx
diff --git a/figs/dataset.svg b/figs/dataset.svg
diff --git a/figs/manifold.pdf b/figs/manifold.pdf
diff --git a/figs/naisr.png b/figs/naisr.png
diff --git a/figs/overview.svg b/figs/overview.svg
diff --git a/figs/recon.pptx b/figs/recon.pptx
diff --git a/figs/reconstruction.pdf b/figs/reconstruction.pdf
diff --git a/figs/reconstruction.png b/figs/reconstruction.png
diff --git a/figs/reconstructions.pdf b/figs/reconstructions.pdf