GENNI: Visualising the Geometry of Equivalences for Neural Network Identifiability

Disclaimer

This is code associated with the paper "GENNI: Visualising the Geometry of Equivalences for Neural Network Identifiability," published in the NeurIPS Workshop on Differential Geometry meets Deep Learning 2020.

If you have any questions, please feel free to reach out to us or make an issue.

Installing

Genni is available from PyPI here. In order to install simply use pip

pip install genni

Usage

In order to use the package, please set genni.yml in the top directory of your project and add / set the variable genni_home pointing to where genni should keep all of the generated files.

Generating a run

In order to calculate the approximate equivalence classes of parameters of your network architecture that leads to the same function you first need to create an experiment. An example file of how to do this can be found in scripts/experiment.py which has some architectures predefined, but you can add your own if you want to by looking at how the file is designed.

Generating an experiment can be done by calling

python scripts/experiment.py

We have included a experiments directory with the trained models that were used to genereate the images in the paper.

Getting directories and run IDs

After generating an experiment this will populate ${GENNI_HOME}/experiment with a directory having as a name the timestamp of when it was run. An easy way to look at the generated experiments is use the tree command. Below is an example output when running this after generating a couple of experiments

tree $GENNI_HOME/experiments -d -L 3

with the output

experiments
└── Nov09_19-52-12_isak-arch
    ├── models
    │   └── 1604947934.637504
    └── runs
        └── 1604947934.637504

where Nov09_19-52-12_isak-arch is the identifier of the experiment and 1604947934.637504 is an ID of a hyperparameter setting of this experiment.

Plotting

We have prepared a notebook called notebooks/SubspaceAnalysis.ipynb showing how to

• Load your experiment together with necessary paths and experiment ids
• Compute grids and values for plotting
• Different ways of visualising the approximate equivalence classes in the form of a
  • Contour plot
  • 3d iso-surface plot
  • UMAP projected 2d plot of 3d iso-surface

How to use FCN_Sample or LeNet_Sample to replicate the plots seen in the paper:

In the second cell of the notebook, choose experiment_idx=1 for FCN_Sample and experiment_idx=0 for LeNet_Sample. The griding parameters are set for FCN_Sample, to run LeNet_Sample change grid_bounds to [-20, 20].

Citing

If you use GENNI anywhere in your work, please cite use using

@article{2020,
    title={GENNI: Visualising the Geometry of Equivalences for Neural Network Identifiability},
    author={Lengyel, Daniel and Petangoda, Janith and Falk, Isak and Highnam, Kate and Lazarou, Michalis and Kolbeinsson, Arinbjörn and Deisenroth, Marc Peter and Jennings, Nicholas R.},
    booktitle={NeurIPS Workshop on Differential Geometry meets Deep Learning},
    year={2020}
}