dnnv_manage fails while installing verifiers

[yusiyoh]

Hello and thank you for this work.

I wanted to use this framework however, dnnv_manage fails with all verifiers. I have built dnnv in linux and have dependencies.

Error is: dnnv._manage.errors.InstallError: Failed to install dependency: <verifier_name>

Thanks in advance

[dlshriver]

I cannot reproduce the issue. What is the exact output that you are getting?

Also, what is your OS, CPU architecture, and Python version? These may help diagnose the issue.

[yusiyoh]

Example error (I have same error in the end for other verifiers too):

 dnnv_manage install eran
INFO     2023-06-07 10:59:12,753 (dnnv_manage.install) installing eran
+ cd /home/ycs/.local/share/dnnv/verifier_virtualenvs
+ rm -rf eran
+ python -m venv eran
+ . eran/bin/activate
+ deactivate nondestructive
+ [ -n  ]
+ [ -n  ]
+ [ -n  -o -n  ]
+ [ -n  ]
+ unset VIRTUAL_ENV
+ [ ! nondestructive = nondestructive ]
+ VIRTUAL_ENV=/home/ycs/.local/share/dnnv/verifier_virtualenvs/eran
+ export VIRTUAL_ENV
+ _OLD_VIRTUAL_PATH=/home/ycs/.local/share/dnnv/opt/gurobi912/linux64/bin:/home/ycs/.cache/dnnv/m4-1.4.1/bin:/home/ycs/.cache/dnnv/gmp-6.1.2/bin:/home/ycs/.cache/dnnv/mpfr-4.1.0/bin:/home/ycs/.cache/dnnv/cddlib-0.94m/bin:/home/ycs/.local/bin:/home/ycs/miniconda3/envs/dnnv/bin:/home/ycs/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/gurobi1001/win64/bin:/mnt/c/Program Files (x86)/Common Files/Oracle/Java/javapath:/mnt/c/Program Files/Eclipse Adoptium/jdk-11.0.18.10-hotspot/bin:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0:/mnt/c/WINDOWS/System32/OpenSSH:/mnt/c/Program Files/dotnet:/mnt/c/Program Files/TortoiseSVN/bin:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Python310:/mnt/c/Program Files (x86)/Graphviz2.38/bin:/mnt/c/Program Files/Docker/Docker/resources/bin:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Python37/Scripts:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Python37:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Launcher:/mnt/c/Users/yusuf.simsek/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Microsoft VS Code/bin:/mnt/c/Users/yusuf.simsek/CMake/bin:/mnt/c/Program Files/CMake/bin:/snap/bin
+ PATH=/home/ycs/.local/share/dnnv/verifier_virtualenvs/eran/bin:/home/ycs/.local/share/dnnv/opt/gurobi912/linux64/bin:/home/ycs/.cache/dnnv/m4-1.4.1/bin:/home/ycs/.cache/dnnv/gmp-6.1.2/bin:/home/ycs/.cache/dnnv/mpfr-4.1.0/bin:/home/ycs/.cache/dnnv/cddlib-0.94m/bin:/home/ycs/.local/bin:/home/ycs/miniconda3/envs/dnnv/bin:/home/ycs/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/gurobi1001/win64/bin:/mnt/c/Program Files (x86)/Common Files/Oracle/Java/javapath:/mnt/c/Program Files/Eclipse Adoptium/jdk-11.0.18.10-hotspot/bin:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0:/mnt/c/WINDOWS/System32/OpenSSH:/mnt/c/Program Files/dotnet:/mnt/c/Program Files/TortoiseSVN/bin:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Python310:/mnt/c/Program Files (x86)/Graphviz2.38/bin:/mnt/c/Program Files/Docker/Docker/resources/bin:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Python37/Scripts:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Python37:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Launcher:/mnt/c/Users/yusuf.simsek/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Microsoft VS Code/bin:/mnt/c/Users/yusuf.simsek/CMake/bin:/mnt/c/Program Files/CMake/bin:/snap/bin
+ export PATH
+ [ -n  ]
+ [ -z  ]
+ _OLD_VIRTUAL_PS1=$
+ [ x(eran)  != x ]
+ PS1=(eran) $
+ export PS1
+ [ -n  -o -n  ]
+ pip install --upgrade pip
Requirement already satisfied: pip in ./eran/lib/python3.7/site-packages (22.0.4)
Collecting pip
  Using cached pip-23.1.2-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 22.0.4
    Uninstalling pip-22.0.4:
      Successfully uninstalled pip-22.0.4
Successfully installed pip-23.1.2
+ pip install numpy>=1.19,<1.22 tensorflow>=2.4,<2.8 onnx>=1.8,<1.11 onnxruntime>=1.7,<1.11 torch>=1.8,<1.11 torchvision>=0.9,<0.12 mpmath>=1.2,<1.3 pillow>=8.1 protobuf<=3.20
Collecting numpy<1.22,>=1.19
  Using cached numpy-1.21.6-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
Collecting tensorflow<2.8,>=2.4
  Using cached tensorflow-2.7.4-cp37-cp37m-manylinux2010_x86_64.whl (495.5 MB)
Collecting onnx<1.11,>=1.8
  Using cached onnx-1.10.2-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (12.7 MB)
Collecting onnxruntime<1.11,>=1.7
  Using cached onnxruntime-1.10.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.9 MB)
Collecting torch<1.11,>=1.8
  Using cached torch-1.10.2-cp37-cp37m-manylinux1_x86_64.whl (881.9 MB)
Collecting torchvision<0.12,>=0.9
  Using cached torchvision-0.11.3-cp37-cp37m-manylinux1_x86_64.whl (23.2 MB)
Collecting mpmath<1.3,>=1.2
  Using cached mpmath-1.2.1-py3-none-any.whl (532 kB)
Collecting pillow>=8.1
  Using cached Pillow-9.5.0-cp37-cp37m-manylinux_2_28_x86_64.whl (3.4 MB)
Collecting protobuf<=3.20
  Using cached protobuf-3.20.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB)
Collecting absl-py>=0.4.0 (from tensorflow<2.8,>=2.4)
  Using cached absl_py-1.4.0-py3-none-any.whl (126 kB)
Collecting astunparse>=1.6.0 (from tensorflow<2.8,>=2.4)
  Using cached astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting libclang>=9.0.1 (from tensorflow<2.8,>=2.4)
  Using cached libclang-16.0.0-py2.py3-none-manylinux2010_x86_64.whl (22.9 MB)
Collecting flatbuffers<3.0,>=1.12 (from tensorflow<2.8,>=2.4)
  Using cached flatbuffers-2.0.7-py2.py3-none-any.whl (26 kB)
Collecting google-pasta>=0.1.1 (from tensorflow<2.8,>=2.4)
  Using cached google_pasta-0.2.0-py3-none-any.whl (57 kB)
Collecting h5py>=2.9.0 (from tensorflow<2.8,>=2.4)
  Using cached h5py-3.8.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB)
Collecting keras-preprocessing>=1.1.1 (from tensorflow<2.8,>=2.4)
  Using cached Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Collecting opt-einsum>=2.3.2 (from tensorflow<2.8,>=2.4)
  Using cached opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Collecting protobuf<=3.20
  Using cached protobuf-3.19.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
Collecting six>=1.12.0 (from tensorflow<2.8,>=2.4)
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting termcolor>=1.1.0 (from tensorflow<2.8,>=2.4)
  Using cached termcolor-2.3.0-py3-none-any.whl (6.9 kB)
Collecting typing-extensions>=3.6.6 (from tensorflow<2.8,>=2.4)
  Downloading typing_extensions-4.6.3-py3-none-any.whl (31 kB)
Collecting wheel<1.0,>=0.32.0 (from tensorflow<2.8,>=2.4)
  Using cached wheel-0.40.0-py3-none-any.whl (64 kB)
Collecting wrapt>=1.11.0 (from tensorflow<2.8,>=2.4)
  Using cached wrapt-1.15.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting gast<0.5.0,>=0.2.1 (from tensorflow<2.8,>=2.4)
  Using cached gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting tensorboard~=2.6 (from tensorflow<2.8,>=2.4)
  Using cached tensorboard-2.11.2-py3-none-any.whl (6.0 MB)
Collecting tensorflow-estimator<2.8,~=2.7.0rc0 (from tensorflow<2.8,>=2.4)
  Using cached tensorflow_estimator-2.7.0-py2.py3-none-any.whl (463 kB)
Collecting keras<2.8,>=2.7.0rc0 (from tensorflow<2.8,>=2.4)
  Using cached keras-2.7.0-py2.py3-none-any.whl (1.3 MB)
Collecting tensorflow-io-gcs-filesystem>=0.21.0 (from tensorflow<2.8,>=2.4)
  Using cached tensorflow_io_gcs_filesystem-0.32.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.4 MB)
Collecting grpcio<2.0,>=1.24.3 (from tensorflow<2.8,>=2.4)
  Downloading grpcio-1.54.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.1/5.1 MB 7.0 MB/s eta 0:00:00
Collecting google-auth<3,>=1.6.3 (from tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Downloading google_auth-2.19.1-py2.py3-none-any.whl (181 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.3/181.3 kB 6.3 MB/s eta 0:00:00
Collecting google-auth-oauthlib<0.5,>=0.4.1 (from tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting markdown>=2.6.8 (from tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached Markdown-3.4.3-py3-none-any.whl (93 kB)
Collecting requests<3,>=2.21.0 (from tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Downloading requests-2.31.0-py3-none-any.whl (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 8.4 MB/s eta 0:00:00
Requirement already satisfied: setuptools>=41.0.0 in ./eran/lib/python3.7/site-packages (from tensorboard~=2.6->tensorflow<2.8,>=2.4) (47.1.0)
Collecting tensorboard-data-server<0.7.0,>=0.6.0 (from tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)
Collecting tensorboard-plugin-wit>=1.6.0 (from tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached tensorboard_plugin_wit-1.8.1-py3-none-any.whl (781 kB)
Collecting werkzeug>=1.0.1 (from tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached Werkzeug-2.2.3-py3-none-any.whl (233 kB)
Collecting cachetools<6.0,>=2.0.0 (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Downloading cachetools-5.3.1-py3-none-any.whl (9.3 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Downloading urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.1/143.1 kB 6.9 MB/s eta 0:00:00
Collecting requests-oauthlib>=0.7.0 (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting importlib-metadata>=4.4 (from markdown>=2.6.8->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached importlib_metadata-6.6.0-py3-none-any.whl (22 kB)
Collecting charset-normalizer<4,>=2 (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached charset_normalizer-3.1.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (171 kB)
Collecting idna<4,>=2.5 (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached certifi-2023.5.7-py3-none-any.whl (156 kB)
Collecting MarkupSafe>=2.1.1 (from werkzeug>=1.0.1->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Downloading MarkupSafe-2.1.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting zipp>=0.5 (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached zipp-3.15.0-py3-none-any.whl (6.8 kB)
Collecting pyasn1<0.6.0,>=0.4.6 (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting oauthlib>=3.0.0 (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached oauthlib-3.2.2-py3-none-any.whl (151 kB)
Installing collected packages: tensorflow-estimator, tensorboard-plugin-wit, mpmath, libclang, keras, flatbuffers, zipp, wrapt, wheel, urllib3, typing-extensions, termcolor, tensorflow-io-gcs-filesystem, tensorboard-data-server, six, pyasn1, protobuf, pillow, oauthlib, numpy, MarkupSafe, idna, grpcio, gast, charset-normalizer, certifi, cachetools, absl-py, werkzeug, torch, rsa, requests, pyasn1-modules, opt-einsum, onnxruntime, onnx, keras-preprocessing, importlib-metadata, h5py, google-pasta, astunparse, torchvision, requests-oauthlib, markdown, google-auth, google-auth-oauthlib, tensorboard, tensorflow
Successfully installed MarkupSafe-2.1.3 absl-py-1.4.0 astunparse-1.6.3 cachetools-5.3.1 certifi-2023.5.7 charset-normalizer-3.1.0 flatbuffers-2.0.7 gast-0.4.0 google-auth-2.19.1 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 grpcio-1.54.2 h5py-3.8.0 idna-3.4 importlib-metadata-6.6.0 keras-2.7.0 keras-preprocessing-1.1.2 libclang-16.0.0 markdown-3.4.3 mpmath-1.2.1 numpy-1.21.6 oauthlib-3.2.2 onnx-1.10.2 onnxruntime-1.10.0 opt-einsum-3.3.0 pillow-9.5.0 protobuf-3.19.6 pyasn1-0.5.0 pyasn1-modules-0.3.0 requests-2.31.0 requests-oauthlib-1.3.1 rsa-4.9 six-1.16.0 tensorboard-2.11.2 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 tensorflow-2.7.4 tensorflow-estimator-2.7.0 tensorflow-io-gcs-filesystem-0.32.0 termcolor-2.3.0 torch-1.10.2 torchvision-0.11.3 typing-extensions-4.6.3 urllib3-1.26.16 werkzeug-2.2.3 wheel-0.40.0 wrapt-1.15.0 zipp-3.15.0
+ cd /home/ycs/.local/share/dnnv/opt/gurobi912/linux64
+ python setup.py install
running install
running build
running build_py
creating build
creating build/lib
creating build/lib/gurobipy
copying lib/python3.7_utf32/gurobipy/__init__.py -> build/lib/gurobipy
copying lib/python3.7_utf32/gurobipy/gurobipy.so -> build/lib/gurobipy
running install_lib
creating /home/ycs/.local/share/dnnv/verifier_virtualenvs/eran/lib/python3.7/site-packages/gurobipy
copying build/lib/gurobipy/gurobipy.so -> /home/ycs/.local/share/dnnv/verifier_virtualenvs/eran/lib/python3.7/site-packages/gurobipy
copying build/lib/gurobipy/__init__.py -> /home/ycs/.local/share/dnnv/verifier_virtualenvs/eran/lib/python3.7/site-packages/gurobipy
byte-compiling /home/ycs/.local/share/dnnv/verifier_virtualenvs/eran/lib/python3.7/site-packages/gurobipy/__init__.py to __init__.cpython-37.pyc
running install_egg_info
Writing /home/ycs/.local/share/dnnv/verifier_virtualenvs/eran/lib/python3.7/site-packages/gurobipy-9.1.2-py3.7.egg-info
removing /home/ycs/.local/share/dnnv/opt/gurobi912/linux64/build
+ cd /home/ycs/.cache/dnnv/eran-8771d3158b2c64a360d5bdfd4433490863257dd6
+ [ ! -e eran ]
+ cd eran
+ cp -r tf_verify/__main__.py tf_verify/ai_milp.py tf_verify/analyzer.py tf_verify/config.py tf_verify/constraint_utils.py tf_verify/deeppoly_nodes.py tf_verify/deepzono_nodes.py tf_verify/eran.py tf_verify/eranlayers.py tf_verify/krelu.py tf_verify/onnx_translator.py tf_verify/optimizer.py tf_verify/read_net_file.py tf_verify/read_zonotope_file.py tf_verify/refine_activation.py tf_verify/refine_gpupoly.py tf_verify/spatial tf_verify/tensorflow_translator.py tf_verify/utils.py /home/ycs/.local/share/dnnv/verifier_virtualenvs/eran/lib/python3.7/site-packages/
Traceback (most recent call last):
  File "/home/ycs/miniconda3/envs/dnnv/bin/dnnv_manage", line 8, in <module>
    sys.exit(_main())
  File "/home/ycs/miniconda3/envs/dnnv/lib/python3.7/site-packages/dnnv/_manage/__main__.py", line 95, in _main
    return args.command(**params)
  File "/home/ycs/miniconda3/envs/dnnv/lib/python3.7/site-packages/dnnv/_manage/linux/__init__.py", line 18, in install
    import_verifier_module(verifier).install(environment)
  File "/home/ycs/miniconda3/envs/dnnv/lib/python3.7/site-packages/dnnv/_manage/linux/verifiers/eran.py", line 332, in install
    ProgramDependency("grbgetkey", installer=gurobi_installer),
  File "/home/ycs/miniconda3/envs/dnnv/lib/python3.7/site-packages/dnnv/_manage/linux/environment/base.py", line 94, in ensure_dependencies
    self.ensure_dependency(dependency)
  File "/home/ycs/miniconda3/envs/dnnv/lib/python3.7/site-packages/dnnv/_manage/linux/environment/base.py", line 89, in ensure_dependency
    raise InstallError(f"Failed to install dependency: {dependency.name}")
dnnv._manage.errors.InstallError: Failed to install dependency: eran

I am using conda 4.12.0 on:

Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy

and my environment has the attached packages.dnnv_conda.txt

My CPU:

 *-cpu
      product: 11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz
      vendor: Intel Corp.
      physical id: 1
      bus info: cpu@0
      version: 6.141.1
      width: 64 bits

Python ==> python=3.7.16

Thanks again.

[dlshriver]

I think the issue is probably the use of conda. It uses its own virtual environment setup that is not compatible with the verifier installation scripts.