Could not load library libcudnn_cnn_infer.so.8

[rayryeng]

Right after the model checkpoint is loaded, the following error occurs:

Error: Could not load library libcudnn_cnn_infer.so.8

Resolution

This happens because cuDNN 8.0 does not exist after setting up the environment, most likely due to the older version of PyTorch being used (2.0.0) and it only supporting CUDA 11.8. To resolve this, you need to do the following. These instructions are for Ubuntu 22.04, both WSL2 and native.

1. Download cuDNN from the NVIDIA archives: https://developer.nvidia.com/rdp/cudnn-archive. Choose version 8.9.7 for whatever OS you're using. For example, I am using WSL2 with Ubuntu 22.04, so I downloaded this file: cudnn-local-repo-ubuntu2204-8.9.7.29_1.0-1_amd64.deb
2. Extract the package and install it: sudo dpkg -i cudnn-local-repo-ubuntu2204-8.9.7.29_1.0-1_amd64.deb
3. You will need to install the GPG key after it's installed. It will tell you this after you install the package: sudo cp /var/cudnn-local-repo-ubuntu2204-8.9.7.29/cudnn-local-8AE81B24-keyring.gpg /usr/share/keyrings/
4. Install cuDNN 8: sudo apt-get install --reinstall libcudnn8 libcudnn8-dev libcudnn8-samples
5. Check that cuDNN 8 has been installed: ls /usr/lib/x86_64-linux-gnu/libcudnn* | grep libcudnn_cnn_infer.so.8
6. Update your LD_LIBRARY_PATH so that it points to /usr/lib/x86_64-linux-gnu: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/x86_64-linux-gnu/

Now if you run the script in /scripts/run.sh, it should work. If desired, I can also update the README to specify these instructions.