pytorch_geometric: OSError: libcusparse.so.11: cannot open shared object file: No such file or directory

📚 Installation


Error occurs when I run from torch_geometric.data import Data

OSError Traceback (most recent call last) <ipython-input-1-767c8a8c19d6> in <module> 1 import torch ----> 2 from torch_geometric.data import Data

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/init.py in <module> 1 from .debug import is_debug_enabled, debug, set_debug ----> 2 import torch_geometric.nn 3 import torch_geometric.data 4 import torch_geometric.datasets 5 import torch_geometric.transforms

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/nn/init.py in <module> 1 from .meta import MetaLayer ----> 2 from .data_parallel import DataParallel 3 from .reshape import Reshape 4 from .conv import * # noqa 5 from .norm import * # noqa

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/nn/data_parallel.py in <module> 3 4 import torch ----> 5 from torch_geometric.data import Batch 6 7

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/data/init.py in <module> ----> 1 from .data import Data 2 from .batch import Batch 3 from .dataset import Dataset 4 from .in_memory_dataset import InMemoryDataset 5 from .dataloader import DataLoader, DataListLoader, DenseDataLoader

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/data/data.py in <module> 6 import torch 7 import torch_geometric ----> 8 from torch_sparse import coalesce, SparseTensor 9 from torch_geometric.utils import (contains_isolated_nodes, 10 contains_self_loops, is_undirected)

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_sparse/init.py in <module> 11 ]: 12 torch.ops.load_library(importlib.machinery.PathFinder().find_spec( —> 13 library, [osp.dirname(file)]).origin) 14 15 if torch.cuda.is_available() and torch.version.cuda: # pragma: no cover

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch/_ops.py in load_library(self, path) 103 # static (global) initialization code in order to register custom 104 # operators with the JIT. –> 105 ctypes.CDLL(path) 106 self.loaded_libraries.add(path) 107

~/anaconda3/envs/pytorch_env/lib/python3.7/ctypes/init.py in init(self, name, mode, handle, use_errno, use_last_error) 362 363 if handle is None: –> 364 self._handle = _dlopen(self._name, mode) 365 else: 366 self._handle = handle

OSError: libcusparse.so.11: cannot open shared object file: No such file or directory

Environment

  • OS: Ubuntu 18.04
  • Python version: 3.7.6
  • PyTorch version: 1.7.1
  • CUDA/cuDNN version: 11.0
  • GCC version: 7.5.0
  • How did you try to install PyTorch Geometric and its extensions (wheel, source): Binaries
  • Any other relevant information: followed this guide Installation via Binaries

Checklist

  • [y] I followed the installation guide.
  • I cannot find my error message in the FAQ.
  • I set up CUDA correctly and can compile CUDA code via nvcc. <— not sure how to do this, but I have been running pytorch on GPU fine all this while
  • I do have multiple CUDA versions on my machine. <— not sure how to check but when I run python -c "import torch; print(torch.__version__)" returns 1.7.1+cu110 and python -c "import torch; print(torch.version.cuda)" returns 11.0

Additional context

Installed via:

pip install --no-index torch-scatter -f https://pytorch-geometric.com/whl/torch-1.7.0+cu110.html
pip install --no-index torch-sparse -f https://pytorch-geometric.com/whl/torch-1.7.0+cu110.html
pip install --no-index torch-cluster -f https://pytorch-geometric.com/whl/torch-1.7.0+cu110.html
pip install --no-index torch-spline-conv -f https://pytorch-geometric.com/whl/torch-1.7.0+cu110.html
pip install torch-geometric

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 1
  • Comments: 31 (10 by maintainers)

Most upvoted comments

You can try to add the minconda CUDA libs installed by PyTorch to $LD_LIBRARY_PATH, e.g.:

export LD_LIBRARY_PATH="your_miniconda_path/lib:$LD_LIBRARY_PATH"

This folder should contain libcusparse.so.11.

I have write it “export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH” in ~/.bashrc, but it doesn’t work, it’s so weird.

However, I can use geometric in terminal, but can’t pycharm

@smiles724 @zilangch @domilay I found a solution in Pycharm. It can be solved by manually adding LIBRARY_PATH in the run/debug configurations. image

@rusty1s I did this but still was not working. I have just got it to work through downgrading the CUDA version for both PyTorch and the PyTorch Geometric dependencies from 11.1 to 10.2

I have write it “export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH” in ~/.bashrc, but it doesn’t work, it’s so weird.

However, I can use geometric in terminal, but can’t pycharm

If you are inside a custom miniconda environment, you may also find them inside miniconda3/envs/{env_name}/lib. Can you check? Note that you need to run conda install cudatoolkit=... -c pytorch first.

You can try to add the minconda CUDA libs installed by PyTorch to $LD_LIBRARY_PATH, e.g.:

export LD_LIBRARY_PATH="your_miniconda_path/lib:$LD_LIBRARY_PATH"

This folder should contain libcusparse.so.11.

@rusty1s I find that my miniconda/lib folder do not contain the “libcusparse.so.11”. How can I install it?

I had similar issue with OP.
I was set conda environment with python 3.8.x. pytorch 1.8.1 with cuda 11.1, and pyg 1.7.2. It works totally fine with cuda 11.1 but when I upgraded pytorch 1.8.1 to 1.9.0 along with compatible torch-geometric by using pip and try to import some modules e.g., GCNConv it will show the error as mentioned in OP: OSError: libcusparse.so.11: cannot open shared object file: No such file or directory.

I checked environment’s lib, there actually is libcusparse.so.11 but it seems pyg could not find it somehow after upgrading. I linked the folder containing libcusparse.so.11 to LD_LIBRARY_PATH, however, it doesn’t work for me.

So I tried to create new conda-environment with python 3.9.6 and install pytorch 1.9.0 with cuda11.1 plus compatible pyg by using conda. I am able to execute pyg’s lib without any error so far. I could not provide an accurate reason about this issue but hope this information can help someone somehow.

I have write it “export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH” in ~/.bashrc, but it doesn’t work, it’s so weird. However, I can use geometric in terminal, but can’t pycharm

I met a similar case. I can run it in the terminal but not in the jupyter notebook

I solved this by:

Steps:

  1. Open the .bashrc file in your home directory (for example, /home/your-user-name/.bashrc) in a text editor.
  2. Add export “export LD_LIBRARY_PATH=“your_minicondaORanaconda_path/lib:$LD_LIBRARY_PATH”” to the last line of the file
  3. Save the .bashrc file.
  4. Restart(maybe)