data: import fails on Linux due to missing library libcrypto.so.10

torchvision is seeing CI failures on Linux for a missing extension loaded by torchdata. I can reproduce with docker:

docker run python:3.7 bash -c \
    'pip install --pre torch torchdata --extra-index-url https://download.pytorch.org/whl/nightly/cpu && python -c "import torchdata"'

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.7/site-packages/torchdata/__init__.py", line 7, in <module>
    from torchdata import _extension  # noqa: F401
  File "/usr/local/lib/python3.7/site-packages/torchdata/_extension.py", line 34, in <module>
    _init_extension()
  File "/usr/local/lib/python3.7/site-packages/torchdata/_extension.py", line 31, in _init_extension
    from torchdata import _torchdata as _torchdata
ImportError: libcrypto.so.10: cannot open shared object file: No such file or directory

#399 might be the offender. #415 did not make it into yesterday’s nightly.

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 15 (15 by maintainers)

Most upvoted comments

@ejguan Thanks a lot for the help. As far as I can tell, this fixed the problem. If no additional action is required on your side, I think we can close the issue.

@datumbox Thank you for confirmation. Let’s keep this issue opened until I fix AWS binary.

ejguan on May 19, 2022

I don’t think you need to disable CI. I can re-upload a new TorchData binary without AWSSDK. This should resolve the problem for TorchVision CI. I will open a separate PR to support AWSSDK

ejguan on May 19, 2022

I am working on a PR to compile AWS with static libcrypto and shipped with TorchData.

ejguan on May 19, 2022

I think openssl-static in […] is culprit

Nope. As you can see above, libcrypto.so.10 is already present in the vanilla image without installing anything else.

$ docker run -it quay.io/pypa/manylinux2014_x86_64 bash
$ ls -l /usr/lib64 | grep libcrypto
lrwxrwxrwx  1 root root      19 Apr 18 12:29 libcrypto.so.10 -> libcrypto.so.1.0.2k
-rwxr-xr-x  1 root root 2520920 Mar 28 15:42 libcrypto.so.1.0.2k
$ yum -y install openssl-devel openssl-static curl-devel zlib-devel
[...]
$ ls -l /usr/lib64 | grep libcrypto
-rw-r--r--  1 root root 4697014 Mar 28 15:42 libcrypto.a
lrwxrwxrwx  1 root root      19 May 18 13:57 libcrypto.so -> libcrypto.so.1.0.2k
lrwxrwxrwx  1 root root      19 Apr 18 12:29 libcrypto.so.10 -> libcrypto.so.1.0.2k
-rwxr-xr-x  1 root root 2520920 Mar 28 15:42 libcrypto.so.1.0.2k

pmeier on May 18, 2022