vision: image.so on Apple Silicon can't load libpng16.16.dylib and libjpeg.9.dylib

🐛 Describe the bug

code:

import torchvision

UserWarning:

/Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: dlopen(/Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/image.so, 0x0006): Library not loaded: @rpath/libpng16.16.dylib
  Referenced from: /Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/image.so
  Reason: tried: '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/lib-dynload/../../libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/bin/../lib/libpng16.16.dylib' (no such file), '/usr/local/lib/libpng16.16.dylib' (no such file), '/usr/lib/libpng16.16.dylib' (no such file)
  warn(f"Failed to load image Python extension: {e}")

on M1 Max. In https://github.com/pytorch/vision/issues/5137#issuecomment-1023708994, it is said to be fixed but I don’t think so.

Therefore you can’t use torchvision.io.read_img().

import torchvision
img = torchvision.io.read_img('./1.jpg')

RuntimeError: No such operator image::read_file

In https://discuss.pytorch.org/t/failed-to-load-image-python-extension-could-not-find-module/140278/20, Andrade has the same problem.

Versions

Collecting environment information… PyTorch version: 1.10.2 Is debug build: False CUDA used to build PyTorch: None ROCM used to build PyTorch: N/A

OS: macOS 12.1 (arm64) GCC version: Could not collect Clang version: 13.0.0 (clang-1300.0.29.30) CMake version: version 3.22.0 Libc version: N/A

Python version: 3.9.7 (default, Sep 16 2021, 23:53:23) [Clang 12.0.0 ] (64-bit runtime) Python platform: macOS-12.1-arm64-arm-64bit Is CUDA available: False CUDA runtime version: No CUDA GPU models and configuration: No CUDA Nvidia driver version: No CUDA cuDNN version: No CUDA HIP runtime version: N/A MIOpen runtime version: N/A

Versions of relevant libraries: [pip3] mypy-extensions==0.4.3 [pip3] numpy==1.22.2 [pip3] torch==1.10.2 [pip3] torchaudio==0.10.2 [pip3] torchvision==0.11.3 [conda] Could not collect

cc @seemethere @malfet

About this issue

Original URL
State: open
Created 2 years ago
Reactions: 2
Comments: 17 (5 by maintainers)

Most upvoted comments

commands for mac

brew install libpng libjpeg
sudo ln -s /opt/homebrew/lib/* /usr/local/lib

fabiofumarola on Jun 9, 2022

@jdsgomes @malfet cc plz After thinking it for a long while, I find the key is libpng and libjpeg.

The image.so is indeed in torchvision/ even when torchvision=0.11.3, I only checked the torchvision/io/ folder before. So sorry for the previous title.

Then I take a deep look at the warning(same as the top):

/Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: dlopen(/Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/image.so, 0x0006): Library not loaded: @rpath/libpng16.16.dylib
  Referenced from: /Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/image.so
  Reason: tried: '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/lib-dynload/../../libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/bin/../lib/libpng16.16.dylib' (no such file), '/usr/local/lib/libpng16.16.dylib' (no such file), '/usr/lib/libpng16.16.dylib' (no such file)
  warn(f"Failed to load image Python extension: {e}")

I think the missing of @rpath/libpng16.16.dylib is the key. And the warning on my Mac even has a malfet name, which is so surprising.

Following the Reason: tried: ..... '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' ....., I try conda install libpng in my base conda env. Then I activate the torchvision env and import torchvision, the warning becomes Library not loaded: @rpath/libjpeg.9.dylib. This is very confusing.

What I think for now is that dlopen image.so needs both libpng and libjpeg and both of them can’t be found in ARM Mac, but I’m not sure.

alanyoungCN on Feb 14, 2022

FYI, I believe this problem can be solved by installing libpng and libjpeg via Homebrew, as it now works for me after doing so. Credit to https://discuss.pytorch.org/t/failed-to-load-image-python-extension-could-not-find-module/140278/26

cliberatore on May 23, 2022

Pinging @jdsgomes: we should try to triple check with the release engineers that M1 will be properly supported in the upcoming release.

NicolasHug on Feb 14, 2022

I think this would probably be resolved if we were building these in CI rather than on a local laptop. Maybe related to https://github.com/pytorch/vision/issues/5171 ?

seemethere on Mar 4, 2022

@zhiqwang You can try reinstalling pillow==6.1(6.x could be okay), which is mentioned at https://github.com/python-pillow/Pillow/issues/4130#issuecomment-570452778. Installing torchvision by conda will automatically use the latest pillow(7.0). I met this problem when I used conda to install torch. But UnsatisfiableError is weird.

alanyoungCN on Feb 14, 2022