io: Can not build `tensorflow-io` from source on Apple M1

First I tried using python setup.py bdist_wheel, which builds wheel, but the installation prompts me

ERROR: Could not find a version that satisfies the requirement tensorflow-io-gcs-filesystem==0.24.0 (from tensorflow-io) (from versions: none)
ERROR: No matching distribution found for tensorflow-io-gcs-filesystem==0.24.0

So I changed it to install pip install dist/tensorflow_io-0.24.0-cp38-cp38-macosx_11_0_arm64.whl --no-deps without dependencies, so although it can be installed normally, it will prompt me that there is no dynamic library when running

/Users/sunruiqi/Desktop/io-0.24.0/tensorflow_io/python/ops/__init__.py:98: UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths: ['/Users/sunruiqi/Desktop/io-0.24.0/tensorflow_io/python/ops/libtensorflow_io_plugins.so']
caused by: ["[Errno 2] The file to load file system plugin from does not exist.: '/Users/sunruiqi/Desktop/io-0.24.0/tensorflow_io/python/ops/libtensorflow_io_plugins.so'"]
warnings.warn(f"unable to load libtensorflow_io_plugins.so: {e}")
/Users/sunruiqi/Desktop/io-0.24.0/tensorflow_io/python/ops/__init__.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths: ['/Users/sunruiqi/Desktop/io-0.24.0/tensorflow_io/python/ops/libtensorflow_io.so']
caused by: ["dlopen(/Users/sunruiqi/Desktop/io-0.24.0/tensorflow_io/python/ops/libtensorflow_io.so, 0x0006): tried: '/Users/sunruiqi/Desktop/io-0.24.0/tensorflow_io/python/ops/libtensorflow_io.so' (no such file), '/usr/local/lib/libtensorflow_io.so' (no such file), '/usr/lib/libtensorflow_io.so' (no such file)"]
warnings.warn(f"file system plugins are not loaded: {e}")

After the above method fails, I refer to the build method in the documentation, unfortunately, run this command bazel build -s --verbose_failures $BAZEL_OPTIMIZATION //tensorflow_io/... //tensorflow_io_gcs_filesystem/... will produce the following error:

ERROR: While resolving toolchains for target //tensorflow_io/core:golang_ops: no matching toolchains found for types @io_bazel_rules_go//go:toolchain
ERROR: Analysis of target '//tensorflow_io/core:golang_ops' failed; build aborted: no matching toolchains found for types @io_bazel_rules_go//go:toolchain

So, what should I do?

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 8
Comments: 32 (4 by maintainers)

Most upvoted comments

@ZaibanAli Thank you very much for your help, but still there is no dynamic library error.

sun1638650145 on Feb 19, 2022

@flippinroo2 tensorflow does not seem to be planning official support for Apple silicon. I have provided several tensorflow libraries whl files, and if you need them, you can download them directly from here.

sun1638650145 on Jan 9, 2024

Hello, I’m also facing the same issue while importing tensorflow-io 0.27.0. What exactly needs to be done to resolve these warnings?

I’m receiving the same warnings

(tfod-api-cpu) guites@macos io % python                   
Python 3.9.15 (main, Nov 24 2022, 08:28:41) 
[Clang 14.0.6 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow_io
~/another-test/io/tensorflow_io/python/ops/__init__.py:98: UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths: ['~/another-test/io/tensorflow_io/python/ops/libtensorflow_io_plugins.so']
caused by: ["[Errno 2] The file to load file system plugin from does not exist.: '~/another-test/io/tensorflow_io/python/ops/libtensorflow_io_plugins.so'"]
  warnings.warn(f"unable to load libtensorflow_io_plugins.so: {e}")
~/another-test/io/tensorflow_io/python/ops/__init__.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths: ['~/another-test/io/tensorflow_io/python/ops/libtensorflow_io.so']
caused by: ["dlopen(~/another-test/io/tensorflow_io/python/ops/libtensorflow_io.so, 0x0006): tried: '~/another-test/io/tensorflow_io/python/ops/libtensorflow_io.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/guites/another-test/io/tensorflow_io/python/ops/libtensorflow_io.so' (no such file), '~/another-test/io/tensorflow_io/python/ops/libtensorflow_io.so' (no such file)"]
  warnings.warn(f"file system plugins are not loaded: {e}")
>>> tensorflow_io.__version__
'0.26.0'

Tried using an arm64 tensorflow_addons wheel as provided by @sun1638650145, but the warnings persist.

still, I am able to run objecte detection jobs (https://www.tensorflow.org/hub/tutorials/tf2_object_detection).

guites on Nov 29, 2022

@yongtang Any update yet? I’m running into this issue as well. It does not let me use tfio.audio as a result.

What would be a workaround you recommend?

leochoo on May 29, 2022

@yongtang no dependency issues on my side.

ZaibanAli on Feb 28, 2022

Now, io can be built perfectly on Apple silicon. If anyone needs it, you can find the tutorial here. I hope it can be helpful to everyone.

sun1638650145 on Sep 9, 2023

I’m getting this message on every run, and it takes some like 5 seconds. After that the script follows.

/Users/tiagogouvea/python/io/tensorflow_io/python/ops/__init__.py:98: UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths: ['/Users/tiagogouvea/python/io/tensorflow_io/python/ops/libtensorflow_io_plugins.so']
caused by: ["[Errno 2] The file to load file system plugin from does not exist.: '/Users/tiagogouvea/python/io/tensorflow_io/python/ops/libtensorflow_io_plugins.so'"]
  warnings.warn(f"unable to load libtensorflow_io_plugins.so: {e}")
/Users/tiagogouvea/python/io/tensorflow_io/python/ops/__init__.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths: ['/Users/tiagogouvea/python/io/tensorflow_io/python/ops/libtensorflow_io.so']
caused by: ["dlopen(/Users/tiagogouvea/python/io/tensorflow_io/python/ops/libtensorflow_io.so, 0x0006): tried: '/Users/tiagogouvea/python/io/tensorflow_io/python/ops/libtensorflow_io.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/tiagogouvea/python/io/tensorflow_io/python/ops/libtensorflow_io.so' (no such file), '/Users/tiagogouvea/python/io/tensorflow_io/python/ops/libtensorflow_io.so' (no such file)"]
  warnings.warn(f"file system plugins are not loaded: {e}")

Using a conda environment, this is my build and install results:

> python setup.py build
Project: tensorflow-io
Exclude: ['tests', 'tests.*', 'tensorflow_io_gcs_filesystem', 'tensorflow_io_gcs_filesystem.*']
Install Requires: ['tensorflow-io-gcs-filesystem==0.32.0']
Project Rootpath: tensorflow_io
running build
running build_py
running build_ext
> ython setup.py install
Project: tensorflow-io
Exclude: ['tests', 'tests.*', 'tensorflow_io_gcs_filesystem', 'tensorflow_io_gcs_filesystem.*']
Install Requires: ['tensorflow-io-gcs-filesystem==0.32.0']
Project Rootpath: tensorflow_io
running install
/Users/tiagogouvea/anaconda3/envs/py310/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer, pypa/build or
        other standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
/Users/tiagogouvea/anaconda3/envs/py310/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer, pypa/build or
        other standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  self.initialize_options()
running bdist_egg
running egg_info
writing tensorflow_io.egg-info/PKG-INFO
writing dependency_links to tensorflow_io.egg-info/dependency_links.txt
writing requirements to tensorflow_io.egg-info/requires.txt
writing top-level names to tensorflow_io.egg-info/top_level.txt
reading manifest file 'tensorflow_io.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'tensorflow_io.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-11.1-arm64/egg
running install_lib
running build_py
running build_ext
creating build/bdist.macosx-11.1-arm64/egg
....
copying tensorflow_io.egg-info/top_level.txt -> build/bdist.macosx-11.1-arm64/egg/EGG-INFO
creating 'dist/tensorflow_io-0.32.0-py3.10-macosx-11.1-arm64.egg' and adding 'build/bdist.macosx-11.1-arm64/egg' to it
removing 'build/bdist.macosx-11.1-arm64/egg' (and everything under it)
Processing tensorflow_io-0.32.0-py3.10-macosx-11.1-arm64.egg
removing '/Users/tiagogouvea/anaconda3/envs/py310/lib/python3.10/site-packages/tensorflow_io-0.32.0-py3.10-macosx-11.1-arm64.egg' (and everything under it)
creating /Users/tiagogouvea/anaconda3/envs/py310/lib/python3.10/site-packages/tensorflow_io-0.32.0-py3.10-macosx-11.1-arm64.egg
Extracting tensorflow_io-0.32.0-py3.10-macosx-11.1-arm64.egg to /Users/tiagogouvea/anaconda3/envs/py310/lib/python3.10/site-packages
Adding tensorflow-io 0.32.0 to easy-install.pth file

Installed /Users/tiagogouvea/anaconda3/envs/py310/lib/python3.10/site-packages/tensorflow_io-0.32.0-py3.10-macosx-11.1-arm64.egg
Processing dependencies for tensorflow-io==0.32.0
Searching for tensorflow-io-gcs-filesystem==0.32.0
Reading https://pypi.org/simple/tensorflow-io-gcs-filesystem/
No local packages or working download links found for tensorflow-io-gcs-filesystem==0.32.0
error: Could not find suitable distribution for Requirement.parse('tensorflow-io-gcs-filesystem==0.32.0')

Thank you for your attention @kulinseth

TiagoGouvea on Jun 15, 2023

I built tensorflow from source, without using tensorflow-macos, but I’m still having issues building tensorflow-io.

Hi @sun1638650145 and @TiagoGouvea , can you please paste the build error ?

kulinseth on Jun 15, 2023

I built tensorflow from source, without using tensorflow-macos, but I’m still having issues building tensorflow-io.

sun1638650145 on Apr 7, 2023

The only difference I can see, is that I haven’t installed XCode (only use command line tools). I can try install xcode.

yongtang on Mar 11, 2022

@ZaibanAli @sun1638650145 My recent M1 laptop is having a firmware issue and is with apple repair now. I may only be able to take a look at building tensorflow-io after the laptop is back.

Before the firmware issue I was using pip to install tensorflow_macos though I remember I had an issue with hdf5 dependency. Do you encounter the same hdf5 dependencies?

yongtang on Feb 28, 2022