rapidyaml: Installing rapidyaml via pip on macOS fails
I am currently trying to install rapidyaml in a conda environment on macOS (Sequoia 15.3.2 (24D81)) via pip. Unfortunately, this install fails with the following error
Error
Collecting rapidyaml
Using cached rapidyaml-0.1.0.post60.tar.gz (651 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: rapidyaml
Building wheel for rapidyaml (pyproject.toml) ... error
error: subprocess-exited-with-error
Γ Building wheel for rapidyaml (pyproject.toml) did not run successfully.
β exit code: 1
β°β> [217 lines of output]
/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools_scm/_integration/setuptools.py:92: UserWarning: version of rapidyaml already set
warnings.warn(f"version of {dist_name} already set")
running bdist_wheel
running build
running build_py
creating build/lib.macosx-11.1-arm64-cpython-313/ryml
copying api/python/ryml/version.py -> build/lib.macosx-11.1-arm64-cpython-313/ryml
copying api/python/ryml/__init__.py -> build/lib.macosx-11.1-arm64-cpython-313/ryml
running egg_info
writing api/python/rapidyaml.egg-info/PKG-INFO
writing dependency_links to api/python/rapidyaml.egg-info/dependency_links.txt
writing top-level names to api/python/rapidyaml.egg-info/top_level.txt
ERROR setuptools_scm._file_finders.git listing git files failed - pretending there aren't any
reading manifest file 'api/python/rapidyaml.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE.txt'
writing manifest file 'api/python/rapidyaml.egg-info/SOURCES.txt'
/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/command/build_py.py:212: _Warning: Package 'ryml.tests' is absent from the `packages` configuration.
!!
********************************************************************************
############################
# Package would be ignored #
############################
Python recognizes 'ryml.tests' as an importable package[^1],
but it is absent from setuptools' `packages` configuration.
This leads to an ambiguous overall configuration. If you want to distribute this
package, please make sure that 'ryml.tests' is explicitly added
to the `packages` configuration field.
Alternatively, you can also rely on setuptools' discovery methods
(for example by using `find_namespace_packages(...)`/`find_namespace:`
instead of `find_packages(...)`/`find:`).
You can read more about "package discovery" on setuptools documentation page:
- https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
If you don't want 'ryml.tests' to be distributed and are
already explicitly excluding 'ryml.tests' via
`find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
you can try to use `exclude_package_data`, or `include-package-data=False` in
combination with a more fine grained `package-data` configuration.
You can read more about "package data files" on setuptools documentation page:
- https://setuptools.pypa.io/en/latest/userguide/datafiles.html
[^1]: For Python, any directory (with suitable naming) can be imported,
even if it does not contain any `.py` files.
On the other hand, currently there is no concept of package data
directory, all directories are treated like packages.
********************************************************************************
!!
check.warn(importable)
creating build/lib.macosx-11.1-arm64-cpython-313/ryml/tests
copying api/python/ryml/tests/parse.py -> build/lib.macosx-11.1-arm64-cpython-313/ryml/tests
copying api/python/ryml/tests/parse_bm.py -> build/lib.macosx-11.1-arm64-cpython-313/ryml/tests
running build_ext
-- The CXX compiler identification is AppleClang 16.0.0.16000026
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- ryml: using C++ standard: C++11
-- ryml: setting C++ standard: 11
-- ryml: setting C++ standard required: ON
-- ryml: importing subproject c4core (SUBDIRECTORY)... /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b/ext/c4core
-- c4core: using C++ standard: C++11
CMake Warning (dev) at ext/c4core/cmake/c4Project.cmake:2229 (install):
Policy CMP0177 is not set: install() DESTINATION paths are normalized. Run
"cmake --help-policy CMP0177" for policy details. Use the cmake_policy
command to set the policy and suppress this warning.
Call Stack (most recent call first):
ext/c4core/cmake/c4Project.cmake:2306 (__c4_install_exports)
ext/c4core/CMakeLists.txt:81 (c4_install_exports)
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at ext/c4core/cmake/c4Project.cmake:2278 (install):
Policy CMP0177 is not set: install() DESTINATION paths are normalized. Run
"cmake --help-policy CMP0177" for policy details. Use the cmake_policy
command to set the policy and suppress this warning.
Call Stack (most recent call first):
ext/c4core/cmake/c4Project.cmake:2306 (__c4_install_exports)
ext/c4core/CMakeLists.txt:81 (c4_install_exports)
This warning is for project developers. Use -Wno-dev to suppress it.
-- ryml: -----> target ryml PUBLIC incorporating lib c4core
CMake Warning (dev) at ext/c4core/cmake/c4Project.cmake:2229 (install):
Policy CMP0177 is not set: install() DESTINATION paths are normalized. Run
"cmake --help-policy CMP0177" for policy details. Use the cmake_policy
command to set the policy and suppress this warning.
Call Stack (most recent call first):
ext/c4core/cmake/c4Project.cmake:2306 (__c4_install_exports)
CMakeLists.txt:69 (c4_install_exports)
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at ext/c4core/cmake/c4Project.cmake:2278 (install):
Policy CMP0177 is not set: install() DESTINATION paths are normalized. Run
"cmake --help-policy CMP0177" for policy details. Use the cmake_policy
command to set the policy and suppress this warning.
Call Stack (most recent call first):
ext/c4core/cmake/c4Project.cmake:2306 (__c4_install_exports)
CMakeLists.txt:69 (c4_install_exports)
This warning is for project developers. Use -Wno-dev to suppress it.
-- ryml: packing the project
-- ryml: enabling API
CMake Error at /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/cmake/data/share/cmake-4.0/Modules/FindPackageHandleStandardArgs.cmake:227 (message):
Could NOT find SWIG (missing: SWIG_EXECUTABLE SWIG_DIR)
Call Stack (most recent call first):
/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/cmake/data/share/cmake-4.0/Modules/FindPackageHandleStandardArgs.cmake:591 (_FPHSA_FAILURE_MESSAGE)
/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/cmake/data/share/cmake-4.0/Modules/FindSWIG.cmake:184 (find_package_handle_standard_args)
api/CMakeLists.txt:11 (find_package)
-- Configuring incomplete, errors occurred!
==> Configuring:
$ cmake -S /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b -B /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b/build/temp.macosx-11.1-arm64-cpython-313_ryml.ryml -G Ninja -DCMAKE_MAKE_PROGRAM=/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/bin/ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b/build/lib.macosx-11.1-arm64-cpython-313/ryml -DRYML_BUILD_API:BOOL=ON
==> Building:
$ cmake --build /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b/build/temp.macosx-11.1-arm64-cpython-313_ryml.ryml --config Release
==> Installing:
$ cmake --install /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b/build/temp.macosx-11.1-arm64-cpython-313_ryml.ryml --config Release --component python
Traceback (most recent call last):
File "/Users/david/miniconda3/envs/_tmp/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
main()
~~~~^^
File "/Users/david/miniconda3/envs/_tmp/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
json_out["return_val"] = hook(**hook_input["kwargs"])
~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/david/miniconda3/envs/_tmp/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 280, in build_wheel
return _build_backend().build_wheel(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
wheel_directory, config_settings, metadata_directory
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 438, in build_wheel
return _build(['bdist_wheel', '--dist-info-dir', str(metadata_directory)])
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 426, in _build
return self._build_with_temp_dir(
~~~~~~~~~~~~~~~~~~~~~~~~~^
cmd,
^^^^
...<3 lines>...
self._arbitrary_args(config_settings),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 407, in _build_with_temp_dir
self.run_setup()
~~~~~~~~~~~~~~^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 522, in run_setup
super().run_setup(setup_script=setup_script)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 320, in run_setup
exec(code, locals())
~~~~^^^^^^^^^^^^^^^^
File "<string>", line 69, in <module>
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/__init__.py", line 117, in setup
return distutils.core.setup(**attrs)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/core.py", line 186, in setup
return run_commands(dist)
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
dist.run_commands()
~~~~~~~~~~~~~~~~~^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
self.run_command(cmd)
~~~~~~~~~~~~~~~~^^^^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/dist.py", line 1104, in run_command
super().run_command(command)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
cmd_obj.run()
~~~~~~~~~~~^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/command/bdist_wheel.py", line 370, in run
self.run_command("build")
~~~~~~~~~~~~~~~~^^^^^^^^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
self.distribution.run_command(command)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/dist.py", line 1104, in run_command
super().run_command(command)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
cmd_obj.run()
~~~~~~~~~~~^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
self.distribution.run_command(command)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/dist.py", line 1104, in run_command
super().run_command(command)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
cmd_obj.run()
~~~~~~~~~~~^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/cmake_build_extension/build_extension.py", line 114, in run
self.build_extension(ext)
~~~~~~~~~~~~~~~~~~~~^^^^^
File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/cmake_build_extension/build_extension.py", line 241, in build_extension
subprocess.check_call(configure_command)
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
File "/Users/david/miniconda3/envs/_tmp/lib/python3.13/subprocess.py", line 421, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '-S', '/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b', '-B', '/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b/build/temp.macosx-11.1-arm64-cpython-313_ryml.ryml', '-G', 'Ninja', '-DCMAKE_MAKE_PROGRAM=/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/bin/ninja', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_INSTALL_PREFIX:PATH=/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b/build/lib.macosx-11.1-arm64-cpython-313/ryml', '-DRYML_BUILD_API:BOOL=ON']' returned non-zero exit status 1.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for rapidyaml
Failed to build rapidyaml
ERROR: Failed to build installable wheels for some pyproject.toml based projects (rapidyaml)
It should be noted that pip resorts to rapidyaml version 0.1.0 out of the box.
Is this a known issue or are there any workarounds to install the latest rapidyaml version on pip on macOS? Thanks in advance:)
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 2
- Comments: 23 (12 by maintainers)
Commits related to this issue
- Fix wheel import problem in macosx Kudos to @davidrudlstorfer for identifying the fix in #516. — committed to biojppm/rapidyaml by biojppm 2 years ago
- Fix wheel import problem in macosx Kudos to @davidrudlstorfer for identifying the fix in #516. — committed to biojppm/rapidyaml by biojppm 2 years ago
- Fix wheel import problem in macosx Kudos to @davidrudlstorfer for identifying the fix in #516. — committed to biojppm/rapidyaml by biojppm 2 years ago
- Fix wheel import problem in macosx Kudos to @davidrudlstorfer for identifying the fix in #516. — committed to biojppm/rapidyaml by biojppm 2 years ago
@davidrudlstorfer kudos! That allows all the wheel tests to pass. Thanks for sticking through this investigation! Now, please hold on while I clean up and prepare the PR.
@biojppm thanks for your nice insights into this topic - this looks really good.
overall I must confess that I am pretty overwhelmed by the whole setup process to package the wheel for rapidyaml. Therefore, I did a lot of debugging with the help of ChatGPT and other tools.
After a lot of online research I found this article https://github.com/swig/swig/issues/2637
I tested it out locally with the following change
And somehow this fixed the problem. At import I no longer get a segfault but the following error regarding the version
This is probably connected to my weird version hack to build the wheel.
Overall I do not really know why this now works (I guess?)
Let me know what you think
EDIT: it works with
set_property(TARGET ${t} APPEND PROPERTY LINK_FLAGS "-flat_namespace")as well. Here https://stackoverflow.com/questions/32277351/what-does-expected-in-flat-namespace-mean is more information on thatI am really curious about your opinion π
@biojppm thanks a lot for the release!
because the wheels are only published for macOS x86-x64 the install does not work out of the box on recent ARM/M-series processors via
but with
it now works flawlessly with v0.9.0 π
I looked through the failing pipelines and was not able to locate the problems with wheel packaging on macOS 14/15/latest (which uses ARM based systems)?
For now this approach works fine but it would be for sure nice to also have the correct ARM based wheels on pypi. If necessary I could open a dedicated issue to also package for ARM and if thereβs a problem with that I would also be available look into any problems at any time:)
Release should be made within a couple of weeks.
Thanks for your hints @biojppm - the wheel built without a problem but the segfault is still present at import. I will investigate further