rapidyaml: Installing rapidyaml via pip on macOS fails

I am currently trying to install rapidyaml in a conda environment on macOS (Sequoia 15.3.2 (24D81)) via pip. Unfortunately, this install fails with the following error

Error
Collecting rapidyaml
  Using cached rapidyaml-0.1.0.post60.tar.gz (651 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: rapidyaml
  Building wheel for rapidyaml (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  Γ— Building wheel for rapidyaml (pyproject.toml) did not run successfully.
  β”‚ exit code: 1
  ╰─> [217 lines of output]
      /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools_scm/_integration/setuptools.py:92: UserWarning: version of rapidyaml already set
        warnings.warn(f"version of {dist_name} already set")
      running bdist_wheel
      running build
      running build_py
      creating build/lib.macosx-11.1-arm64-cpython-313/ryml
      copying api/python/ryml/version.py -> build/lib.macosx-11.1-arm64-cpython-313/ryml
      copying api/python/ryml/__init__.py -> build/lib.macosx-11.1-arm64-cpython-313/ryml
      running egg_info
      writing api/python/rapidyaml.egg-info/PKG-INFO
      writing dependency_links to api/python/rapidyaml.egg-info/dependency_links.txt
      writing top-level names to api/python/rapidyaml.egg-info/top_level.txt
      ERROR setuptools_scm._file_finders.git listing git files failed - pretending there aren't any
      reading manifest file 'api/python/rapidyaml.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      adding license file 'LICENSE.txt'
      writing manifest file 'api/python/rapidyaml.egg-info/SOURCES.txt'
      /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/command/build_py.py:212: _Warning: Package 'ryml.tests' is absent from the `packages` configuration.
      !!
      
              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'ryml.tests' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.
      
              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'ryml.tests' is explicitly added
              to the `packages` configuration field.
      
              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).
      
              You can read more about "package discovery" on setuptools documentation page:
      
              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
      
              If you don't want 'ryml.tests' to be distributed and are
              already explicitly excluding 'ryml.tests' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.
      
              You can read more about "package data files" on setuptools documentation page:
      
              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
      
      
              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************
      
      !!
        check.warn(importable)
      creating build/lib.macosx-11.1-arm64-cpython-313/ryml/tests
      copying api/python/ryml/tests/parse.py -> build/lib.macosx-11.1-arm64-cpython-313/ryml/tests
      copying api/python/ryml/tests/parse_bm.py -> build/lib.macosx-11.1-arm64-cpython-313/ryml/tests
      running build_ext
      -- The CXX compiler identification is AppleClang 16.0.0.16000026
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- ryml: using C++ standard: C++11
      -- ryml: setting C++ standard: 11
      -- ryml: setting C++ standard required: ON
      -- ryml: importing subproject c4core (SUBDIRECTORY)... /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b/ext/c4core
      -- c4core: using C++ standard: C++11
      CMake Warning (dev) at ext/c4core/cmake/c4Project.cmake:2229 (install):
        Policy CMP0177 is not set: install() DESTINATION paths are normalized.  Run
        "cmake --help-policy CMP0177" for policy details.  Use the cmake_policy
        command to set the policy and suppress this warning.
      Call Stack (most recent call first):
        ext/c4core/cmake/c4Project.cmake:2306 (__c4_install_exports)
        ext/c4core/CMakeLists.txt:81 (c4_install_exports)
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      CMake Warning (dev) at ext/c4core/cmake/c4Project.cmake:2278 (install):
        Policy CMP0177 is not set: install() DESTINATION paths are normalized.  Run
        "cmake --help-policy CMP0177" for policy details.  Use the cmake_policy
        command to set the policy and suppress this warning.
      Call Stack (most recent call first):
        ext/c4core/cmake/c4Project.cmake:2306 (__c4_install_exports)
        ext/c4core/CMakeLists.txt:81 (c4_install_exports)
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      -- ryml: -----> target ryml PUBLIC incorporating lib c4core
      CMake Warning (dev) at ext/c4core/cmake/c4Project.cmake:2229 (install):
        Policy CMP0177 is not set: install() DESTINATION paths are normalized.  Run
        "cmake --help-policy CMP0177" for policy details.  Use the cmake_policy
        command to set the policy and suppress this warning.
      Call Stack (most recent call first):
        ext/c4core/cmake/c4Project.cmake:2306 (__c4_install_exports)
        CMakeLists.txt:69 (c4_install_exports)
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      CMake Warning (dev) at ext/c4core/cmake/c4Project.cmake:2278 (install):
        Policy CMP0177 is not set: install() DESTINATION paths are normalized.  Run
        "cmake --help-policy CMP0177" for policy details.  Use the cmake_policy
        command to set the policy and suppress this warning.
      Call Stack (most recent call first):
        ext/c4core/cmake/c4Project.cmake:2306 (__c4_install_exports)
        CMakeLists.txt:69 (c4_install_exports)
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      -- ryml: packing the project
      -- ryml: enabling API
      CMake Error at /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/cmake/data/share/cmake-4.0/Modules/FindPackageHandleStandardArgs.cmake:227 (message):
        Could NOT find SWIG (missing: SWIG_EXECUTABLE SWIG_DIR)
      Call Stack (most recent call first):
        /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/cmake/data/share/cmake-4.0/Modules/FindPackageHandleStandardArgs.cmake:591 (_FPHSA_FAILURE_MESSAGE)
        /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/cmake/data/share/cmake-4.0/Modules/FindSWIG.cmake:184 (find_package_handle_standard_args)
        api/CMakeLists.txt:11 (find_package)
      
      
      -- Configuring incomplete, errors occurred!
      
      ==> Configuring:
      $ cmake -S /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b -B /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b/build/temp.macosx-11.1-arm64-cpython-313_ryml.ryml -G Ninja -DCMAKE_MAKE_PROGRAM=/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/bin/ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b/build/lib.macosx-11.1-arm64-cpython-313/ryml -DRYML_BUILD_API:BOOL=ON
      
      ==> Building:
      $ cmake --build /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b/build/temp.macosx-11.1-arm64-cpython-313_ryml.ryml --config Release
      
      ==> Installing:
      $ cmake --install /private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b/build/temp.macosx-11.1-arm64-cpython-313_ryml.ryml --config Release --component python
      
      Traceback (most recent call last):
        File "/Users/david/miniconda3/envs/_tmp/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
          main()
          ~~~~^^
        File "/Users/david/miniconda3/envs/_tmp/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
          json_out["return_val"] = hook(**hook_input["kwargs"])
                                   ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/david/miniconda3/envs/_tmp/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 280, in build_wheel
          return _build_backend().build_wheel(
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
              wheel_directory, config_settings, metadata_directory
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          )
          ^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 438, in build_wheel
          return _build(['bdist_wheel', '--dist-info-dir', str(metadata_directory)])
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 426, in _build
          return self._build_with_temp_dir(
                 ~~~~~~~~~~~~~~~~~~~~~~~~~^
              cmd,
              ^^^^
          ...<3 lines>...
              self._arbitrary_args(config_settings),
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          )
          ^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 407, in _build_with_temp_dir
          self.run_setup()
          ~~~~~~~~~~~~~~^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 522, in run_setup
          super().run_setup(setup_script=setup_script)
          ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 320, in run_setup
          exec(code, locals())
          ~~~~^^^^^^^^^^^^^^^^
        File "<string>", line 69, in <module>
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/__init__.py", line 117, in setup
          return distutils.core.setup(**attrs)
                 ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/core.py", line 186, in setup
          return run_commands(dist)
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
          dist.run_commands()
          ~~~~~~~~~~~~~~~~~^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
          self.run_command(cmd)
          ~~~~~~~~~~~~~~~~^^^^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/dist.py", line 1104, in run_command
          super().run_command(command)
          ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
          cmd_obj.run()
          ~~~~~~~~~~~^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/command/bdist_wheel.py", line 370, in run
          self.run_command("build")
          ~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
          self.distribution.run_command(command)
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/dist.py", line 1104, in run_command
          super().run_command(command)
          ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
          cmd_obj.run()
          ~~~~~~~~~~~^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
          ~~~~~~~~~~~~~~~~^^^^^^^^^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
          self.distribution.run_command(command)
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/dist.py", line 1104, in run_command
          super().run_command(command)
          ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
          cmd_obj.run()
          ~~~~~~~~~~~^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/cmake_build_extension/build_extension.py", line 114, in run
          self.build_extension(ext)
          ~~~~~~~~~~~~~~~~~~~~^^^^^
        File "/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/lib/python3.13/site-packages/cmake_build_extension/build_extension.py", line 241, in build_extension
          subprocess.check_call(configure_command)
          ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
        File "/Users/david/miniconda3/envs/_tmp/lib/python3.13/subprocess.py", line 421, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['cmake', '-S', '/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b', '-B', '/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b/build/temp.macosx-11.1-arm64-cpython-313_ryml.ryml', '-G', 'Ninja', '-DCMAKE_MAKE_PROGRAM=/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-build-env-c94rpxc5/overlay/bin/ninja', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_INSTALL_PREFIX:PATH=/private/var/folders/vy/7czp7f117p99xpj9gm6m9_dw0000gn/T/pip-install-2ivuj_xl/rapidyaml_f34cb2e5260842b0a9807b8e75e1dd7b/build/lib.macosx-11.1-arm64-cpython-313/ryml', '-DRYML_BUILD_API:BOOL=ON']' returned non-zero exit status 1.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for rapidyaml
Failed to build rapidyaml
ERROR: Failed to build installable wheels for some pyproject.toml based projects (rapidyaml)

It should be noted that pip resorts to rapidyaml version 0.1.0 out of the box.

Is this a known issue or are there any workarounds to install the latest rapidyaml version on pip on macOS? Thanks in advance:)

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 2
  • Comments: 23 (12 by maintainers)

Commits related to this issue

Most upvoted comments

@davidrudlstorfer kudos! That allows all the wheel tests to pass. Thanks for sticking through this investigation! Now, please hold on while I clean up and prepare the PR.

@biojppm thanks for your nice insights into this topic - this looks really good.

overall I must confess that I am pretty overwhelmed by the whole setup process to package the wheel for rapidyaml. Therefore, I did a lot of debugging with the help of ChatGPT and other tools.

After a lot of online research I found this article https://github.com/swig/swig/issues/2637

I tested it out locally with the following change

diff --git a/api/CMakeLists.txt b/api/CMakeLists.txt
index 4777e7c..9b81111 100644
--- a/api/CMakeLists.txt
+++ b/api/CMakeLists.txt
@@ -100,6 +100,9 @@ if(RYML_BUILD_API_PYTHON)
         ARCHIVE_OUTPUT_DIRECTORY "${r}/lib"
         LIBRARY_OUTPUT_DIRECTORY "${r}"
         RUNTIME_OUTPUT_DIRECTORY "${r}")
+    if(APPLE)
+        set_property(TARGET ${t} APPEND PROPERTY LINK_FLAGS "-flat_namespace -undefined suppress")
+    endif()
     if(WIN32)
         target_compile_definitions(${t} PUBLIC __WIN32__)
     endif()

And somehow this fixed the problem. At import I no longer get a segfault but the following error regarding the version

python -c "import ryml"                                                             
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/david/miniconda3/envs/_tmp/lib/python3.12/site-packages/ryml/__init__.py", line 2, in <module>
    from .version import *
ModuleNotFoundError: No module named 'ryml.version'

This is probably connected to my weird version hack to build the wheel.

Overall I do not really know why this now works (I guess?)

Let me know what you think

EDIT: it works with set_property(TARGET ${t} APPEND PROPERTY LINK_FLAGS "-flat_namespace") as well. Here https://stackoverflow.com/questions/32277351/what-does-expected-in-flat-namespace-mean is more information on that

I am really curious about your opinion πŸ˜…

@biojppm thanks a lot for the release!

because the wheels are only published for macOS x86-x64 the install does not work out of the box on recent ARM/M-series processors via

pip install rapidyaml

but with

conda install -c conda-forge swig
pip install git+https://github.com/biojppm/rapidyaml

it now works flawlessly with v0.9.0 πŸŽ‰

I looked through the failing pipelines and was not able to locate the problems with wheel packaging on macOS 14/15/latest (which uses ARM based systems)?

For now this approach works fine but it would be for sure nice to also have the correct ARM based wheels on pypi. If necessary I could open a dedicated issue to also package for ARM and if there’s a problem with that I would also be available look into any problems at any time:)

Release should be made within a couple of weeks.

Thanks for your hints @biojppm - the wheel built without a problem but the segfault is still present at import. I will investigate further