catboost: GitHub actions Build fails on macOS with python 3.8 - catboost 1.2
Problem:
Since the 1.2 update, installing catboost on "macos-latest" GitHub Actions runners fails. This affects a number of projects that I work on that have catboost as a dependency.
The build failure can be found in this GitHub workflow.
As of version 1.1.1 this issue was not happening, so I’m guessing it’s related to the latest build https://pypi.org/project/catboost/1.2/#files
Expand to see build logs
Build logs as taken directly from the github runner are shown here. See also the GitHub workflow.
Requirement already satisfied: pip in /Users/runner/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages (23.1.1)
Collecting pip
Using cached pip-23.1.2-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 23.1.1
Uninstalling pip-23.1.1:
Successfully uninstalled pip-23.1.1
Successfully installed pip-23.1.2
Collecting catboost>=1.2
Downloading catboost-1.2.tar.gz (95.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 95.9/95.9 MB 20.8 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting graphviz (from catboost>=1.2)
Downloading graphviz-0.20.1-py3-none-any.whl (47 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 47.0/47.0 kB 7.2 MB/s eta 0:00:00
Collecting matplotlib (from catboost>=1.2)
Using cached matplotlib-3.7.1-cp38-cp38-macosx_10_12_x86_64.whl (7.4 MB)
Collecting numpy>=1.16.0 (from catboost>=1.2)
Using cached numpy-1.24.3-cp38-cp38-macosx_10_9_x86_64.whl (19.8 MB)
Collecting pandas>=0.24 (from catboost>=1.2)
Using cached pandas-2.0.1-cp38-cp38-macosx_10_9_x86_64.whl (11.6 MB)
Collecting scipy (from catboost>=1.2)
Using cached scipy-1.10.1-cp38-cp38-macosx_10_9_x86_64.whl (35.0 MB)
Collecting plotly (from catboost>=1.2)
Downloading plotly-5.14.1-py2.py3-none-any.whl (15.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.3/15.3 MB 41.6 MB/s eta 0:00:00
Collecting six (from catboost>=1.2)
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting python-dateutil>=2.8.2 (from pandas>=0.24->catboost>=1.2)
Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pytz>=2020.1 (from pandas>=0.24->catboost>=1.2)
Using cached pytz-2023.3-py2.py3-none-any.whl (502 kB)
Collecting tzdata>=2022.1 (from pandas>=0.24->catboost>=1.2)
Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 341.8/341.8 kB 29.1 MB/s eta 0:00:00
Collecting contourpy>=1.0.1 (from matplotlib->catboost>=1.2)
Downloading contourpy-1.0.7-cp38-cp38-macosx_10_9_x86_64.whl (243 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 244.0/244.0 kB 21.8 MB/s eta 0:00:00
CollectBuilding cycler>=0.10 (from matplotlib->catboost>=1.2)
Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting fonttools>=4.22.0 (from matplotlib->catboost>=1.2)
Downloading fonttools-4.39.3-py3-none-any.whl (1.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 37.3 MB/s eta 0:00:00
Collecting kiwisolver>=1.0.1 (from matplotlib->catboost>=1.2)
Downloading kiwisolver-1.4.4-cp38-cp38-macosx_10_9_x86_64.whl (65 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.4/65.4 kB 9.1 MB/s eta 0:00:00
Collecting packaging>=20.0 (from matplotlib->catboost>=1.2)
Using cached packaging-23.1-py3-none-any.whl (48 kB)
Collecting pillow>=6.2.0 (from matplotlib->catboost>=1.2)
Downloading Pillow-9.5.0-cp38-cp38-macosx_10_10_x86_64.whl (3.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 25.8 MB/s eta 0:00:00
Collecting pyparsing>=2.3.1 (from matplotlib->catboost>=1.2)
Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.3/98.3 kB 14.3 MB/s eta 0:00:00
Collecting importlib-resources>=3.2.0 (from matplotlib->catboost>=1.2)
Using cached importlib_resources-5.12.0-py3-none-any.whl (36 kB)
Collecting tenacity>=6.2.0 (from plotly->catboost>=1.2)
Downloading tenacity-8.2.2-py3-none-any.whl (24 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=3.2.0->matplotlib->catboost>=1.2)
Using cached zipp-3.15.0-py3-none-any.whl (6.8 kB)
Building wheels for collected packages: catboost
Building wheel for catboost (pyproject.toml): started
Building wheel for catboost (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Building wheel for catboost (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [96 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.15-x86_64-cpython-38
creating build/lib.macosx-10.15-x86_64-cpython-38/catboost
copying catboost/monoforest.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost
copying catboost/plot_helpers.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost
copying catboost/metrics.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost
copying catboost/version.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost
copying catboost/text_processing.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost
copying catboost/datasets.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost
copying catboost/__init__.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost
copying catboost/core.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost
copying catboost/utils.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost
copying catboost/dev_utils.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost
creating build/lib.macosx-10.15-x86_64-cpython-38/catboost/widget
copying catboost/widget/__init__.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/widget
copying catboost/widget/metrics_plotter.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/widget
copying catboost/widget/ipythonwidget.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/widget
copying catboost/widget/callbacks.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/widget
creating build/lib.macosx-10.15-x86_64-cpython-38/catboost/eval
copying catboost/eval/catboost_evaluation.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/eval
copying catboost/eval/_fold_model.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/eval
copying catboost/eval/_readers.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/eval
copying catboost/eval/log_config.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/eval
copying catboost/eval/_splitter.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/eval
copying catboost/eval/__init__.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/eval
copying catboost/eval/execution_case.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/eval
copying catboost/eval/_fold_storage.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/eval
copying catboost/eval/factor_utils.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/eval
copying catboost/eval/utils.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/eval
copying catboost/eval/evaluation_result.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/eval
copying catboost/eval/_fold_models_handler.py -> build/lib.macosx-10.15-x86_64-cpython-38/catboost/eval
running build_ext
Buildling _catboost with cmake and ninja
target_platform=darwin-x86_64. Building targets _catboost with PIC
Running "cmake /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-install-rik3hogp/catboost_edaea41cb61849f5a85631e37874fcd2/catboost_all_src -B /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-install-rik3hogp/catboost_edaea41cb61849f5a85631e37874fcd2/build/temp.macosx-10.15-x86_64-cpython-38 -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-install-rik3hogp/catboost_edaea41cb61849f5a85631e37874fcd2/catboost_all_src/build/toolchains/clang.toolchain --log-level=VERBOSE -DCMAKE_POSITION_INDEPENDENT_CODE=On -DCATBOOST_COMPONENTS=python-package -DCMAKE_OSX_DEPLOYMENT_TARGET=11.0 -DHAVE_CUDA=no -DPython3_ROOT_DIR=/Users/runner/hostedtoolcache/Python/3.8.16/x64"
CMake Error: CMake was unable to find a build program corresponding to "Ninja". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool.
CMake Error: CMAKE_ASM_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
Traceback (most recent call last):
File "/Users/runner/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/Users/runner/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/Users/runner/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 416, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
self.run_setup()
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 487, in run_setup
super(_BuildMetaLegacyBackend,
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 338, in run_setup
exec(code, locals())
File "<string>", line 723, in <module>
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 107, in setup
return distutils.core.setup(**attrs)
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "<string>", line 389, in run
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 343, in run
self.run_command("build")
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "<string>", line 324, in run
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run
self.run_command(cmd_name)
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-b27d888_/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "<string>", line 436, in run
File "<string>", line 454, in build_with_cmake_and_ninja
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-install-rik3hogp/catboost_edaea41cb61849f5a85631e37874fcd2/catboost_all_src/build/build_native.py", line 517, in build
cmd_runner.run(cmake_cmd, env=build_environ)
File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-install-rik3hogp/catboost_edaea41cb61849f5a85631e37874fcd2/catboost_all_src/build/build_native.py", line 164, in run
subprocess.run(cmd, check=True, **subprocess_run_kwargs)
File "/Users/runner/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cmake', '/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-install-rik3hogp/catboost_edaea41cb61849f5a85631e37874fcd2/catboost_all_src', '-B', '/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-install-rik3hogp/catboost_edaea41cb61849f5a85631e37874fcd2/build/temp.macosx-10.15-x86_64-cpython-38', '-G', 'Ninja', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_TOOLCHAIN_FILE=/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-install-rik3hogp/catboost_edaea41cb61849f5a85631e37874fcd2/catboost_all_src/build/toolchains/clang.toolchain', '--log-level=VERBOSE', '-DCMAKE_POSITION_INDEPENDENT_CODE=On', '-DCATBOOST_COMPONENTS=python-package', '-DCMAKE_OSX_DEPLOYMENT_TARGET=11.0', '-DHAVE_CUDA=no', '-DPython3_ROOT_DIR=/Users/runner/hostedtoolcache/Python/3.8.16/x64']' returned non-zero exit status 1.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for catboost
ERROR: Could not build wheels for catboost, which is required to install pyproject.toml-based projects
Failed to build catboost
catboost version: 1.2
Operating System: macOS 12 Monterey, at least according to About GitHub-hosted runners
CPU: 3-core CPU (x86_64), according to About GitHub-hosted runners
GPU: None
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 16 (4 by maintainers)
Commits related to this issue
- Update check-test-release.yml https://github.com/catboost/catboost/issues/2371 — committed to iterative/mlem by aguschin a year ago
- Fix Macos tests (#675) the problem described in https://github.com/catboost/catboost/issues/2371 — committed to iterative/mlem by aguschin a year ago
- Fix catboost installation error for Github macos runners - https://github.com/autogluon/autogluon/pull/3190#issuecomment-1540599280 - https://github.com/catboost/catboost/issues/2371 - https://gith... — committed to ddelange/autogluon by ddelange a year ago
- pin catboost version; see https://github.com/catboost/catboost/issues/2371 — committed to mlfoundations/tableshift by jpgard a year ago
So this is definitively a “won’t fix” on GitHub side.
This means that
catboost>=1.2 & macos-latest & py<=3.8would remain broken in all Github CI setups of all downstream projects.@andrey-khropov could the catboost team once more consider backwards compatible wheels? I think it’ll save many a dev the debug journey!
The good news is, with the release of the new python versions, this was auto-fixed (as expected).
if your pipeline didn’t auto-update to the latest version yet, you can force this via “check-latest: true” in the action setup
having a
macosx_10_9_x86_64wheel on pypi does not mean catboost explicitly supports osx 10 per se, from a python packaging perspective this wheel (even if it’s built on osx 11) is simply backward compatible by designwould it be possible for catboost to build separate wheels for intel and arm macs?
the cibuildwheel defaults will generate the following wheels:
you will be able to reach mostly any mac user with these two without tradeoff.
in contrast, universal wheels are essentially duplicating binaries (x86 + arm64) to maximize portability (packaged in a mac app for instance), but I think it also sacrifices on device compatibility and increases local storage usage.
No, not looked at this issue on my end yet, for now I’m setting the dependency to
"catboost<1.2". Not sure whymacosx-10.15is being detected on the GitHub runner. Do post here if you find a solution.This issue is two-fold.
For some reason you installation uses
macosx-10.15as a python platform tag. As macOS 10.15 reached its end-of-life minimal macOS version for CatBoost wheels on PyPI has been bumped to 11.0 with the CatBoost release 1.2. So if you fix your platform tag wheels from PyPI should work.As a compatible wheel has not been found now python tries to build from source (as it is now finally supported - #830). But fails because it requires ninja to be available from the command line. So for successful build from source it must be added to the build environment.