arrow: [Python] Failed to build pyarrow using python:3.10-alpine docker image
Describe the bug, including details regarding any error messages, version, and platform.
Bumped into error when the build stage reach pyarrow building:
-- Found Python3Alt: /usr/local/bin/python
#0 678.5 CMake Error at CMakeLists.txt:268 (find_package):
#0 678.5 By not providing "FindArrow.cmake" in CMAKE_MODULE_PATH this project has
#0 678.5 asked CMake to find a package configuration file provided by "Arrow", but
#0 678.5 CMake did not find one.
#0 678.5
#0 678.5 Could not find a package configuration file provided by "Arrow" with any of
#0 678.5 the following names:
#0 678.5
#0 678.5 ArrowConfig.cmake
#0 678.5 arrow-config.cmake
#0 678.5
#0 678.5 Add the installation prefix of "Arrow" to CMAKE_PREFIX_PATH or set
#0 678.5 "Arrow_DIR" to a directory containing one of the above files. If "Arrow"
#0 678.5 provides a separate development package or SDK, be sure it has been
#0 678.5 installed.
#0 678.5
#0 678.5
#0 678.5 -- Configuring incomplete, errors occurred!
#0 678.5 error: command '/usr/bin/cmake' failed with exit code 1
#0 678.5 [end of output]
#0 678.5
#0 678.5 note: This error originates from a subprocess, and is likely not a problem with pip.
#0 678.5 ERROR: Failed building wheel for pyarrow
Component(s)
Python
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 3
- Comments: 16 (6 by maintainers)
Commits related to this issue
- feat: switch to debian based build When trying to upgrade arcgis to latest, the version of pyarrow that it requires was not building properly like before, and the version of pyarrow it requires doesn... — committed to GeoNet/base-images by CallumNZ 2 years ago
- feat: switch to debian based build When trying to upgrade arcgis to latest, the version of pyarrow that it requires was not building properly like before, and the version of pyarrow it requires doesn... — committed to GeoNet/base-images by CallumNZ 2 years ago
- feat: switch to debian based build When trying to upgrade arcgis to latest, the version of pyarrow that it requires was not building properly like before, and the version of pyarrow it requires doesn... — committed to GeoNet/base-images by CallumNZ 2 years ago
I ran into the same issues, but have managed to build pyarrow with the Python alpine image with the following example:
@numbworks If you had checked out the issue you would know that we are not against providing musl wheels, someone just has to do the work.
Hey, cool thing about this being an open source project: you are part of the team! So if this is something you would like to see the fastest way to get it is to contribute to a solution.
@assignUser Hey Jacob, thank you for your answer, but it seems from the thread that your solution doesn’t work.
Do you have a Dockerfile example that demonstrates that your proposal works?
I also read the following answer from you in another thread:
Are there plan to change this? Because, I don’t know if you are aware of it, but PyArrow will be a mandatory dependency for Pandas starting Pandas v3.0.0 - please read more here: https://pandas.pydata.org/pdeps/0010-required-pyarrow-dependency.html.
One of the official Python images on Docker Hub is based on Alpine Linux, which it’s also the more optimized on a resources perspective. The lack of PyArrow wheels for Alpine means that, starting Pandas 3.0.0 (maybe in six months from now), thousands of Python developers and data scientists won’t be able to do their work in a containerized environment.
The only alternative at the moment is to use the Debian-based image on Python’s Docker Hub, which it’s 15x more resource hungry than Alpine:
I hope you can discuss this matter within the team and assign the right priority to it.
Thank you.
Hello guys,
I have the same issue, but from the thread is not clear to me how to solve it.
How should I change the Dockerfile in order to make
pip install pyarrowsucceed?Thank you for the help!
Dockerfile:
Output:
Closing as we have a confirmed fix/workaround, thanks @timetheoretical !
I had the same issue with a dockerfile, this worked smoothly.
Thank you. Will give it a try