meson-python: Build failures using meson-python 0.11.0 with cibuildwheel
On Windows, building for win32 fails with
Program python found: YES (C:\Users\runneradmin\AppData\Local\Temp\cibw-run-dbnhe2vj\cp37-win32\build\venv\Scripts\python.exe)
Need python for 64-bit, but found 32-bit
..\..\meson.build:24:3: ERROR: Python dependency not found
it may be a cibuildwheel issue or a Meson issue. Full log here https://github.com/dnicolodi/python-siphash24/actions/runs/3517527064/jobs/5895402010
On macOS, building succeeds but test fail. The test pass when the package is not built with cibuildwheel. No idea what is going on as it seems that the test process segfaults. Maybe a linking issue. Full log here https://github.com/dnicolodi/python-siphash24/actions/runs/3517527064/jobs/5895402097
On Linux, linking issue:
FAILED: siphash24.cpython-37m-x86_64-linux-gnu.so
cc -o siphash24.cpython-37m-x86_64-linux-gnu.so siphash24.cpython-37m-x86_64-linux-gnu.so.p/meson-generated_siphash24.pyx.c.o -Wl,--as-needed -Wl,--allow-shlib-undefined -shared -fPIC -Wl,--start-group subprojects/libcsiphash-1/src/libcsiphash-1.a -L/opt/_internal/cpython-3.7.13/lib -lpython3.7m -Wl,--end-group
/opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld: cannot find -lpython3.7m
it may be Python 3.7 specific. Again not seen when building without cibuildwheel, maybe related to the setup used in the manylinux container. Full log here https://github.com/dnicolodi/python-siphash24/actions/runs/3517527064/jobs/5895401917
EDIT: a summary of the actual issue is in https://github.com/mesonbuild/meson-python/issues/222#issuecomment-1323610463, and a Meson issue is linked from https://github.com/mesonbuild/meson-python/issues/222#issuecomment-1324214338. There is also useful discussion on cross-compiling in the comments below.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 58 (58 by maintainers)
Commits related to this issue
- BUG: allow to overwrite macOS platform tag via _PYTHON_HOST_PLATFORM Use sysconfig.get_platform() instead of platform.mac_ver() to base the platform tag computation. The ability to overwrite the val... — committed to dnicolodi/meson-python by dnicolodi 2 years ago
- BUG: allow to overwrite macOS platform tag via _PYTHON_HOST_PLATFORM Use sysconfig.get_platform() instead of platform.mac_ver() to base the platform tag computation. The ability to overwrite the val... — committed to dnicolodi/meson-python by dnicolodi 2 years ago
- BUG: allow to overwrite macOS platform tag via _PYTHON_HOST_PLATFORM Use sysconfig.get_platform() instead of platform.mac_ver() to base the platform tag computation. The ability to overwrite the val... — committed to dnicolodi/meson-python by dnicolodi 2 years ago
- BUG: allow to overwrite macOS platform tag via _PYTHON_HOST_PLATFORM Use sysconfig.get_platform() instead of platform.mac_ver() to base the platform tag computation. The ability to overwrite the val... — committed to dnicolodi/meson-python by dnicolodi 2 years ago
- BUG: allow to overwrite macOS platform tag via _PYTHON_HOST_PLATFORM Use sysconfig.get_platform() instead of platform.mac_ver() to base the platform tag computation. The ability to overwrite the val... — committed to dnicolodi/meson-python by dnicolodi 2 years ago
- BUG: allow to overwrite macOS platform tag via _PYTHON_HOST_PLATFORM Use sysconfig.get_platform() instead of platform.mac_ver() to base the platform tag computation. The ability to overwrite the val... — committed to dnicolodi/meson-python by dnicolodi 2 years ago
- BUG: allow to overwrite macOS platform tag via _PYTHON_HOST_PLATFORM Use sysconfig.get_platform() instead of platform.mac_ver() to base the platform tag computation. The ability to overwrite the val... — committed to dnicolodi/meson-python by dnicolodi 2 years ago
- ENH: allow to overwrite macOS platform tag via _PYTHON_HOST_PLATFORM Use sysconfig.get_platform() instead of platform.mac_ver() to base the platform tag computation. The ability to overwrite the val... — committed to dnicolodi/meson-python by dnicolodi 2 years ago
- ENH: allow to overwrite macOS platform tag via _PYTHON_HOST_PLATFORM Use sysconfig.get_platform() instead of platform.mac_ver() to base the platform tag computation. The ability to overwrite the val... — committed to dnicolodi/meson-python by dnicolodi 2 years ago
- ENH: allow to overwrite macOS platform tag via _PYTHON_HOST_PLATFORM Use sysconfig.get_platform() instead of platform.mac_ver() to base the platform tag computation. The ability to overwrite the val... — committed to dnicolodi/meson-python by dnicolodi 2 years ago
- ENH: allow to overwrite macOS platform tag via _PYTHON_HOST_PLATFORM Use sysconfig.get_platform() instead of platform.mac_ver() to base the platform tag computation. The ability to overwrite the val... — committed to dnicolodi/meson-python by dnicolodi 2 years ago
- ENH: allow to overwrite macOS platform tag via _PYTHON_HOST_PLATFORM Use sysconfig.get_platform() instead of platform.mac_ver() to base the platform tag computation. The ability to overwrite the val... — committed to mesonbuild/meson-python by dnicolodi 2 years ago
Side comment: When Meson fails, we should raise SystemExit (like setuptools usually does), not throw an exception. Pip/build report a mess when there’s an uncaught exception from the build hooks, while SystemExit is handled gracefully without nearly as many extra reports about missing build hooks and changed exceptions. The actual failure is printed by Meson.
If both MSVC and the MingGW compilers are on the path, which one should Meson choose, and why? Which one should other build systems choose, and why?
Meson historically relies on distutils for 2 things:
sys.path
has patched distutils to adapt to the patch, but left sysconfig brokenIn both cases, the use of distutils is limited to a probe script that dumps json to stdout describing the capabilities of the found python. And in one of those cases, we were limited by “whatever Debian does”, which is beyond frustrating but there you have it.
We do intend to fix this before python 3.12 final, and ideally as soon as possible for alpha testing, but the main point is that whether or not Meson uses it isn’t something that individual packages like SciPy need to care about.
But “be ready for distutils’ removal” is a complex topic, and the main issue that many meson-python adapters have is less about the removal of distutils from the stdlib, and more about what that means for setuptools’ release velocity, API stability, and the infeasibility of maintaining numpy.distutils. The use of either distutils or setuptools isn’t healthy for large projects especially, because of various limitations which mainly boil down to “use a real, battle-tested build system that supports introspection, parallel builds, custom build rules that aren’t just open-coded in python, etc.” Having an
import distutils
may or may not be a hindrance to these goals, and as it happens, isn’t one.There’s a quick hack to make this work on python 3.12 alpha, simply install setuptools as a build dependency – it includes a copy of distutils, suitable for example for checking whether there’s a debian scheme in it 😄 and which also contains stdlib-compatible logic for determining whether libpython is expected on that OS to be linked to in extensions. This is of course a hack, not a solution, which is why it will not be needed for the final release. I’ve already been looking into what is needed here.