pyzmq: BUG: Error compiling wheels using crossenv

This is a pyzmq bug

  • This is a pyzmq-specific bug, not an issue of zmq socket behavior. Don’t worry if you’re not sure! We’ll figure it out together.

What pyzmq version?

25.1.2

What libzmq version?

4.3.5

Python version (and how it was installed)

python 3.11.8, cross compiled using ndk

OS

Host - Ubuntu

What happened?

I’m trying to cross compile pyzmq for Android, using crossenv, I’m running :

ZMQ_PREFIX=/home/rudra/Documents/libzmq-builds/arm64 LDFLAGS+="-L../lib/ -lpython3.11" CFLAGS="-I../include/python3.11" python setup.py bdist_wheel

Output :

(cross) rudra@rudra-1-2:~/Documents/compilation/aarch64/pyzmq$ ZMQ_PREFIX=/home/rudra/Documents/libzmq-builds/arm64 LDFLAGS+=“-L…/lib/ -lpython3.11” CFLAGS=“-I…/include/python3.11” python setup.py bdist_wheel /home/rudra/Documents/compilation/aarch64/venv/build/lib/python3.11/site-packages/setuptools/_distutils/dist.py:265: UserWarning: Unknown distribution option: ‘cffi_modules’ warnings.warn(msg) running bdist_wheel running build running build_py creating build creating build/lib.linux-aarch64-cpython-311 creating build/lib.linux-aarch64-cpython-311/zmq copying zmq/constants.py -> build/lib.linux-aarch64-cpython-311/zmq copying zmq/error.py -> build/lib.linux-aarch64-cpython-311/zmq copying zmq/decorators.py -> build/lib.linux-aarch64-cpython-311/zmq copying zmq/_typing.py -> build/lib.linux-aarch64-cpython-311/zmq copying zmq/init.py -> build/lib.linux-aarch64-cpython-311/zmq copying zmq/_future.py -> build/lib.linux-aarch64-cpython-311/zmq copying zmq/asyncio.py -> build/lib.linux-aarch64-cpython-311/zmq creating build/lib.linux-aarch64-cpython-311/zmq/green copying zmq/green/poll.py -> build/lib.linux-aarch64-cpython-311/zmq/green copying zmq/green/device.py -> build/lib.linux-aarch64-cpython-311/zmq/green copying zmq/green/core.py -> build/lib.linux-aarch64-cpython-311/zmq/green copying zmq/green/init.py -> build/lib.linux-aarch64-cpython-311/zmq/green creating build/lib.linux-aarch64-cpython-311/zmq/green/eventloop copying zmq/green/eventloop/zmqstream.py -> build/lib.linux-aarch64-cpython-311/zmq/green/eventloop copying zmq/green/eventloop/ioloop.py -> build/lib.linux-aarch64-cpython-311/zmq/green/eventloop copying zmq/green/eventloop/init.py -> build/lib.linux-aarch64-cpython-311/zmq/green/eventloop creating build/lib.linux-aarch64-cpython-311/zmq/devices copying zmq/devices/proxydevice.py -> build/lib.linux-aarch64-cpython-311/zmq/devices copying zmq/devices/basedevice.py -> build/lib.linux-aarch64-cpython-311/zmq/devices copying zmq/devices/monitoredqueue.py -> build/lib.linux-aarch64-cpython-311/zmq/devices copying zmq/devices/proxysteerabledevice.py -> build/lib.linux-aarch64-cpython-311/zmq/devices copying zmq/devices/monitoredqueuedevice.py -> build/lib.linux-aarch64-cpython-311/zmq/devices copying zmq/devices/init.py -> build/lib.linux-aarch64-cpython-311/zmq/devices creating build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/interop.py -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/monitor.py -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/z85.py -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/garbage.py -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/strtypes.py -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/jsonapi.py -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/win32.py -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/init.py -> build/lib.linux-aarch64-cpython-311/zmq/utils creating build/lib.linux-aarch64-cpython-311/zmq/sugar copying zmq/sugar/context.py -> build/lib.linux-aarch64-cpython-311/zmq/sugar copying zmq/sugar/poll.py -> build/lib.linux-aarch64-cpython-311/zmq/sugar copying zmq/sugar/tracker.py -> build/lib.linux-aarch64-cpython-311/zmq/sugar copying zmq/sugar/version.py -> build/lib.linux-aarch64-cpython-311/zmq/sugar copying zmq/sugar/frame.py -> build/lib.linux-aarch64-cpython-311/zmq/sugar copying zmq/sugar/socket.py -> build/lib.linux-aarch64-cpython-311/zmq/sugar copying zmq/sugar/init.py -> build/lib.linux-aarch64-cpython-311/zmq/sugar copying zmq/sugar/attrsettr.py -> build/lib.linux-aarch64-cpython-311/zmq/sugar copying zmq/sugar/stopwatch.py -> build/lib.linux-aarch64-cpython-311/zmq/sugar creating build/lib.linux-aarch64-cpython-311/zmq/ssh copying zmq/ssh/tunnel.py -> build/lib.linux-aarch64-cpython-311/zmq/ssh copying zmq/ssh/forward.py -> build/lib.linux-aarch64-cpython-311/zmq/ssh copying zmq/ssh/init.py -> build/lib.linux-aarch64-cpython-311/zmq/ssh creating build/lib.linux-aarch64-cpython-311/zmq/log copying zmq/log/handlers.py -> build/lib.linux-aarch64-cpython-311/zmq/log copying zmq/log/main.py -> build/lib.linux-aarch64-cpython-311/zmq/log copying zmq/log/init.py -> build/lib.linux-aarch64-cpython-311/zmq/log creating build/lib.linux-aarch64-cpython-311/zmq/auth copying zmq/auth/base.py -> build/lib.linux-aarch64-cpython-311/zmq/auth copying zmq/auth/certs.py -> build/lib.linux-aarch64-cpython-311/zmq/auth copying zmq/auth/thread.py -> build/lib.linux-aarch64-cpython-311/zmq/auth copying zmq/auth/ioloop.py -> build/lib.linux-aarch64-cpython-311/zmq/auth copying zmq/auth/init.py -> build/lib.linux-aarch64-cpython-311/zmq/auth copying zmq/auth/asyncio.py -> build/lib.linux-aarch64-cpython-311/zmq/auth creating build/lib.linux-aarch64-cpython-311/zmq/backend copying zmq/backend/select.py -> build/lib.linux-aarch64-cpython-311/zmq/backend copying zmq/backend/init.py -> build/lib.linux-aarch64-cpython-311/zmq/backend creating build/lib.linux-aarch64-cpython-311/zmq/backend/cffi copying zmq/backend/cffi/context.py -> build/lib.linux-aarch64-cpython-311/zmq/backend/cffi copying zmq/backend/cffi/utils.py -> build/lib.linux-aarch64-cpython-311/zmq/backend/cffi copying zmq/backend/cffi/error.py -> build/lib.linux-aarch64-cpython-311/zmq/backend/cffi copying zmq/backend/cffi/socket.py -> build/lib.linux-aarch64-cpython-311/zmq/backend/cffi copying zmq/backend/cffi/message.py -> build/lib.linux-aarch64-cpython-311/zmq/backend/cffi copying zmq/backend/cffi/_poll.py -> build/lib.linux-aarch64-cpython-311/zmq/backend/cffi copying zmq/backend/cffi/devices.py -> build/lib.linux-aarch64-cpython-311/zmq/backend/cffi copying zmq/backend/cffi/init.py -> build/lib.linux-aarch64-cpython-311/zmq/backend/cffi creating build/lib.linux-aarch64-cpython-311/zmq/backend/cython copying zmq/backend/cython/init.py -> build/lib.linux-aarch64-cpython-311/zmq/backend/cython creating build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_decorators.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_monitor.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_monqueue.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_z85.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_asyncio.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_cython.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_draft.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_pubsub.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_mypy.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_cffi_backend.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_ext.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_socket.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_pair.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_context.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_error.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_includes.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_ssh.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_poll.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_proxy_steerable.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_multipart.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_log.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/conftest.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_message.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_imports.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_future.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_ioloop.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_zmqstream.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_retry_eintr.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_device.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_security.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_etc.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_constants.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_win32_shim.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_reqrep.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_version.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/test_auth.py -> build/lib.linux-aarch64-cpython-311/zmq/tests copying zmq/tests/init.py -> build/lib.linux-aarch64-cpython-311/zmq/tests creating build/lib.linux-aarch64-cpython-311/zmq/eventloop copying zmq/eventloop/_deprecated.py -> build/lib.linux-aarch64-cpython-311/zmq/eventloop copying zmq/eventloop/zmqstream.py -> build/lib.linux-aarch64-cpython-311/zmq/eventloop copying zmq/eventloop/future.py -> build/lib.linux-aarch64-cpython-311/zmq/eventloop copying zmq/eventloop/ioloop.py -> build/lib.linux-aarch64-cpython-311/zmq/eventloop copying zmq/eventloop/init.py -> build/lib.linux-aarch64-cpython-311/zmq/eventloop copying zmq/init.pxd -> build/lib.linux-aarch64-cpython-311/zmq copying zmq/init.pyi -> build/lib.linux-aarch64-cpython-311/zmq copying zmq/py.typed -> build/lib.linux-aarch64-cpython-311/zmq copying zmq/devices/monitoredqueue.pxd -> build/lib.linux-aarch64-cpython-311/zmq/devices copying zmq/utils/buffers.pxd -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/pyversion_compat.h -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/ipcmaxlen.h -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/mutex.h -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/getpid_compat.h -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/zmq_compat.h -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/config.json -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/utils/compiler.json -> build/lib.linux-aarch64-cpython-311/zmq/utils copying zmq/sugar/init.pyi -> build/lib.linux-aarch64-cpython-311/zmq/sugar copying zmq/backend/init.pyi -> build/lib.linux-aarch64-cpython-311/zmq/backend copying zmq/backend/cffi/_cdefs.h -> build/lib.linux-aarch64-cpython-311/zmq/backend/cffi copying zmq/backend/cython/message.pxd -> build/lib.linux-aarch64-cpython-311/zmq/backend/cython copying zmq/backend/cython/socket.pxd -> build/lib.linux-aarch64-cpython-311/zmq/backend/cython copying zmq/backend/cython/checkrc.pxd -> build/lib.linux-aarch64-cpython-311/zmq/backend/cython copying zmq/backend/cython/libzmq.pxd -> build/lib.linux-aarch64-cpython-311/zmq/backend/cython copying zmq/backend/cython/init.pxd -> build/lib.linux-aarch64-cpython-311/zmq/backend/cython copying zmq/backend/cython/context.pxd -> build/lib.linux-aarch64-cpython-311/zmq/backend/cython copying zmq/backend/cython/constant_enums.pxi -> build/lib.linux-aarch64-cpython-311/zmq/backend/cython copying zmq/tests/cython_ext.pyx -> build/lib.linux-aarch64-cpython-311/zmq/tests running build_ext running configure {‘libraries’: [‘zmq’], ‘include_dirs’: [‘/home/rudra/Documents/libzmq-builds/arm64/include’], ‘library_dirs’: [‘/home/rudra/Documents/libzmq-builds/arm64/lib’], ‘runtime_library_dirs’: [‘/home/rudra/Documents/libzmq-builds/arm64/lib’], ‘extra_link_args’: []}


Configure: Autodetecting ZMQ settings… Custom ZMQ dir: /home/rudra/Documents/libzmq-builds/arm64 Checking for timer_create ** Errors about missing timer_create are a normal part of this process ** /android-ndk-r26c/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android24-clang -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I…/include/python3.11 -I/home/rudra/Documents/compilation/aarch64/venv/cross/include -I/usr/include/python3.11 -c /tmp/timer_createfylxmpzs.c -o tmp/timer_createfylxmpzs.o /android-ndk-r26c/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android24-clang tmp/timer_createfylxmpzs.o -L/usr/lib -o a.out /android-ndk-r26c/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android24-clang -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I…/include/python3.11 -I/home/rudra/Documents/libzmq-builds/arm64/include -Izmq/utils -c build/temp.linux-aarch64-cpython-311/scratch/vers.c -o build/temp.linux-aarch64-cpython-311/scratch/vers.o /android-ndk-r26c/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android24-clang build/temp.linux-aarch64-cpython-311/scratch/vers.o -L/home/rudra/Documents/libzmq-builds/arm64/lib -Wl,–enable-new-dtags,-R/home/rudra/Documents/libzmq-builds/arm64/lib -lzmq -o build/temp.linux-aarch64-cpython-311/scratch/vers Error running version detection script:

Failed to run [‘build/temp.linux-aarch64-cpython-311/scratch/vers’]: OSError(8, ‘Exec format error’)

error: Error running version detection script:

Failed to run [‘build/temp.linux-aarch64-cpython-311/scratch/vers’]: OSError(8, ‘Exec format error’)

Fatal: Falling back on bundled libzmq, but config has explicitly prohibited building the libzmq extension.

Code to reproduce bug

No response

Traceback, if applicable

No response

More info

No response

About this issue

  • Original URL
  • State: open
  • Created 4 months ago
  • Comments: 19 (9 by maintainers)

Most upvoted comments

I still don’t know how to setup (or test) a build for android, but I have managed to build the latest pyzmq prerelease with crossenv. In my case, I cross-compiled from aarch64 to x86_64, and it built a wheel that I could then install and use in an x86_64 container (python:3.11).

I used this Dockerfile.

It seems with $CC and $CXX defined, it works. I either needed to build libsodium and libzmq with --host $HOST before linking from pyzmq, or build static libzmq & libsodium by specifying PYZMQ_LIBSODIUM_CONFIGURE_ARGS="--host $HOST".

I initially thought pyzmq should be able to do this automatically, but it doesn’t seem to know that it’s cross compiling even though it is, so it’s hard to set the host triplet. Setting it directly works, though.

It’s unclear how much can be expected to work for cross compiles without any help, but at least I now know for sure that building with crossenv does work. We just need to understand what’s going on with your android env.