pyzmq: BUG: pyzmq v26.0.0 installation fails on Bullseye (build from sources)
What pyzmq version?**
26.0.0
What libzmq version?**
4.3.5 (seperatly build and installed)
Python version (and how it was installed)
python 3.9 via apt-get
OS
Raspberry Pi OS Bullseye 32-bit / Github CI Docker Debian Bullseye
What happened?
In our projekt we build pyzmq from sources with a seperatly prebuild libzmq installation. Prior to the v26.0.0 release this succeeded on Bullseye and Bookworm. Since the new version release this fails on Bullseye.
Traceback, if applicable
Collecting pyzmq
Downloading pyzmq-26.0.0.tar.gz (266 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 266.6/266.6 kB 4.9 MB/s eta 0:00:00
Installing build dependencies: started
Running command pip subprocess to install build dependencies
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple, https://www.piwheels.org/simple
Ignoring cffi: markers 'implementation_name == "pypy"' don't match your environment
Collecting cython>=3.0.0
Downloading https://www.piwheels.org/simple/cython/Cython-3.0.10-cp39-cp39-linux_armv7l.whl (10.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.7/10.7 MB 3.7 MB/s eta 0:00:00
Collecting packaging
Downloading https://www.piwheels.org/simple/packaging/packaging-24.0-py3-none-any.whl (53 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.5/53.5 kB 5.0 MB/s eta 0:00:00
Collecting scikit-build-core
Downloading https://www.piwheels.org/simple/scikit-build-core/scikit_build_core-0.8.2-py3-none-any.whl (140 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.5/140.5 kB 8.8 MB/s eta 0:00:00
Collecting exceptiongroup (from scikit-build-core)
Downloading https://www.piwheels.org/simple/exceptiongroup/exceptiongroup-1.2.0-py3-none-any.whl (16 kB)
Collecting tomli>=1.1 (from scikit-build-core)
Downloading https://www.piwheels.org/simple/tomli/tomli-2.0.1-py3-none-any.whl (12 kB)
Installing collected packages: tomli, packaging, exceptiongroup, cython, scikit-build-core
Successfully installed cython-3.0.10 exceptiongroup-1.2.0 packaging-24.0 scikit-build-core-0.8.2 tomli-2.0.1
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Running command Getting requirements to build wheel
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Running command pip subprocess to install backend dependencies
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple, https://www.piwheels.org/simple
Collecting pyproject_metadata
Downloading https://www.piwheels.org/simple/pyproject-metadata/pyproject_metadata-0.7.1-py3-none-any.whl (7.4 kB)
Collecting pathspec
Downloading https://www.piwheels.org/simple/pathspec/pathspec-0.12.1-py3-none-any.whl (31 kB)
Collecting cmake>=3.14
Downloading cmake-3.29.2.tar.gz (30 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'error'
error: subprocess-exited-with-error
× pip subprocess to install backend dependencies did not run successfully.
│ exit code: 2
╰─> [55 lines of output]
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple, https://www.piwheels.org/simple, https://www.piwheels.org/simple
Collecting ninja>=1.5
Downloading https://www.piwheels.org/simple/ninja/ninja-1.11.1.1-cp39-cp39-linux_armv7l.whl (132 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 406.0 kB/s eta 0:00:00
Collecting cmake
Using cached cmake-3.29.2.tar.gz (30 kB)
ERROR: Exception:
Traceback (most recent call last):
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
status = run_func(*args)
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
return func(self, options, args)
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 377, in run
requirement_set = resolver.resolve(
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
result = self._result = resolver.resolve(
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
if not criterion.candidates:
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__
return bool(self._sequence)
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
return any(self)
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
return (c for c in iterator if id(c) not in self._incompatible_ids)
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
candidate = func()
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 182, in _make_candidate_from_link
base: Optional[BaseCandidate] = self._make_base_candidate_from_link(
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 228, in _make_base_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 290, in __init__
super().__init__(
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
self.dist = self._prepare()
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 222, in _prepare
dist = self._prepare_distribution()
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 301, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 525, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 640, in _prepare_linked_requirement
dist = _get_prepared_distribution(
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 70, in _get_prepared_distribution
with build_tracker.track(req, tracker_id):
File "/usr/lib/python3.9/contextlib.py", line 117, in __enter__
return next(self.gen)
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/operations/build/build_tracker.py", line 137, in track
self.add(req, tracker_id)
File "/home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_internal/operations/build/build_tracker.py", line 103, in add
raise LookupError(message)
LookupError: https://files.pythonhosted.org/packages/80/bf/4f9a9f754507992be28b985d1e9b17f93a2271106b5916a212efe1d65205/cmake-3.29.2.tar.gz (from https://pypi.org/simple/cmake/) (requires-python:>=3.7) is already being built: cmake>=3.14 from https://files.pythonhosted.org/packages/80/bf/4f9a9f754507992be28b985d1e9b17f93a2271106b5916a212efe1d65205/cmake-3.29.2.tar.gz
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× pip subprocess to install backend dependencies did not run successfully.
│ exit code: 2
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× pip subprocess to install backend dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /home/testuser/RPi-Jukebox-RFID/.venv/bin/python3 /home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/__pip-runner__.py install --ignore-installed --no-user --prefix /tmp/pip-build-env-vnwo_md6/normal --no-warn-script-location --no-binary pyzmq --only-binary :none: -i https://pypi.org/simple --extra-index-url https://www.piwheels.org/simple -- pyproject_metadata pathspec 'cmake>=3.14'
cwd: [inherit]
Installing backend dependencies: finished with status 'error'
error: subprocess-exited-with-error
× pip subprocess to install backend dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
More info
-
libzmq prebuild and copied to “/usr/local”.
-
Build pyzmq ZMQ_PREFIX=“/usr/local” ZMQ_DRAFT_API=1 pip install -v pyzmq --no-binary pyzmq
We had also problems with the prerelease some time ago: https://github.com/zeromq/pyzmq/issues/1937
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 22 (10 by maintainers)
Hello, there seem to be an issue, many builds are missing from https://pypi.org/project/pyzmq/26.0.1/#files , especially amd64 build, which were present in https://pypi.org/project/pyzmq/26.0.0/#files
I believe #1977 fixed the issue preventing the piwheels builds of pyzmq 26, I’ll do 26.0.1 with that tonight or tomorrow.
See this answer.
This is unrelated to pyzmq, but ld itself. ld has a cache, so you may need to run ldconfig to refresh its cache after installing a new library, especially when you already have another version of the same library at a lower priority on ld’s path.
“Custom” prefix usually needs to be combined with setting the runtime path (rpath), e.g
-Wl,-rpath,$ZMQ_PREFIX/lib, as demonstrated here, and shown to be working here.I made a mistake in my suggestion where I specified the prefix (
-Wl,-rpath,$ZMQ_PREFIX), which should have been the library dir (-Wl,-rpath,$ZMQ_PREFIX/lib).It’s arguable whether this should be done by pyzmq by default or not, because it doesn’t seem to be standard practice for other packages to do this, but I did used to do it in setup.py, so I’ve put it back in #1983.
Can you add
and share the output again?
EDIT: added missing
/libon rpathCan you add
sudo ldconfigbetween installing libzmq and building pyzmq? Not sure if it would help, but worth a try.it would also help to have the full build output of pyzmq, just to see what libzmq it is finding, as well as
lddoutput for the .so file in pyzmq.based on the output, it really seems like pyzmq is built with a libzmq with drafts, but a different libzmq is being found at runtime. Check if there is more than one on your system. It may be that you need to add rpath arguments if there’s more than one (ldconfig might also help with this) to ensure that the right one is loaded.
The 26.0.2 builds appeared to work fine on piwheels: https://www.piwheels.org/project/pyzmq/
The CI upload took a while today, should be all there now.