python: Python images (At least bookwork-slim) relased today break installation of sdist packages
Following the discussion her https://github.com/pypa/wheel/issues/660#issuecomment-2788333871 - images relesed today started to break CI - Airflow, but also other people have similar issue.
I explained it in the issue but let me copy it here as well:
Example failure: https://github.com/apache/airflow/actions/runs/14347343802/job/40220981486#step:6:809
6 13.37 Building wheels for collected packages: pyspark
docker-library/official-images#6 13.37 Building wheel for pyspark (setup.py): started
docker-library/official-images#6 14.11 Building wheel for pyspark (setup.py): finished with status 'error'
docker-library/official-images#6 14.11 error: subprocess-exited-with-error
docker-library/official-images#6 14.11
docker-library/official-images#6 14.11 × python setup.py bdist_wheel did not run successfully.
docker-library/official-images#6 14.11 │ exit code: 1
docker-library/official-images#6 14.11 ╰─> [8 lines of output]
docker-library/official-images#6 14.11 /home/airflow/.local/lib/python3.9/site-packages/setuptools/dist.py:717: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
docker-library/official-images#6 14.11 warnings.warn(
docker-library/official-images#6 14.11 usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
docker-library/official-images#6 14.11 or: setup.py --help [cmd1 cmd2 ...]
docker-library/official-images#6 14.11 or: setup.py --help-commands
docker-library/official-images#6 14.11 or: setup.py cmd --help
docker-library/official-images#6 14.11
docker-library/official-images#6 14.11 error: invalid command 'bdist_wheel'
docker-library/official-images#6 14.11 [end of output]
docker-library/official-images#6 14.11
docker-library/official-images#6 14.11 note: This error originates from a subprocess, and is likely not a problem with pip.
docker-library/official-images#6 14.11 ERROR: Failed building wheel for pyspark
docker-library/official-images#6 14.12 Running setup.py clean for pyspark
And after some (non-obvious investigation I found out what the root cause is and that’s somethign you might want to talk to the maintainers of official Python images in dockerhub, because probably a lot of people using them will have similar issue after 0.46+ wheel that does not contain bdist_wheel any more and rely on setuptools > 70.1.
The problem is that some - latest - official python images available in dockerhub are causing this problem to happen.
If you look - for example - at latest Python debian bookworm slim images you will see this:
[jarek:~/code/airflow] main ± docker run -it python:3.9-slim-bookworm bash
root@fb321ef4723e:/# pip list
Package Version
---------- -------
pip 23.0.1
setuptools 58.1.0
wheel 0.45.1
[notice] A new release of pip is available: 23.0.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip
root@fb321ef4723e:/# exit
exit
[jarek:~/code/airflow] main 14s ± docker run -it python:3.10-slim-bookworm bash
root@2234dee7aba0:/# pip list
Package Version
---------- -------
packaging 24.2
pip 23.0.1
setuptools 65.5.1
wheel 0.46.1
[notice] A new release of pip is available: 23.0.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip
root@2234dee7aba0:/# exit
exit
[jarek:~/code/airflow] main 25s ± docker run -it python:3.11-slim-bookworm bash
root@b62343d129a6:/# pip list
Package Version
---------- -------
packaging 24.2
pip 24.0
setuptools 65.5.1
wheel 0.46.1
[jarek:~/code/airflow] main 1m5s ± docker run -it python:3.12-slim-bookworm bash
root@411574e67a7e:/# pip list
Package Version
------- -------
pip 25.0.1
root@411574e67a7e:/#
As you can see - default version of setuptools in all those images causes problem:
- Python 3.9 -> both wheel and setuptools are old, but if you just upgrade wheel but do not upgrade setuptools, sdist installations will fail
- Python 3.10, 3.11 -> wheel is already updated, but setuptools is < 70.1 - which mean that by default installing somethign from sdist wil fail there - even if you do not touch setuptools and wheel
- Python 3.12 -> all good, no setuptools, no wheel, newest versions of those will be installed
I am going to upgrade setuptools and wheels to latest versions in our images, so we will solve the problem in Airflow, but I think currently released “official images” need to be rebuild and latest setuptools should be installed in all of them.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 18
- Comments: 21 (8 by maintainers)
We’ll be making a PR to pin wheel shortly as that seems to be the least breaking change.
Hi Jarek!
The repo for the Docker Python images is: https://github.com/docker-library/python/issues
(Maybe someone with permissions can use the issue transfer feature to move this issue there? I’m not a maintainer)
Those images intentionally don’t use latest setuptools (since there are breaking changes in newer setuptools), but instead use the same version provided by the
ensurepipcommand (with a slight fixup for65.5.0->65.5.1).The issue is that
ensurepipdoesn’t include wheel (only pip and setuptools; at least until Python 3.12+ where it also drops setuptools), so the Python docker image scripts don’t have a “reference” version of wheel they can use, and instead pull in the latest wheel version here: https://github.com/docker-library/python/blob/f3c69c5acce909a76e287ef1adf8f10c476eb2f0/Dockerfile-linux.template#L299-L304This hasn’t historically caused problems, since wheel hasn’t made many breaking changes. However, with wheel 0.46 dropping support for older setuptools, this is no longer the case.
The options presumably are:
wheel<0.46) so the installed wheel version remains compatible with the older setuptools. This could even just be hardcoded, since for Python 3.12+ the images don’t install either setuptools or wheel, so there won’t be any image combination where wheel 0.46+ needs to be installed.ensurepipbehaviour.)ensurepipbehaviour. This will cause pip to use isolated build environments by default when building sdists, and inside those isolated environments newer setuptools will be used that doesn’t depend on wheel. This presumably would be almost as breaking as (2) from a setuptools perspective, but not quite (since it doesn’t affect the version of setuptools used at run-time, outside of those PEP 517 isolated build environments). However, it would be more breaking for people who use wheel directly.Out of those, (1) seems the safest to me - and I suppose is in keeping with the ~“we don’t upgrade tooling to latest, but generally prefer stability” approach?
cc @tianon @yosifkit (who are the Python docker image maintainers)
For 3.11-slim
RUN pip install --no-cache-dir setuptools==78.1.0 wheel==0.46.1 COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt
Also, not really sure how to log the version that was working previously 😦
Btw the different behaviour for Python 3.9 is temporary - and due to the 3.9.22 release being delayed by https://github.com/python/cpython/issues/132287 - ie: the Python 3.9 images haven’t been rebuilt yet since newer wheel came out. Once https://github.com/docker-library/official-images/pull/18807 is merged the Python 3.9.22 images will have newer wheel too (and so also be broken like the 3.10-3.11 images).
I wish this was the case, but since June 2024,
setuptoolsdoes import and usewheelif it’s installed! A bad decision IMHO.Once this is fixed, it might be worth adding an sdist pip install test - either in this existing test, or in a new one: https://github.com/docker-library/official-images/blob/master/test/tests/python-pip-requests-ssl/container.py
It’s also worth noting, that even if the wheel maintainer were to yank the 0.46.1 wheel release it wouldn’t automatically fix the Python docker images, since they would still need a rebuild (and a manifest bump like https://github.com/docker-library/official-images/pull/18803) to pick up the yank. So the fix needs to occur on the docker images side regardless.
Yep. I updated the description to clarify it.
Not really - wheel version does not matter or it even can be not installed at all if setuptools > 70.1 . Setuptools 70.1 vendored-in the
bdist_wheelcommand so essentially they don’t care what version of wheel is installed. So my proposal is quite different from 1 😃 … Just bump to a fixed setuptools version that is > 70.1If you meant fixing the wheel version too, to match, then yeah I agree - and that’s “option 1” above 😃
The versions of setuptools in the images is already fixed, eg: https://github.com/docker-library/python/blob/f3c69c5acce909a76e287ef1adf8f10c476eb2f0/versions.sh#L158-L188 https://github.com/docker-library/python/blob/f3c69c5acce909a76e287ef1adf8f10c476eb2f0/versions.json#L8-L9