You can use PyYaml 5.3.1 until the issue is resolved.

Affecting us too and our security policy won’t let us downgrade to 5.3 because of pre-5.4 vulnerabilities

But pip install "cython<3.0.0" && pip install --no-build-isolation pyyaml==6.0 did work (as per the linked issue)

You can also upgrade to 6.0.1, which pins the Cython < 3.0.0.

We cannot use PyYAML 5.3 due to dependencies requiring 5.4. On Python 3.10+3.11, using PyYAML 6.0 also works, because it provides wheel archives for these Python versions.

Is there a way to have PyYAML use Cython<3 for its installation?

Cython 3.0 came out since Friday.

If you have a dependency that depends on PyYAML 5.4.1 you can install these before:

pip3 install wheel -v

pip3 install "cython<3.0.0" pyyaml==5.4.1 --no-build-isolation -v

Cython 3 was released 4 hours ago: https://pypi.org/project/Cython/3.0.0/#history

This coincides with when our PyYAML 6.0.0 installs via Poetry in Alpine Linux containers started failing. 😢

A little recap, do correct me if I’m wrong:

If PyYAML is your own dependency or your dependencies support PyYAML~=6

• If you’re on Python 3: bump PyYAML to at least 6.0.1.
• If you need to support Python 2: use PyYAML 5.3.1 (NOT RECOMMENDED due to security issues. Consider updating Python.)

If your problem is related to awscli

• If you can bump awscli to at least 1.29.4, do so.
• If you can’t, see the solution for aws-sam-cli below.

If your problem is related to aws-sam-cli or another package which requires PyYAML < 6

• Before installing your other dependencies: pip install "cython<3.0.0" wheel && pip install pyyaml==5.4.1 --no-build-isolation
• Some users report not needing wheel; that might be dependent on your preinstalled packages; on CI, I needed to include it.
• If you’re still getting errors (for missing commands during installation), try including setuptools alongside wheel.

Given the 5.3.1 work around has CVE: https://github.com/advisories/GHSA-8q59-q68h-6hv4 When will an updated release be available and what version do you anticipate it being?

You can also upgrade to 6.0.1, which pins the Cython < 3.0.0.

Confirm. Upgrading to 6.0.1 helped me too!

This has broken Python 3.12 as well; there aren’t pre-built wheels for 3.12 yet (ABI is now supposed to be stable as of beta 4, so you can add them 😉 )

Setting:

"pyyaml!=6.0.0,!=5.4.0,!=5.4.1", # pyyaml is broken with cython 3

Does work for now on 3.12.

https://stackoverflow.com/questions/77490435/attributeerror-cython-sources/77491847#77491847

The steps outlined in the post worked for me to get docker-compose working on arm64

pip install "cython<3.0.0" wheel
pip install "pyyaml==5.4.1" --no-build-isolation
pip install docker-compose

I just had the same issue with pyyaml 6.0.0

Duplicate of https://github.com/yaml/pyyaml/issues/723?

https://stackoverflow.com/questions/77490435/attributeerror-cython-sources/77491847#77491847

Out of curiosity, any chance of https://github.com/yaml/pyyaml/pull/702 being merged in soon, or should everybody go ahead and implement local workarounds? There are quite a few projects relying on PyYaml to be working…

If you have a dependency that depends on PyYAML 5.4.1 you can install these before:

pip3 install wheel -v

pip3 install "cython<3.0.0" pyyaml==5.4.1 --no-build-isolation -v

That actually fixed the issue for me. Thanks a lot!

Thanks @luabida ! Should we backport https://github.com/yaml/pyyaml/pull/702 for PyYAML>=5.4,<6 for a more permanent workaround/fix?

echo "cython<3" > constraint.txt
export PIP_CONSTRAINT=$PWD/constraint.txt
pip install pyyaml

Tested this on Python 3.10 image, works as expected.

You can use PyYaml 5.3.1 until the issue is resolved.

Please do not use this version. PyYAML version 5.3.1 is associated with CVE-2020-14343 that was fixed in version 5.4.

Instead use 6.0.1

Thanks @olliemath, your command saved the day. It also works with PyYaml < 6.0, and now I can at least move forward with the environment installation: pip install "cython<3.0.0" && pip install --no-build-isolation "pyyaml<6.0"

Freezing pyyaml to 5.3.1 and 6.0.1 solves the issue. I prefer 6.0.1.

but docker-compose require PyYAML < 6. will there be a 5.4.2?

that is one of the reasons we started to work on a rebundle of the docker-compose v2: https://pypi.org/project/compose-go/

cc @luabida

With python version 3.11 you can use PyYAML 5.4.1 and above with no problem, but you might check the dependencies in other libraries

For me, this is still failing with upgrading to PyYAML to 6.0.1 and Cython 3.0.0.

6.0.1 solved this by capping Cython < 3. So you want Cython < 3 until the next release of pyyaml. Use Cython < 3 until a new pyyaml release with a proper fix is out.

and on a personal note, thanks for the ego boost after a crappy couple of weeks. I know you understand, so I really didn’t expect that from you

Apologies if you took anything personal, I didn’t mean anything personal, and the rapidity of response in getting a 6.0.1 (after Cython 3.0 came out) was excellent. I personally didn’t call for any 5.x releases (at least I’m pretty sure I didn’t, don’t know why I would), I was just showing workarounds that could be used to fix those releases. I’m just a bit frustrated when I see the response to “we’ve known this would break for six months” is “let’s cap things we don’t know are going to break and make it impossible for people who know what they are doing to use our software so people who don’t know how to build code won’t bother us”. As you can tell from my blog post, it’s a pet peeve, and not at all personal. 😃

BTW, breakages caused by capping (most breakages above are due to capping pyyaml < 6, by the way!), can’t be fixed like this. And “normal” systems, ones that have binary wheels, are absolutely fine. They don’t need Cython or setuptools. It’s entirely “weird” systems (not Python 3.18, I’m referring to Python 3.13, or Python 3.9 running on some new architecture, or the next macOS, or the next package manager, or the next brew update, etc.) that are building from binary.

Again, absolutely, thank you for 6.0.1, 99% of users didn’t even know something broke since they were using the wheels you provide, don’t let the 1% get you down!

@henryiii I hate it too (and yes, I understand the tradeoffs), but after recent events and the prospect of an endless line of setuptools breakages preventing future installs on everything we’ve ever shipped, I’ll choose continued build stability of released code over “look ma, I can build this 5 year old release of PyYAML on Python 3.18!”. Clearly your average Python user/dev doesn’t understand this stuff well enough (nor should they need to?) to figure out how to constrain a build for an old version once it’s been broken by a newly-released build dep. If it worked yesterday, it should work today… Of course there will be situations where capped releases will cause problems too, but I’d rather re-release the latest branch a hundred times doing nothing but moving build-dep pins than try to keep binary builds on non-Linux OSs functioning for old branches until the heat death of the universe.

in this case, pyyaml had six months with the open report to avoid the issue and make a fixed release before any normal user broke

You seem to be conflating “keeping new releases/unreleased things working with bleeding-edge build deps” with “keeping already-released code building reliably” when they’re nearly completely unrelated activities with competing goals. Life gets really difficult trying to reconstruct binary wheel builds for non-pure Python wheels on ancient Python/Windows/Mac versions when all publicly-available infra for building those things has rotted/disappeared. I’ve been following the Cython 3 progress for a long time, but clearly I missed that the release was actually occurring. In any case, that problem was handled in pretty short order. That has absolutely zero to do with every other release we’ve ever shipped since going to PEP517 being broken, since I don’t have a time machine.

The loudest voices here continue to shout “fix my problem by re-releasing old stuff- I don’t care who else it breaks”- IMO that’d be a pretty irresponsible approach.

… and on a personal note, thanks for the ego boost after a crappy couple of weeks. I know you understand, so I really didn’t expect that from you. 😐

I’m closing this issue, and will close everything else related to releases or backports on older branches of PyYAML- there are plenty of workarounds available, and we’re in a known state of stability at the moment. When setuptools starts breaking stuff too, the same workarounds should apply.

The way to fix this for packages that can’t use 6.0.1 is the following:

echo "cython<3" > constraint.txt
export PIP_CONSTRAINT=$PWD/constraint.txt
pip install pyyaml

For example:

$ docker run --rm -it python:2.7 bash
# echo "cython<3" > constraint.txt
# export PIP_CONSTRAINT=$PWD/constraint.txt
# pip install pyyaml --no-binary=pyyaml
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting pyyaml
  Downloading PyYAML-5.4.1.tar.gz (175 kB)
     |████████████████████████████████| 175 kB 4.8 MB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: pyyaml
  Building wheel for pyyaml (PEP 517) ... done
  Created wheel for pyyaml: filename=PyYAML-5.4.1-cp27-cp27mu-linux_x86_64.whl size=398023 sha256=c7f11faa04d9a6e0319c251b1cab970281db8d8db4704cca2947635fd2161dfb
  Stored in directory: /root/.cache/pip/wheels/ac/f7/59/730c380c3ff2d8cb0ee6c31443fab00321b1e55728b7f3a33f
Successfully built pyyaml
Installing collected packages: pyyaml
Successfully installed pyyaml-5.4.1
WARNING: You are using pip version 20.0.2; however, version 20.3.4 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.

That’s how you can fix this as a user. You can’t fix a cap (like pyyaml<6) as a user. Binaries are still fine, of course, so this needs to be on a system that doesn’t have binaries (like alpine, or above I just required pip to not see the binary).

This really needs to be fixed soon, though, as Cython 3 was a major rewrite and I’d expect package to start requiring it (I already know some that were requiring the alphas!), and I’d also rather expect some future version (maybe Python 3.13) to not be supported by Cython 0.

Now I’m going to go cry about how this example made me run Python 2, I think for the first time this year…

pip install pyyaml==6.0.1 is successful for me, while it fails with pip install pyyaml==6.0.0. So should this issue be closed as fixed?

No, it’s not fixed yet. There are many packages that depends on PyYAML>=5.4,<6 thus all the traffic in this issue. See also https://github.com/yaml/pyyaml/pull/726.

I also would like to share with you our fix for this issue, hoping it will help others and maybe adopted officially by the maintainers and pushed to pypi to close this issue:

We have an internal python package repository, so we could publish 5.4.2 internally to fix this issue. We have users of Python 3.9 that are not effected by this issue (because binary wheels are available), but also Python>=3.10 that are.

As @nitzmahone noted in https://github.com/yaml/pyyaml/pull/726#issuecomment-1640397938 we also wanted to be extra cautious and to make sure we don’t break anything for Python3.9 users.

So here’s what I did:

1. Downloaded the 5.4.1 sdist from pypi Extracted it and made the following changes
2. Replaced 5.4.1 with 5.4.2 in all the relevant files.
3. Applied the proposed fix to pyproject.toml (requires = ["setuptools", "wheel", "Cython<3.0"]).
4. Changed the minimum Python version of the package to Python>=3.10
5. packed as PyYAML-5.4.2.tar.gz. download link
6. uploaded to our internal package repo.
7. Problem solved to Python>=3.10 users. Python3.9 users are not effected, as pip skips it and installs 5.4.1 wheels as before.

Here’s the output for diff PyYAML-5.4.1/ PyYAML-5.4.2/:

diff --color PyYAML-5.4.1/CHANGES PyYAML-5.4.2/CHANGES
6a7,10
> 5.4.2 (2023-07-23)
> 
> * Patch 5.4.1 to force Cython<3, on Python>=3.10
> 
diff --color PyYAML-5.4.1/PKG-INFO PyYAML-5.4.2/PKG-INFO
3c3
< Version: 5.4.1
---
> Version: 5.4.2
33,34d32
< Classifier: Programming Language :: Python :: 2
< Classifier: Programming Language :: Python :: 2.7
36,39c34,35
< Classifier: Programming Language :: Python :: 3.6
< Classifier: Programming Language :: Python :: 3.7
< Classifier: Programming Language :: Python :: 3.8
< Classifier: Programming Language :: Python :: 3.9
---
> Classifier: Programming Language :: Python :: 3.10
> Classifier: Programming Language :: Python :: 3.11
44c40
< Requires-Python: >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*
---
> Requires-Python: >=3.10
Common subdirectories: PyYAML-5.4.1/examples and PyYAML-5.4.2/examples
Common subdirectories: PyYAML-5.4.1/lib and PyYAML-5.4.2/lib
Common subdirectories: PyYAML-5.4.1/lib3 and PyYAML-5.4.2/lib3
diff --color PyYAML-5.4.1/pyproject.toml PyYAML-5.4.2/pyproject.toml
2c2
< requires = ["setuptools", "wheel", "Cython"]
---
> requires = ["setuptools", "wheel", "Cython<3.0"]
diff --color PyYAML-5.4.1/setup.py PyYAML-5.4.2/setup.py
3c3
< VERSION = '5.4.1'
---
> VERSION = '5.4.2'
30,31d29
<     "Programming Language :: Python :: 2",
<     "Programming Language :: Python :: 2.7",
33,36c31,32
<     "Programming Language :: Python :: 3.6",
<     "Programming Language :: Python :: 3.7",
<     "Programming Language :: Python :: 3.8",
<     "Programming Language :: Python :: 3.9",
---
>     "Programming Language :: Python :: 3.10",
>     "Programming Language :: Python :: 3.11",
295c291
<         python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*',
---
>         python_requires='>=3.10',
Common subdirectories: PyYAML-5.4.1/tests and PyYAML-5.4.2/tests
Common subdirectories: PyYAML-5.4.1/yaml and PyYAML-5.4.2/yaml

Yea seems like now things are going to break more loudly for others since Cython3 was released.

it works using PyYAML~=6.0

For me, this is still failing with upgrading to PyYAML to 6.0.1 and Cython 3.0.0. I am calling it as pip-tools with pip-compile

What’s the next step for me? I am using Python 3.8 for upgrading a legacy repo and cannot upgrade Python to newer version yet.

OS: macOS 13.5 (22G74) Platform: Macbook Pro Processor: Apple M1 Max

❯ pip-compile --resolver=backtracking requirements.in

/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
    error: subprocess-exited-with-error

    × Getting requirements to build wheel did not run successfully.
    │ exit code: 1
    ╰─> [48 lines of output]
        running egg_info
        writing lib/PyYAML.egg-info/PKG-INFO
        writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt
        writing top-level names to lib/PyYAML.egg-info/top_level.txt
        Traceback (most recent call last):
          File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
            main()
          File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
            json_out['return_val'] = hook(**hook_input['kwargs'])
          File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
            return hook(config_settings)
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
            return self._get_build_requires(config_settings, requirements=['wheel'])
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
            self.run_setup()
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 338, in run_setup
            exec(code, locals())
          File "<string>", line 288, in <module>
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 107, in setup
            return distutils.core.setup(**attrs)
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
            return run_commands(dist)
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
            dist.run_commands()
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
            self.run_command(cmd)
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1234, in run_command
            super().run_command(command)
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
            cmd_obj.run()
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 314, in run
            self.find_sources()
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 322, in find_sources
            mm.run()
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 551, in run
            self.add_defaults()
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults
            sdist.add_defaults(self)
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/command/sdist.py", line 104, in add_defaults
            super().add_defaults()
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
            self._add_defaults_ext()
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
            self.filelist.extend(build_ext.get_source_files())
          File "<string>", line 204, in get_source_files
          File "/private/var/folders/j9/ybchk9m917g8lzkh2p91_tmm0000gn/T/pip-build-env-_4v02h5g/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
            raise AttributeError(attr)
        AttributeError: cython_sources
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
Traceback (most recent call last):
  File "/Users/xbeta/FOOBAR/data-processing/venv/bin/pip-compile", line 8, in <module>
    sys.exit(cli())
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/piptools/scripts/compile.py", line 592, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/piptools/resolver.py", line 593, in resolve
    is_resolved = self._do_resolve(
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/piptools/resolver.py", line 625, in _do_resolve
    resolver.resolve(
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
    if not criterion.candidates:
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__
    return bool(self._sequence)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
    return any(self)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
    candidate = func()
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 293, in __init__
    super().__init__(
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
    self.dist = self._prepare()
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
    dist = self._prepare_distribution()
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 304, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 538, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 653, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 69, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 48, in prepare_distribution_metadata
    self._install_build_reqs(finder)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 118, in _install_build_reqs
    build_reqs = self._get_build_requires_wheel()
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 95, in _get_build_requires_wheel
    return backend.get_requires_for_build_wheel()
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/utils/misc.py", line 697, in get_requires_for_build_wheel
    return super().get_requires_for_build_wheel(config_settings=cs)
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 166, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 311, in _call_hook
    self._subprocess_runner(
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/utils/subprocess.py", line 252, in runner
    call_subprocess(
  File "/Users/xbeta/FOOBAR/data-processing/venv/lib/python3.8/site-packages/pip/_internal/utils/subprocess.py", line 224, in call_subprocess
    raise error
pip._internal.exceptions.InstallationSubprocessError: Getting requirements to build wheel exited with 1

If it helps had the same issue yesterday with pyyaml==6.0, I upgraded to 6.0.1 and it seemed to work (others have also reported this makes it work https://github.com/labgrid-project/labgrid/pull/1240). I am unable to find changelogs for 6.0.1 so would not be able to explain why it works with this version but it definitely solved my issue 😃

6.0.1 can support python3 user, but python2.7 support is removed in 6.0.0，so python2.7 user needs a fix in 5.4.x

We are experiencing the same issue today with pyyaml@5.4.1. What I don’t understand yet is why we were able to install this version on Friday and not today? What has changed since Friday?

On Friday:

Today: Failed to install /home/vscode/.cache/pypoetry/artifacts/b6/23/45/f5dfdd6e8ba0f620504858ddeb20b47f50b03d0c4b18f873f6575d2e78/PyYAML-5.4.1.tar.gz

Both are duplicates of https://github.com/yaml/pyyaml/issues/601. This has been on the horizon for a long time apparently.

We have the same problem when using python 3.12. On python 3.11 works OK.

pip install PyYAML==6.0

# OUTPUT:
Collecting PyYAML==6.0 (from -r requirements-test.txt (line 1))
  Downloading PyYAML-6.0.tar.gz (124 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.0/125.0 kB 2.7 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [54 lines of output]
      running egg_info
      writing lib/PyYAML.egg-info/PKG-INFO
      writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt
      writing top-level names to lib/PyYAML.egg-info/top_level.txt
      Traceback (most recent call last):
        File "/opt/hostedtoolcache/Python/3.12.1/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/opt/hostedtoolcache/Python/3.12.1/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/hostedtoolcache/Python/3.12.1/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 11[8](https://github.com/tarantool/multivac/actions/runs/7710528936/job/21014070098?pr=142#step:5:9), in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 2[9](https://github.com/tarantool/multivac/actions/runs/7710528936/job/21014070098?pr=142#step:5:10)5, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 288, in <module>
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line [10](https://github.com/tarantool/multivac/actions/runs/7710528936/job/21014070098?pr=142#step:5:11)3, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 963, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 321, in run
          self.find_sources()
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 329, in find_sources
          mm.run()
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 551, in run
          self.add_defaults()
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults
          sdist.add_defaults(self)
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/command/sdist.py", line [11](https://github.com/tarantool/multivac/actions/runs/7710528936/job/21014070098?pr=142#step:5:12)2, in add_defaults
          super().add_defaults()
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.[12](https://github.com/tarantool/multivac/actions/runs/7710528936/job/21014070098?pr=142#step:5:13)/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
          self._add_defaults_ext()
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<string>", line [20](https://github.com/tarantool/multivac/actions/runs/7710528936/job/21014070098?pr=142#step:5:21)4, in get_source_files
        File "/tmp/pip-build-env-a_8wrjkp/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Notice:  A new release of pip is available: [23](https://github.com/tarantool/multivac/actions/runs/7710528936/job/21014070098?pr=142#step:5:24).2.1 -> 23.3.2
Notice:  To update, run: pip install --upgrade pip
Error: Process completed with exit code 1.

if u are using pipenv try :
pip install “cython<3.0.0” wheel pipenv install Cython==0.29.37 pipenv run pip3 install --no-build-isolation pyyaml==6.0 then check with : pipenv run pip list | grep PyYAML pipenv run python3 -c “import yaml; print(yaml.version)”

Thanks! Fixed. Also see https://github.com/yaml/pyyaml/issues/724#issuecomment-1650538427 above.

I would absolutely not recommend this. Read https://iscinumpy.dev/post/bound-version-constraints/. This bug wasn’t too bad because user’s could work around it for older versions with the PIP_CONSTRAINT method. You can’t work around a broken upper cap! If you pin something, especially setuptools which regularly releases new major versions and provides no support to older versions, then you are basically guaranteed to make older versions eventually uninstallable when things outside your control (Python, architectures, or OS’s, for example - exactly the reason you are not getting a wheel in the first place!) update. Or if there’s a CVE. You should treat “x<2” equivalent to using library that is abandoned unless “x<2” is still updated after “x>=2” is released. Which is very rarely the case in they Python ecosystem.

Another problem with pinning build dependencies, is that some ecosystems (Pyodide, Spack, etc) build a single setuptools version, and all projects have to use that. You don’t get to pick just one unless it’s the pre-built one.

Instead, you should enable all warnings as errors, possibly test dev releases in CI, and fix things promptly if they show up. Setuptools (and Cython, etc) should be producing warnings you are using something that is going away.

It’s easy to add a constraint if something breaks, and it’s important to quickly respond if there’s breakage - in this case, pyyaml had six months with the open report to avoid the issue and make a fixed release before any normal user broke.

@farazoman yep, as I’ve mentioned in some other places, it will almost certainly be an issue as setuptools in particular has plans to start removing deprecated functionality in the coming months, so old versions of the project will once again require external constraints or other hackery to build (also another reason that we’re probably not going to re-release old versions of the project, as it’ll potentially be an endless game of whack-a-mole).

The likely plan is that as we add support for new Python versions, the release tags will also cap all build deps to “latest known working versions” in the build metadata to prevent these kinds of problems going forward. That has a lot of other implications that are somewhat incompatible with the historic dev/branch model this project uses, so we’re working through possibly changing to a more standard “main == development” branch model where we’d leave the development branch deps uncapped to be able to detect problems with newly-release build deps more quickly.

Looks like this issue was resolved in the 6.0.1 release.

This was the magic fix that they included: https://github.com/yaml/pyyaml/commit/ae08bdc82b4ddfcd2b93c8aedcd1963766c3307d#diff-50c86b7ed8ac2cf95bd48334961bf0530cdc77b5a56f852c5c61b89d735fd711R2

I’m not sure if the maintainers read the issues (since almost 200 are unresolved). The commit they provided looked like they were updating their CI/CD. So I suspect it started to fail and they updated it for themselves, unaware of the impact that this issue had.

Either way, it looks like you can pull ~6.0 or latest again safely. @MeliJuanmi could probably mark this issue resolved

Hi again @realFranco,

Alright, still, thanks for the help !

Have a great day.

Hello @LouissXI ,

Unfortunately no, I add it as a disclaimer and expose the consequences of install the package in that version.

@NeonDaniel you’re missing build isolation. Pip will not use your environment for building a wheel unless you explicitly tell it to use --no-build-isolation.

Not sure what I’m missing here, but I’m getting the same exceptions when explicitly installing cython<3.0.0 before pyyaml~=5.4 https://github.com/NeonGeckoCom/NeonCore/actions/runs/5590442924/jobs/10220174498

Thanks @olliemath, your command saved the day. It also works with PyYaml < 6.0, and now I can at least move forward with the environment installation: pip install "cython<3.0.0" && pip install --no-build-isolation "pyyaml<6.0"

Found this in the thread which worked to patch things, hopefully only temporarily.

Was able to fix this by updating to the latest awscli v1(1.29.4) as it was a dependency for awscli. This pinned pyyaml to v 6.0.1

What about explicitly specifying working Cython

That’s exactly what was done:

https://github.com/yaml/pyyaml/blob/release/6.0/pyproject.toml

(FYI, you don’t need "wheel" there)

Long term the fix is to fix the issue with Cython, as I’m sure people will want Cython 3 (and Cython 0.x will probably not support an upcoming version of Python if they don’t back port fixes).

I’m using keycloak version 3.1.3 that depends on pyyaml 5.4.1 so I’m not able to change to an older or newer version of it. Also I’m using poetry for the dependency management. Any idea on how to solve this temporarily?

The same issue with docker-compose Python dependency.

Mostly in our case we removed pyyaml