Hello,

the “issue” here is one of the most misunderstood of poetry’s version resolving. poetry checks if the versions that are given in the pyproject.toml are always valid and not just with the currently used python version.

So, if one define that the project is compatible with python ^3.6 this means it’s compatible with all python versions >=3.6 but <4.0.

Let’s have a look at one of the examples above:

dataclasses requires Python >=3.6, <3.7

dataclass is not available for python >3.7 here.

Or this one:

[SolverProblemError]
The current project's Python requirement (>=3.7) is not compatible with some of the required packages Python requirement:
  - taskipy requires Python >=3.6,<4.0
  - taskipy requires Python >=3.6,<4.0

The python requirement for the project has no upper boundary but taskipy has.

A further requirement is, that there is one major version of a package that is compatible with all python version. If this is not the case, the cli is not able to resolve this and we have to help by defining the version in the pyproject.toml. See here how this looks like.

fin swimmer

Why does Poetry not try to find the latest release of a package that DOES fulfill the constraints?

This is so frustrating. I have a brand new project that I have defined as such:

[tool.poetry]
name = "newproject"
version = "1.0.0"
description = "A project"
authors = ["Your Name <you@example.com>"]

[tool.poetry.dependencies]
python = "^3.7"

Super basic. Nothing special. Created it with init, setup the env with poetry env use c:\python37\python.exe and no problem thus far.

Then I run this:

D:\repos\newproject> poetry add pyproj
Using version ^3.3.0 for pyproj

Updating dependencies
Resolving dependencies...

  SolverProblemError

  The current project's Python requirement (>=3.7,<3.8) is not compatible with some of the required packages Python requirement:
    - pyproj requires Python >=3.8, so it will not be satisfied for Python >=3.7,<3.8

  Because no versions of pyproj match >3.3.0,<4.0.0
   and pyproj (3.3.0) requires Python >=3.8, pyproj is forbidden.
  So, because newproject depends on pyproj (^3.3.0), version solving failed.

At what point did I say that newproject depends on pyproj? I am adding it right now. It chose that version, and then complains that the version isn’t compatible. If we look at the pypi for pyproj, it’s clear to see that while 3.3.0 is Python 3.8 or above, 3.2.1 works with Python 3.7. So why won’t poetry use it instead when I have not asked for a specific version of it? If the argument is that Poetry is not a package installer but a dependency manager, it should absolutely be able to find the correct dependency given the constraints.

@nils-werner I know it might be confusing but there is actually a good and logical reason behind the way Poetry behaves in those cases.

As an example, I will take the taskipy example mentioned by @matthijskooijman:

• The project declares being compatible with Python >=3.7
• This project depends on taskipy which declares being compatible only with Python >=3.6,<4.0.
• So, it means that taskipy would not be satisfied for Python >=4.0 (the difference between >=3.7 and >=3.6,<4.0)

At this point, you have two choices, either change the python requirement of your project to >=3.7,<4.0 (however, note that this is not what I would personally recommend because it would mean changing the declared compatible Python versions to accommodate the dependencies) or specify that the taskipy dependency should only be installed for Python >=3.7,<4.0 by using the python property or the markers property:

taskipy = {version = "^1.2.1", python = ">=3.7,<4.0"}

It will now work properly.

This will be better explained when the 1.1 version of Poetry is released:

On another note

I absolutely get that the Python version should be pinned during deployment, but that’s what poetry.lock does/should do, no?

No, the lock file is system and Python version agnostic.

I also ran into this problem just now. I hear that this is intentional, but I’m not sure why.

the “issue” here is one of the most misunderstood of poetry’s version resolving. poetry checks if the versions that are given in the pyproject.toml are always valid and not just with the currently used python version.

I think this is the essence. Why do all python versions given in the pyproject.toml need to valid with all dependencies?

Or, to make it more specific, look at this example:

The current project’s Python requirement (>=3.7) is not compatible with some of the required > packages Python requirement:

• taskipy requires Python >=3.6,<4.0

I can see that installing taskipy would limit the python versions that the project works with to < 4.0. And apparently the solution here, would be to modify the project’s python requirement to be >=3.7,<4.0, making it a subset of the taskipy allowed python versions (did I get that right?).

But why does one need to do this manually? Isn’t this something that poetry could sort out? If every library, and the project, declares what Python versions it works with, then poetry can figure out what the resulting supported Python range is? If this results in an empty set of Python versions, that’s of course problematic, but that does not seem to be the case at hand.

I can see that there is some documentation value in a project’s pyproject.toml listing the effective usable Python versions for the project, but there is also value in documenting the python requirement of the project itself, without any dependency-imposed requirements. Especially since the latter can easily change when updating dependencies, that means that you would end up artificially limiting the python versions for the project, and then having to reconsider (manually check) whether the requirements can be relaxed again whenever dependencies are updated.

If this is is indeed the rationale for this requirement, I guess that the proper solution could be to have two python version requirements for the project: One that shows the requirements of the project itself, and one that shows the requirements of the project and all its requirements combined (where the former is manually configured and the latter is automatically determined by poetry whenever the lock file changes).

Another thing I wondered: Is this same policy applied to normal dependencies, or just to the python version? In other words, if a project depends on package A ^1.0 and also on a package B that depends on A >= 1.0, will this also produce the same error (I suspect not, since then the micromanagement of dependency versions might have been way too complicated). I guess the documentation value of listing the effectively viable versions of a package might be lower than for the python version, so if there is indeed an inconsistency here, that could explain this.

Just experienced the same strange behavior.

Seems like when you use tilde requirement for Python: ~3.7 this will work as expected.

IMO, the caret requirements ^4.2 is very hard to understand and use properly. I propose to discard them at all.

changing ‘python = “^3.8”’ to ‘python = “~3.8”’ allowed to get around the erroneous dependency constraint resulting in an error (as per above)

I @finswimmer, thanks for the explanation.

You have given a few examples for why we should decide/pin the Python version first, and then install dependencies. However, can’t you just use these as arguments against version deciding/pinning, too?

What python versions will I support.

That’s exactly my thought when starting out: What versions do I/does this library support? Most likely it is python>=3.6. Why do I have to go look at each individual dependency and adjust my Python version accordingly?

You need to use a special syntax that is only available in on version

Syntax features are available in a wide range of Python versions. If a user wants to use a specific syntax, say f-strings. That doesn’t mean they have to pick python==3.6, but instead python>=3.6. If now a dependency comes along and requires python>=3.7, that’s fine, f-strings are still there.

you already know that you will need a specific package that is only available for python x.y,

I am not saying I should not care, but why exactly do I have to know this? Say in a scenario where I want to use Django 3, shouldn’t Poetry figure out that django>=3.0 in fact needs python>=3.6?

(Sidenote: the fact that poetry wants me to switch Python versions using poetry env use, but is nevertheless still able to create the correct environment, does not make sense to me, either.)

I absolutely get that the Python version should be pinned during deployment, but that’s what poetry.lock does/should do, no?

The mental model I have: Python is just another dependency, just like any package from PyPI. In my mental model I should not have to treat it specially. If I and all my dependencies define their compatible version range (and I don’t have a conflict somewhere), Poetry should be able to find a valid Python version that suits all. It should be flexible in pyproject.toml and pinned in poetry.lock. And I shouldn’t have to pick a Python version prior to running poetry install. But that’s just my mental model, maybe I am wrong somewhere and am just failing to see it. 😃

I have the same when I want to add PySide2.

$ poetry add pyside2

[SolverProblemError]
The current project's Python requirement (^3.7) is not compatible with some of the required packages Python requirement:
  - pyside2 requires Python >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <3.9

Because no versions of pyside2 match >5.14.0,<6.0.0
 and pyside2 (5.14.0) requires Python >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <3.9, pyside2 is forbidden.
So, because tester depends on pyside2 (^5.14.0), version solving failed.

When I use the ‘fix’ from @MarcBoissonneault (changing to python = "3.7") it works.

I just stumbled upon this myself when using a dependency that depends on the backport of dataclasses.

Installing version 0.6 of dataclasses works fine, but 0.7 has python_requires=">=3.6, <3.7" in setup.py and that seems to trip up poetry’s resolver.

After having been stuck in this rabbit hole for several hours, I would like to add my thoughts to this very old thread. I think the rationale behind the current behaviour is very inconsistent.

Basically, poetry is first lying on our behalf, and a few moments later, it is calling us out on that lie.

What is the lie?
When we write python = "^3.11" in the .toml file, we are lying. We are boldly and confidently declaring that our package will be compatible with any future version of Python 3.

But we can’t know that for certain. Future minor versions of Python 3 may break our own code. Or the packages we depend on may not be upgraded to support those Python versions.

Is the lie acceptable?
Well, if you want to be very hardcore about compatibility, the lie is not acceptable. You should only state compatibility with currently known minor versions, after testing those versions.

However, for many of us, the lie is fully acceptable. We understand that the creator of an imaginary and now abandoned my_hello_world package can’t know with 100% certainty if Python 3.47 is going to deprecate the print statement, so his code stops working. So he is gambling when he writes ^3.11. But if that means that we can also use his package in year 2039 without being forced into using python 3.11, this gamble is acceptable. After all, we had 16 years to find a replacement for his package after he stopped maintaining it.

We also understand that other packages, which our package depends on, are not yet released for Python minor versions, which do not exist yet. There is no pandas package for Python 3.13, because Python 3.13 has not yet been released. But if we include pandas as a dependency in our project, we can be quite confident that there will be a pandas package for Python 3.13 if someone downloads our package to his python 3.13 environment a few years from now. And if there isn’t a pandas package, his package manager should tell him to downgrade his python version anyway.

Does poetry think that the lie is acceptable? Poetry seems to think that the lie is not acceptable. And based on the comments from finswimmer in this thread, this is intentional.

Consequently, when I try to include pandas as a dependency in my package, my screen will be filled with red text stating:

“Hey! What are you doing? You have said that your package is compatible with all future minor versions of Python 3, and now you try to install pandas, which have no packages for Python 3.13.”

Well, duh! Python 3.13 isn’t released yet (as least not in a stable version), so of course there is no pandas package for Python 3.13.

Who made the lie?
This is the funny thing. Poetry did!

When I use poetry for creating a new environment with the command poetry new --src <packagename> , poetry will create a pyproject.toml file containing this:

[tool.poetry.dependencies]
python = "^3.11"

That is not really consistent with poetry’s hardcore approach to dependencies, is it?

Why not let the default entry in pyproject.toml be this:

[tool.poetry.dependencies]
python = "~3.11"

That would be in line with poetry’s hardcore approach to dependency solving, and then it would be up to the developer to slacken the dependencies if he has reason for doing that.

TL;DR In my opinion, the poetry developers need to make a choice and stick consistently to it:

• Either accept hypothetical future incompability of future Python minor versions.
• Or create a pyproject.toml file, which is limited to the current Python minor version.

With the current behaviour, I first have to create the environment with poetry, then manually edit the pyproject.toml file to make the change from ^3.11 to ~3.11, and then I can start adding packages.

@finswimmer I will argue that, since this many people got confused, this is actually a bug that needs to be resolved somehow. We’d at least appreciate the proper way to do this, as currently no solution is posted. Maybe a solution could be included in the docs?

Same problem when trying to add the dataclasses backport package for Python 3.6:

[SolverProblemError]
The current project's Python requirement (^3.6) is not compatible with some of the required packages Python requirement:
  - dataclasses requires Python >=3.6, <3.7

Because dataclasses (0.7) requires Python >=3.6, <3.7
 and no versions of dataclasses match >0.7,<0.8, dataclasses is forbidden.
So, because decred depends on dataclasses (^0.7), version solving failed.

Poetry version: 1.0.3

I have the same with pandas = "^0.25.1" and python = ">=3.5, <3.8"

...

[tool.poetry.dependencies]
python = ">=3.5, <3.8"
numpy = "^1.15.4"
scipy = "^1.2.0"
pandas = "^0.25.1"

...

[SolverProblemError]
The current project's Python requirement (>=3.5, <3.8) is not compatible with some of the required packages Python requirement:
  - pandas requires Python >=3.5.3
  - pandas requires Python >=3.5.3
  - pandas requires Python >=3.5.3

Because no versions of pandas match >0.25.1,<0.25.2 || >0.25.2,<0.25.3 || >0.25.3,<0.26.0
 and pandas (0.25.1) requires Python >=3.5.3, pandas is forbidden.
And because pandas (0.25.2) requires Python >=3.5.3, pandas is forbidden.
So, because pandas (0.25.3) requires Python >=3.5.3
 and thesis depends on pandas (^0.25.1), version solving failed.

Hello @matthijskooijman,

I think this is the essence. Why do all python versions given in the pyproject.toml need to valid with all dependencies?

at first place poetry is a tool for development. When you start with a new python project, the first question you have to answer is: What python versions will I support. The reasons for the answer can be different: You need to use a special syntax that is only available in on version, you know your target platform, you already know that you will need a specific package that is only available for python x.y, …

Once you’ve answer this question and write it down to your pyproject.toml poetry will make sure that with the dependencies you try to add, this goal can be reached.

But why does one need to do this manually? Isn’t this something that poetry could sort out? If every library, and the project, declares what Python versions it works with, then poetry can figure out what the resulting supported Python range is?

Might be possible, but that’s not how should manage your python project. As I said, you first have to decide what python versions you want to support, because this is essential for the python syntax you will use. Having valid dependencies is only the second step. Not the other way round.

In other words, if a project depends on package A ^1.0 and also on a package B that depends on A >= 1.0, will this also produce the same error.

No, this will not produce an error. Because the version constraints for the package tell poetry from which versions it can choose. Which one it will choose depends on the other packages in the project and there dependencies.

fin swimmer

@sdispater’s solution we wait for the dependency to be compatible with our vision of our project muscle. We don’t bend our vision to match those of our dependencies (though we have to keep in mind what you were saying about missing dependencies)

But we do bend our vision, because we specify that taskipy is not needed for python >= 4.0, even when that is not true.

I would greatly prefer to just state the dependencies as they are in reality, so poetry can correctly deduce that my package is not installable for python >= 4.0 and can provide feedback to users that do want to install my package with python 4.0.

It might still be useful if this information is also shared with the current package maintainer, but I do not think it should prevent adding the dependencie. i.e. I can imagine that running poetry add will say "Adding package taskipy limits the effective supported python versions (declared >= 3.7, effective >= 3.7 <= 4.0) and then continue adding it. Maybe this information could be displayed on request as well, or maybe stored in an automatically updated field in pyproject.toml or poetry.lock?

At this point, you have two choices, either change the python requirement of your project to >=3.7,<4.0 (however, note that this is not what I would personally recommend because it would mean changing the declared compatible Python versions to accommodate the dependencies)

Exactly.

or specify that the taskipy dependency should only be installed for Python >=3.7,<4.0 by using the python property or the markers property:

That does not really seem right either: Aren’t you now specifying that for Python >= 4.0, taskipy is not needed? If you would be using Python 4.0, poetry would happily install without taskipy, thinking it has produced a working set of dependencies, while in reality your application will break due to a missing dependency?

Hey @hnykda,

sorry if this was not clear enough. So here is another try 😃

poetry is not a tool to install packages into a given environment like pip.

The important thing I wanted to point out here, was “like pip”. pip’s purpose is to install a package into an environment. Of course Poetry need to do this as well. But this is not the main purpose of Poetry, it’s just a thing that have to be done. From the docs, the goal of Poetry is:

It allows you to declare the libraries your project depends on and it will manage (install/update) them for you.

To reach this goal, Poetry needs to make sure that your project definition is valid. And that’s the main task. Poetry thinks in “projects” and not in “current environment”. And this is why it complains, if the current environments fits to the needs of a dependency, but the project definitions doesn’t fit.

Is it somehow clearer now?

fin swimmer

Why does Poetry not try to find the latest release of a package that DOES fulfill the constraints?

This is so frustrating. I have a brand new project that I have defined as such:

[tool.poetry]
name = "newproject"
version = "1.0.0"
description = "A project"
authors = ["Your Name <you@example.com>"]

[tool.poetry.dependencies]
python = "^3.7"

Super basic. Nothing special. Created it with init, setup the env with poetry env use c:\python37\python.exe and no problem thus far.

Then I run this:

D:\repos\newproject> poetry add pyproj
Using version ^3.3.0 for pyproj

Updating dependencies
Resolving dependencies...

  SolverProblemError

  The current project's Python requirement (>=3.7,<3.8) is not compatible with some of the required packages Python requirement:
    - pyproj requires Python >=3.8, so it will not be satisfied for Python >=3.7,<3.8

  Because no versions of pyproj match >3.3.0,<4.0.0
   and pyproj (3.3.0) requires Python >=3.8, pyproj is forbidden.
  So, because newproject depends on pyproj (^3.3.0), version solving failed.

At what point did I say that newproject depends on pyproj? I am adding it right now. It chose that version, and then complains that the version isn’t compatible. If we look at the pypi for pyproj, it’s clear to see that while 3.3.0 is Python 3.8 or above, 3.2.1 works with Python 3.7. So why won’t poetry use it instead when I have not asked for a specific version of it? If the argument is that Poetry is not a package installer but a dependency manager, it should absolutely be able to find the correct dependency given the constraints.

This is also my question, which is how I ended up here…

• I have python=~3.8, which I understand to mean >=3.8, ≤3.9.
• But when I try to install flake8, it tells me flake8 version 6.1.0 is not compatible with 3.8.1, so it doesn’t satisfy the range. Makes sense…
• But why doesn’t it install 5.0.4, which is compatible with >=3.6?

@nils-werner I know it might be confusing but there is actually a good and logical reason behind the way Poetry behaves in those cases.

As an example, I will take the taskipy example mentioned by @matthijskooijman:

* The project declares being compatible with Python `>=3.7`

* This project depends on `taskipy` which declares being compatible only with Python `>=3.6,<4.0`.

* So, it means that `taskipy` would not be satisfied for Python `>=4.0` (the difference between `>=3.7` and `>=3.6,<4.0`)

At this point, you have two choices, either change the python requirement of your project to >=3.7,<4.0 (however, note that this is not what I would personally recommend because it would mean changing the declared compatible Python versions to accommodate the dependencies) or specify that the taskipy dependency should only be installed for Python >=3.7,<4.0 by using the python property or the markers property:

taskipy = {version = "^1.2.1", python = ">=3.7,<4.0"}

It will now work properly.

This will be better explained when the 1.1 version of Poetry is released:

On another note

I absolutely get that the Python version should be pinned during deployment, but that’s what poetry.lock does/should do, no?

No, the lock file is system and Python version agnostic.

This is really counter-intuitive, a bad API design example tbh

@skorokithakis This won’t be treated as a bug since it’s not one but the intended behavior.

That being said I agree that, since it’s confusing for some people, we should give guidance to resolve the issue. That’s something that is in progress and will be available in the next feature release (1.1). Here is what it will look like:

I had the same problem trying to install coverage 4.5.4 with python = ">=3.7". Changing it to "^3.7" resolved.