poetry: PEP 440 parse exception on packages in /usr/lib/python3/dist-packages/ installed by ubuntu

Although this is closed and although the position appears to be one of “not our problem” (indicated both here and in #6334 by @dimbleby) - I would humbly offer a minor comment/observation:

Yes, it’s important for everyone else to follow standards - that’s part of the purpose of standards… and they only work if people follow them! That said, given that, within the Python ecosystem, there is not a regulatory governmental body with the power to enforce standards (e.g. the FCC/FAA in the US, Ofcom in the UK, BNA/Bundesnetzagentur in DE, etc.) … one must take a slightly more pragmatic approach, in my very humble opinion.

While I completely understand and agree that it would indeed be an impossible task to make changes to accomodate every possible exception, I do not believe it wise (as a general principle of software design) to categorically and unconditionally dismiss anything as someone else’s problem or fault.

That said, as @neersighted points out, we know that distributions (and/or other developers) are making changes to the version numbering making them incompatible with PEP 440. While they ought not to do this, they are. That’s the simple reality. Am I (or even a group of developers) going to ever have a chance of “fixing” this? No. And I don’t think I’m being defeatist - I believe it’s a realistic assessment of what is most likely to happen (or not happen). I admire Poetry’s commitment to and compliance with standards - it’s a great thing. But again, it only works if everyone follows them.

As a note to the above - The Official Debian Policy Manual describing Control files and their fields - 5.6.12 Version specifically allows verison numbers to contain characters prohibited by PEP 440 (e.g. tilde [~], plus [+], hyphen [-], etc.) … further, the Debian Python Packaging Manual makes no mention of PEP 440. I actually can’t find any existing policy guidance from Debian on PEP 440. While it does seem like the policy ought to be updated, that’s not how it stands right now.

As a result, I think that it’s important to make decisions not just on the basis of how things ought to be, but rather to take into consideration how things actually are. Perhaps there is a relatively simple way for Poetry to be slightly more flexible - not to address every single exception to PEP 440 but perhaps a few lines of code could address the majority of the exceptions? Perhaps failing more “gracefully” with a prompt/warning to the user that Poetry is about to make an assumption (in violation of PEP 440) could ease the pain that users feel?

I’ve found this thread after facing an enormous amount of PEP 440 issues myself (I have ubuntu packages installed).

I’m a long time poetry fan, but I’m surprised this decision was made with regards to alienating the grand majority of Ubuntu users. This was a really hard stance to take without providing any way around it. I appreciate how you are trying to push the industry into using a better standard, but “don’t use that package” or “Tell Canonical to abandon their standard” isn’t a viable solution for, well, anyone.

To advocate for a change in Poetry’s parsing behavior, I would suggest presenting a use case that cannot be solved by virtual environments and that requires colliding with the system-provided dist-packages.

I think you don’t need to look very far. Simply look ask “how do I …” for nearly any problem in python, and you’ll see that there are many options within python itself. Look no further than __version__. Edit: I know this isn’t a direct use case. Simply pointing out that there is so much variation in how python is used that forcing a specific path is not really a viable option.

Bottom line: Conventions and compliance are great, but when there’s a long history without, adoption must be flexible. From where I stand, this looks like a tipping point for poetry. The issues surrounding 1.2 version have many of my coworkers starting to suggest that we stop using poetry. I don’t think that the pros yet outweigh the cons, but we are actively rolling back from 1.2.

Hello @pae23,

I’m afraid there’s nothing Poetry can do about it. For proper dependency resolving it is necessary to “understand” the version number of a Package. If a package doesn’t follow PEP-440 this is not possible.

Please see also my comment in https://github.com/python-poetry/poetry/issues/2167#issuecomment-748436046

My advice in your case would be to not use the system-site-packages.

fin swimmer

Maybe if there are potential conflicts a warning message could be given. Another idea would be to have an option (config or cli flag) to ignore potential conflicts when it isn’t parseable. Crashing isn’t the only option here.

Its very concerning that poetry seems to have no issue making breaking changes without any mention in the changelog. Currently we are pinned to the old version that still works, and are looking for alternatives.

This problem prevents me from using Poetry reliably on my Raspbian Linux system. If I turn off system-site-packages I can’t use things like gi which are next to impossible to install in a virtualenv, but if I leave it on this error occurs.

@neersighted I came across this in the docs:

If Poetry detects it’s running within an activated virtual environment, it will never create a new virtual environment, regardless of the value set for virtualenvs.create.

Is there any potential workaround I can apply to allow poetry to create a new venv from within one?

This is off topic for this issue – I’d suggest confining questions to Discussions or Discord (or an issue if you think there might be a bug). That being said, I’m not sure what you mean. Poetry itself should always be installed into a virtual environment (or installed from distro packaging – we don’t provide support for distro installs, but the distro packagers should hopefully take steps to ensure Poetry’s dependencies are stable/protected).

The quote you are referring to is not talking about an install however. It is talking about a virtual environment being active before you run poetry install, poetry add, etc. If a virtual environment is active, Poetry will re-use it instead of creating its own. If you want Poetry to manage the virtual environment for you, don’t activate an external environment. If you want Poetry to not manage the virtual environment for you, activate an environment of your choice.

If for some reason you only want to deactivate the virtual environment for Poetry (why?), you might be able to get away with VIRTUAL_ENV= poetry <command>, but I wouldn’t suggest it as it likely will misbehave if virtualenvs.prefer-active-python true is set. To properly deactivate a virtual environment you need to clear VIRTUAL_ENV and remove the bin directory from the PATH.

If you have questions about this, please join Discord or start a Discussion.

YunoHost version scheme looks like: 0.16.0~ynh1 (Where 0.16.0 is the upstream version and ~ynh1 a suffix of the YunoHost package) … Because of this hard PEP 440 error, i can’t use Poetry v1.2.x now.

I would have expected that first a WARNING is emitted, and non PEP 404 versions are still allowed for a while. But with 1.2.0 this breaks things.

How was poetry able to parse it before then? It works fine in version 1.1.15. The regression seems to be only present in 1.2.x.