poetry: Poetry build is naming the wheel incorrectly

I am on the latest Poetry version.
I have searched the issues of this repo and believe that this is not a duplicate.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
macOS 10.15.7
1.1.4
poetry-core 1.0.0

Issue

This is a spin off of #3508 as further investigation revealed there were two problems uncovered. This is about the naming of the wheels when using poetry build.

Poetry is giving the wrong version tag to the built wheel. The following is the output of Poetry build:

$ poetry build 
Building build_test (0.1.0)
  - Building sdist
  - Built build_test-0.1.0.tar.gz
  - Building wheel
A setup.py file already exists. Using it.
running build
running build_py
  - Built build_test-0.1.0-cp39-cp39-macosx_10_15_x86_64.whl

Note that the Py version is 3.9 but below, the project is shown to be exclusively 3.8. In fact, the only place where Python 3.9 is installed on my system is either through brew as a dependency for something or through the pipx isolated environment used to install Poetry.
Even though the build script does nothing, and even if there was something to do, the build would be executed in the correct venv (I’ve verified this) meaning that the issue really is just with the name of the wheel.

Here is how poetry is installed:

$ pipx list
venvs are in /Users/alexifm/.local/pipx/venvs
apps are exposed on your $PATH at /Users/alexifm/.local/bin
   package poetry 1.1.4, Python 3.9.1
    - poetry

Poetry config:

$ poetry config --list
cache-dir = "/Users/alexifm/Library/Caches/pypoetry"
experimental.new-installer = true
installer.parallel = true
virtualenvs.create = false
virtualenvs.in-project = null
virtualenvs.path = "{cache-dir}/virtualenvs"  # /Users/alexifm/Library/Caches/pypoetry/virtualenvs

And here is the project definition that requires a build (note that the project is exclusively Python 3.8).

Project virtual environment:

$ pyenv versions
  system
  3.8.5
  3.8.5/envs/build_test
* build_test (set by /Users/alexifm/Projects/build_test/.python-version)

pyproject.toml:

[tool.poetry]
name = "build_test"
version = "0.1.0"
description = ""
authors = ["Alex Papanicolaou <alex@infima.io>"]
build = "build.py"

[tool.poetry.dependencies]
python = "~3.8"

[tool.poetry.dev-dependencies]
pytest = "^5.2"

[build-system]
requires = ["poetry_core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

build.py:

"""No build."""

def build(setup_kwargs):
    """Build."""
    pass

Probable Reason

poetry-core generates the information for the wheel tag here: https://github.com/python-poetry/poetry-core/blob/master/poetry/core/masonry/builders/wheel.py#L244

The vendored packaged packaging produces the tags here: https://github.com/python-poetry/poetry-core/blob/master/poetry/core/_vendor/packaging/tags.py#L744

Finally, it looks like since no kwargs are passed to sys_tags, then the argument python_version isn’t passed to cpython_tags and it defaults to the system Python, ie whatever is running Poetry. https://github.com/python-poetry/poetry-core/blob/master/poetry/core/_vendor/packaging/tags.py#L234

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 33 (1 by maintainers)

Commits related to this issue

Add local script for building macos-arm wheels GitHub actions doesn't have M1 and python-poetry/poetry#3509 means that poetry names wheels incorrectly, so cibuildwheel can't be used to cross-compile. — committed to dials/pycbf by ndevenish 2 years ago

Most upvoted comments

I just replicated using Poetry’s own virtual environments so it’s not a pyenv thing.

$ poetry build -vvv                     
Using virtualenv: /Users/alexifm/Library/Caches/pypoetry/virtualenvs/build-test-9tgLzTmA-py3.8   # <- Poetry's Py3.8 environment
Building build_test (0.1.0)
  - Building sdist
  - Adding: /Users/alexifm/Projects/build_test/build_test/__init__.py
  - Adding: pyproject.toml
  - Built build_test-0.1.0.tar.gz
  - Building wheel
  - Adding: /Users/alexifm/Projects/build_test/build_test/__init__.py
running build
running build_py
Skipping: /Users/alexifm/Projects/build_test/COPYING
Skipping: /Users/alexifm/Projects/build_test/LICENSE
  - Built build_test-0.1.0-cp39-cp39-macosx_10_15_x86_64.whl   # <- Named with 3.9 used to run Poetry (pipx)

ETA: It’s also not a pipx thing as I can replicate it with Poetry installed through homebrew.

alexifm on Dec 28, 2020

To be honest though I strongly recommend just not using poetry for building your stuff. It’s too buggy and opinionated around local environments so it doesn’t offer what’s needed to true publishing. Utilizing the build project has been a much nicer experience.

tedivm on Dec 19, 2022

This should be fixed by https://github.com/python-poetry/poetry-core/pull/227 (it has been working for us for months using a locally patched version). There seem to be a lot of open PRs for poetry and poetry-core, so I’m not sure how to get it reviewed.

jrandall on Feb 23, 2022

Basically, if you’re not building anything, then the package is platform independent and you don’t need a fancy wheel name. If you need to build something, say Cython, then the wheel name has to account for the Python version and the platform that did the compiling.

This assumption is fine as a default but doesn’t hold for all use cases. There are cases where the build script is used to create asset files that are needed by the package but which are not platform specific (for example, building a json file for data that’s going into the package). Having an option to explicitly say “this is an platform independent package” would be a lot more flexible than having the program try to guess what developers are attempting to do.

tedivm on Sep 29, 2021

I have a shared object file that needs to be compiled on different platforms.

And to clarify: the md5 mentioned above is not just the name. What I meant to say is that two files with different names have the same md5 because their contents are the same.

leotrs on Dec 20, 2022

Got it!

leotrs on Dec 19, 2022

1. In the `pyproject.toml` file, why is the `build` field in the `[tool.poetry]` section not documented in the docs?

It’s an unsupported feature and it’s why this issue is not too big of a priority.

2. It seems that if the `build` field is missing from the pyproject file the wheel will always have a `none-any.whl` suffix, as [seen here ](https://github.com/python-poetry/poetry-core/blob/master/poetry/core/masonry/builders/wheel.py#L243-L255). Is this intentional? This means unless you put a `build` field in there will never be more specific wheel than `none-any`.

Basically, if you’re not building anything, then the package is platform independent and you don’t need a fancy wheel name. If you need to build something, say Cython, then the wheel name has to account for the Python version and the platform that did the compiling.

alexifm on Jan 1, 2021

Okay, I figured out one issue.

If a user has activated their own venv with a tool like pyenv, set virtualenvs.create to false, but also has an environment in {cache-dir}/virtualenvs/envs.toml, the venv detected will be incorrect. This setup can happen if you don’t set things up perfectly initially and there’s a lingering listed environment in the cache.

The problem comes from this conditional: https://github.com/python-poetry/poetry/blob/1.1.4/poetry/utils/env.py#L463

Here’s what happens

A poetry virtualenv is checked for here: https://github.com/python-poetry/poetry/blob/1.1.4/poetry/utils/env.py#L446-L450
If you have created a virtual environment with poetry already then it’ll show up in the envs.toml file and the variable env is set on this line: https://github.com/python-poetry/poetry/blob/8312e3f2dbfa126cd311c666fea30656941e1bd3/poetry/utils/env.py#L450
If another venv is activated, say with pyenv, that’ll be picked up here: https://github.com/python-poetry/poetry/blob/8312e3f2dbfa126cd311c666fea30656941e1bd3/poetry/utils/env.py#L457
Following #2, this conditional will evaluate True. https://github.com/python-poetry/poetry/blob/8312e3f2dbfa126cd311c666fea30656941e1bd3/poetry/utils/env.py#L463
If you’ve configured Poetry to not create virtualenvs, then this will evaluate True and return the system python. https://github.com/python-poetry/poetry/blob/8312e3f2dbfa126cd311c666fea30656941e1bd3/poetry/utils/env.py#L473

Two solutions:

the user needs to clean up the envs.toml file.
The logic in .get needs adjustment so that it actually uses the environment the user has activated.

alexifm on Dec 28, 2020