poetry: poetry>=1.2 package with group dependencies cannot be installed with default build-system

I am on the latest Poetry version.
I have searched the issues of this repo and believe that this is not a duplicate.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

OS version and name: macOS Monterey 12.0.1
Poetry version: 1.2.0a2
pyproject.toml: default from poetry init -n

Issue

Hello! Thanks a lot for this tool.

When creating a project with poetry>=1.2 it adds the default build-system:

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

If you add a --group dependency in this project, it becomes impossible to install because the latest stable release of poetry-core does not support group and produces

Traceback (most recent call last):
  File "/tmp/tmpbqlvtksr_in_process.py", line 280, in <module>
    main()
  File "/tmp/tmpbqlvtksr_in_process.py", line 263, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "/tmp/tmpbqlvtksr_in_process.py", line 133, in prepare_metadata_for_build_wheel
    return hook(metadata_directory, config_settings)
  File "/tmp/pip-build-env-l3emamb6/overlay/lib/python3.10/site-packages/poetry/core/masonry/api.py", line 43, in prepare_metadata_for_build_wheel
    poetry = Factory().create_poetry(Path(".").resolve(), with_dev=False)
  File "/tmp/pip-build-env-l3emamb6/overlay/lib/python3.10/site-packages/poetry/core/factory.py", line 43, in create_poetry
    raise RuntimeError("The Poetry configuration is invalid:\n" + message)
RuntimeError: The Poetry configuration is invalid:
  - Additional properties are not allowed ('group' was unexpected)

Reproduce

This Dockerfile shows the issue

FROM python

RUN pip install "poetry==1.2.0a2"
# Create a package with a dev dependency
RUN mkdir -p /mypkg/mypkg \
    && cd /mypkg \
    && touch mypkg/__init__.py \
    && touch README.md \ 
    && poetry init -n \
    && poetry add --group dev pytest
 
 # Create a project that will import the package
 RUN mkdir -p /importer \
    && cd /importer \
    && poetry init -n
 
 # Trigger the error
 RUN cd /importer && poetry add /mypkg

Mitigation

Changing the build dependency to requires = ["poetry-core>=1.1.0a6"] will currently fix the issue

Resolution

If poetry 1.1 was forward compatible (as raised here #4628) it could “maybe work” if the only groups are dev dependencies not meant to be installed with the package.
Maybe poetry 1.2 could set a minimum build dependency, which would be poetry-core>=1.1 but i did not find a way to tell the build system to install with --pre to allow for pre-releases, so it would need to be explicitly something like poetry-core>=1.1.0a6.

Maybe this is not an issue, once poetry==1.2 and poetry-core==1.1 are officially release this might all go away.

Thank you very much!

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 11
Comments: 16 (6 by maintainers)

Commits related to this issue

Move group dev dependencies to dev block Using groups is a new feature in the latest preview version of poetry but it's not yet released, and still breaks installations (see https://github.com/python... — committed to JaimeLennox/djoser by JaimeLennox 2 years ago
Move group dev dependencies to dev block Using groups is a new feature in the latest preview version of poetry but it's not yet released, and still breaks installations (see https://github.com/python... — committed to JaimeLennox/djoser by JaimeLennox 2 years ago
Upgrading poetry-core dep to see if tests pass REF: https://github.com/python-poetry/poetry/issues/4983 — committed to leriomaggio/pyscript-cli by leriomaggio 2 years ago
Upgrading poetry-core dep to see if tests pass REF: https://github.com/python-poetry/poetry/issues/4983 — committed to leriomaggio/pyscript-cli by leriomaggio 2 years ago
Upgrading poetry-core dep to see if tests pass REF: https://github.com/python-poetry/poetry/issues/4983 — committed to leriomaggio/pyscript-cli by leriomaggio 2 years ago
Update meta.yaml Attempt based on this https://github.com/python-poetry/poetry/issues/4983 — committed to HeavyTony2/moe-feedstock by HeavyTony2 2 years ago

Most upvoted comments

I have the same issue on version 1.2.0b1.

+13

ljnsn on Mar 25, 2022

To be clear, the original issue this ticket was raised for still exists: A Poetry 1.2-managed project which uses groups (per the original example) still declares an unversioned build-system requirement on poetry-core, and so will fail if this is not manually adjusted to depend on poetry-core>=1.1.0a6 or similar. I just checked this hasn’t changed in Poetry master (from earlier this week, anyway).

Pendulum already has this versioned build-system requirement, so it’s not suffering from the issue in this ticket.

Poetry itself (in master) depends on poetry-core ^1.1.0b3, so it might make sense for Poetry projects created with poetry 1.2 to depend on that version of poetry-core for their build-system from the start; for existing projects, that requirement could be adjusted when some poetry-core-1.0-incompatible change , e.g. groups, or the newer console scripts support, is made via poetry CLI.

It might also make sense for the documentation to mark those pyproject.toml features that depend on poetry-core 1.1 in the documentation so that people editing their pyproject.toml manually with reference to the documentation, e.g., as happened in #4628, will know they need to ensure their build-system dependency is appropriately updated.

And as a further molly-guard, it might make sense to have Poetry notice when pyproject.toml contains features needing poetry-core 1.1 but does not declare this in its build-system block, similar to how it notices when your poetry.lock is out-of-date compared to pyproject.toml.

All this depends on having a clear record and documentation of the poetry-core 1.1 added and changed features, and the ability to recognise them in pyproject.toml. And parse the build-system requirement line to determine which version of poetry-core is being depended upon…

Separately, it might make sense for Poetry 1.2, when given add --group dev, to generate the poetry-core 1.0 format if the build-system requirement is not versioned for poetry-core 1.1 already (as opposed to bumping the poetry-core dependency as I suggested above). However, I assume that option would have been discussed in the initial implementation of generalised group support and not seen as valuable at the time.

TBBle on Jul 15, 2022

I was just stumbling upon this while trying to build pendulum. Is there a way around this?

kakulukia on Mar 1, 2022

Something is wrong in your environment and you are building with an old version of Poetry installed using get-poetry.py, as evidenced by the path ~/.poetry/lib/poetry/_vendor/py3.8/poetry/core/factory.py which indicates both the old installer and a 1.1.x build using the old distribution method.

Please reach out on Discord or through Discussions instead of commenting on old issues; usually any relation in a solved issue is surface level, like this one.

neersighted on Nov 5, 2022

Hey, why is pendulum requiring a preview version of poetry for installation was this necessary ? I had to build from source poetry as the one I use from my classical repos is too outdated, but I find this weird, for one package to be installed, we have to update the whole package manager…

I don’t think this is an issue of Poetry, but a decision of Pendulum maintainers. You should raise the question in the Pendulum repository.

Secrus on Jul 14, 2022