setuptools: Setuptools does not pass config_settings through backend

The build project now has a --config-setting flag to pass options to the backend:

$ python -m build --help
usage: python -m build [-h] [--version] [--sdist] [--wheel] [--outdir dir] [--skip-dependencies]
                       [--no-isolation] [--config-setting CONFIG_SETTING]
                       [srcdir]

- - - >8 - - -

  --config-setting CONFIG_SETTING, -C CONFIG_SETTING
                        pass option to the backend

I wanted to use this to set wheel’s --plat-name argument like so:

$ python -m build --wheel --config-setting plat-name=foo

I found that this worked as expected until the settings got to setuptool’s _build_with_temp_dir function:

https://github.com/pypa/setuptools/blob/d368be2b5be8676ba8a3d55045b45b55090f6982/setuptools/build_meta.py#L190-L200

In this example, _build_with_temp_dir gets called with config_settings={'plat-name': 'foo'}, which then gets ‘fixed’ into config_settings={'plat-name': 'foo', '--global-option': []}, and then setuptools ignores everything in config_settings and only considers --global-option.

It almost seems like --global-option itself could be used for this, but unfortunately it doesn’t work as it’s value is passed to lots of things that don’t accept the arguments I’m hoping to pass:

$ python -m build --wheel --config-setting="--global-option=--plat-name" --config-setting="--global-option=foo" -n
usage: _in_process.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: _in_process.py --help [cmd1 cmd2 ...]
   or: _in_process.py --help-commands
   or: _in_process.py cmd --help

error: option --plat-name not recognized
...

I created a commit that “fixes” this for me as a proof of concept: https://github.com/pypa/setuptools/commit/fc95b3b83d6d5b561dc0a356995edf4c99785a6f

Possibly related issues:

this is very similar to the request in https://github.com/pypa/setuptools/issues/1816, but @gaborbernat is asking for the ability to pass options before the setup command, rather than after.
https://github.com/pypa/setuptools/issues/1928 may also be related

About this issue

Original URL
State: open
Created 4 years ago
Reactions: 11
Comments: 43 (33 by maintainers)

Links to this issue

PEP 660 and setuptools - Packaging - Discussions on Python.org

Commits related to this issue

517: clarify config_settings section Because it is hard to implement in current form https://github.com/pypa/setuptools/issues/2491#issuecomment-761030842 — committed to abitrolly/peps by abitrolly 3 years ago
Add custom build backend to support build args This implements a custom build backend, inspired by the [solution used by Pillow.](https://github.com/python-pillow/Pillow/pull/7171) install-option is... — committed to tobiasah/pycapnp by tobiasah 9 months ago
Add custom build backend to support build args This implements a custom build backend, inspired by the [solution used by Pillow.](https://github.com/python-pillow/Pillow/pull/7171) install-option is... — committed to tobiasah/pycapnp by tobiasah 9 months ago
Add custom build backend to support build args (#328) This implements a custom build backend, inspired by the [solution used by Pillow.](https://github.com/python-pillow/Pillow/pull/7171) install... — committed to capnproto/pycapnp by tobiasah 9 months ago
Only pass `--build-option` to `bdist_wheel` in build_meta In https://github.com/pypa/setuptools/issues/2491#issuecomment-1742859314 the discussion seems to lead to the idea that it is better for now ... — committed to abravalheri/setuptools by abravalheri 9 months ago
Only pass `--build-option` to `bdist_wheel` in build_meta In https://github.com/pypa/setuptools/issues/2491#issuecomment-1742859314 the discussion seems to lead to the idea that it is better for now ... — committed to abravalheri/setuptools by abravalheri 9 months ago
Only pass `--build-option` to `bdist_wheel` in build_meta In https://github.com/pypa/setuptools/issues/2491#issuecomment-1742859314 the discussion seems to lead to the idea that it is better for now ... — committed to abravalheri/setuptools by abravalheri 9 months ago
python-lxml: update to 4.9.4 — committed to msys2/MINGW-packages by MehdiChinoune 6 months ago

Most upvoted comments

FYI I am also in favor of removing the dashes, I don’t think they are needed in PEP 517.

FFY00 on Jan 16, 2021

Hi @jameshilliard, I just pushed the bugfix you provided in v69.1.1, once the CI is done you can see if that works for you. Thank you very much for reporting the problem and working on the fix.

/fyi @sbidoul, the change in #4217 should not introduce problems for pip, but we never known… Please let me know if you find problems.

abravalheri on Feb 23, 2024

I don’t know why the PEP 517 options in setuptools were modeled on pip, but the main audience here isn’t people transitioning from legacy pip installs to isolated pip installs - it’s people transitioning from invoking setup.py directly, to build. A --build-option which only works with bdist_wheel is not an adequate replacement.

Right now, I don’t think we are in a good position even to “paper over the cracks”. The situation with egg_info/dist_info “idempotency” (among the multiple direct/indirect calls triggered by PEP 517/PEP 660 hooks in different sub-processes) is difficult to deal with.

Solving this and related problems is one of the major steps in creating a more definitive solution[^1]. So if the most immediate concern is to offer a solution for people migrating from direct running setup.py, I don’t think is worth bothering with an intermediate stop-gap solution[^2][^3] and we just go with #3896.

On the other hand, if the most immediate concern is to offer a solution for compatibility with pip legacy flags, we can easily create a PR that ignores config_settings["--build-option"] for all the PEP 517 hooks except setuptools.build_meta:build_wheel.

[^1]: I did a summary of current state and challenges here: https://github.com/pypa/setuptools/issues/3896#issuecomment-1656714771.

[^2]: Again, designing a more definitive solution is difficult and can take time. In the discussion in https://github.com/pypa/setuptools/issues/3896#issuecomment-1656714771 and https://github.com/pypa/setuptools/issues/3896#issuecomment-1708438587, I highlighted on how I would go about it, but I invite any member of the community to discuss and contribute.

[^3]: The best thing people looking for an alternative to python setup.py arguments can do right now is to modify setup.cfg or use the DIST_EXTRA_CONFIG env var.

abravalheri on Oct 3, 2023

@hugovk IMO that is a very reasonable way to do it, and would also be my recommendation.

FFY00 on Jun 13, 2023

We’re considering something like https://github.com/python-pillow/Pillow/pull/7171 to work around this with a custom backend. Does anyone have a nicer way to do it?

hugovk on Jun 13, 2023

I am not sure if this is related, but when attempting to use --config-settings to pass py-limited-api, I’m getting the error:

  usage: _in_process.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: _in_process.py --help [cmd1 cmd2 ...]
     or: _in_process.py --help-commands
     or: _in_process.py cmd --help

  error: option --py-limited-api not recognized

alex on Apr 25, 2023

I commented on this upthread:

Incidentally, global options in setuptools seem to mean something different than they do in pip where they are placed in front of the setuptools command. Here, they are appended to the command. pip users might find the discrepancy surprising.

It is not immediately clear to me that the spec mandates that --global-option should come before setup_command […]

This is all setuptools-specific stuff, the spec has no notion of global options nor does it expect that options will be passed to a CLI.

layday on Mar 3, 2022

@dixonwhitmire yes, that’s correct.

abitrolly on Jan 21, 2022

Hi @abitrolly so just to clarify for myself and possibly for others following this issue, is the next step here to discuss potential changes in https://discuss.python.org/c/packaging/14 so that the PEP is updated with well defined statements? Thank you!

dixonwhitmire on Jan 21, 2022

Issues raised against build concerning this behaviour:

layday on Jan 23, 2022

I’m sure the reason why the PEP is non-specific about the format is because it would be difficult for a protocol like that to be specific without being over-constrained for a particular use-case.

I’m not sure why arguments to sdist and bdist_wheel should not be funneled through config_settings verbatim - why is --global-option needed at all?

I imagine that the sdist and bdist_wheel commands have settings that are unique to the command. I also expect there could be settings that are applied globally, independent of any command.

Regardless, what I’d like to see happen is for someone to examine the code and determine all the command-line parameters setuptools currently solicits that would be relevant to a build backend… and determine at what scopes those settings are relevant (globally, per command, other?). From there, we can create a scheme, something simple, maybe namespaced, to accept those parameters through config_settings.

jaraco on Mar 21, 2021