tox: Add support for PEP-517

Currently tox does not yet appear have support for the buildsystem.requires key of pyproject.toml as defined in PEP-518.

PIP has implemented this: https://github.com/pypa/pip/pull/4144 (will be part of PIP 10.0) and it would be nice if tox could also add support for installing packages defined using that key, instead of having to use the broken setup_requires key of ./setup.py.

Example of a pyproject.toml file that creates a setuptools 34+ environment:

[build-system]
requires = ["setuptools >= 34", "wheel"]

Example of a pyproject.toml file that creates an environment for building flit packages:

[build-system]
requires = ["flit"]

(Note that this kind of setup is pretty useless without support for PEP-517.)

Approximate step by step algorithm for supporting both PEP-517 and “legacy” ./setup.py deployments (based on PEP-518 & how PIP choose to implement it):

  1. If skipsdist in the tox section of tox.ini is true, do nothing
  2. Create a new virtualenv to host the sdist generation environment
  3. If a file named pyproject.toml exists in the toxinidir:
    1. Parse the pyproject.toml as TOML file (using the python toml library for instance)
    2. Search for the build-system.requires key and treat each entry in the array as an PEP-508 dependency specification
    3. If the key was missing or not an array, report an informative error message and exit
    4. Install each found build dependency into the virtualenv
  4. Otherwise install setuptools and wheel into the virtualenv
  5. Execute ./setup.py sdist within the virtualenv to generate the sdist used for the subsequent testing steps

(The build environment may be reused between subsequent testing sessions, as long as the list of requirements doesn’t change.)

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 27 (15 by maintainers)

Commits related to this issue

Most upvoted comments

There’s also a pep517 module providing an API for tools to go from a source tree to an sdist/wheel using the PEP 517 interface. I wrote that in the hope that it would make it easier for pip to add support for that interface.

Drawing from the experience implementing this in toxn (and with pip 10 coming out with PEP-518 support too), I’ll add support for this in the next month, drawing up and during the PyCon sprint.

@warsaw no support at the moment, however I plan to add it at some point 👍

Hi @takluyver, thanks for pointing that out. It will be best to have a close eye on the developments in pip and follow their lead then whatever they will use for parsing toml.

TOML libraries: I had a brief look at the code of each a few months back, and at the time I felt more confident in pytoml than in toml. Consequently, pytoml is the one which is vendored into pip, and which will become a dependency of flit once PEP 517 is finalised.

The developer of pytoml did not express great enthusiasm for ongoing maintenance, IIRC. But if it’s used in pip, there’s an incentive for other people to take over the maintenance if it becomes necessary.

@obestwalter: Only insofar as that they respect the tool namespace collision avoidance rules of PEP-518:

All other top-level keys and tables are reserved for future use by other PEPs except for the [tool] table. Within that table, tools can have users specify configuration data as long as they use a sub-table within [tool], e.g. the flit tool would store its configuration in [tool.flit].

So one could probably make up an interesting legal case about whether that already entailes “using” PEP-518 or not 😉.