pip: PEP 518 modifications required for PEP 517

I’ve removed some modifications from gh-4799 that aren’t specifically required for PEP 518 compliance, but are an implementation detail that will greatly ease and simplify PEP 517. I plan to submit a separate PR with these modifications which should be merged after 10.0 is released, but I thought I’d start the discussion about these changes here. Please try to keep the discussion focused on this specific issue, because broader discussions have been shown to be less-than-productive.

Premise: To implement PEP 517 support, pip must run all setup.py scripts with a newer version of setuptools. This is a design decision that will simplify implementation, but I prefer not to discuss it in this specific issue (if this really needs to be discussed, and I think arguing about this point is a waste of time, then let’s open another issue).

Question: How can we run all setup.py scripts with the latest setuptools while preserving backwards compatibility?

Well, we can’t isolate projects without a pyproject.toml due to backwards compatibility concerns. However, what we can do is create a subclass of BuildEnvironment that prepends the newer setuptools to the PYTHONPATH so that it’s loaded before the older setuptools.

In addition to these concerns, implementing this will imply the following points:

• pip will no longer check for setuptools to be installed system-wide, because it will already install it before calling setup.py.
• pip will always require a setuptools and wheel package to be available in the index. This is not normally a problem, but it does mean that we need to modify the tests so that pip can always access a setuptools and wheel package.