pip: Pip 19.1 pip install --no-use-pep517 not working with nested requirements files

Asking people to type the obscure --no-use-pep517 (should people even have to know what PEP 517 is?) when the well-known pip install -e used to work is not a sane workaround. I’m astonished that the pip maintainers estimate it’s fine to break well-known practices just out of theoretical purity.

Of course, there’s still python setup.py develop…

So I get now pip is PEP compliant, but this now breaks peoples workflow without any sane workaround, I consider it as a PEP oversight. PEP-517/8 on purpose did not want to cover editable installs, considering something to be addressed down the line. As such I would argue PEP-517/PEP-518 no longer applies when editable install mode is enabled. It’s the build frontend dependent on how this case is handled for now. In such a case, pip should fallback to the old system, until we define editable installs inside a PEP. @pfmoore want to weigh in? If we want people to explicitly require the old behaviour that’s okay, but then --no-use-pep517 should work independently if the pyproject.toml is there or what it contains.

Right, I wouldn’t mind having to add a new option (though I’m not sure why -e couldn’t imply it automatically), but --no-use-pep517 reminds me of --single-version-externally-managed, only worse 😃

I personally like the idea of saying -e means legacy setup.py develop, and later introducing a new option (--inplace?) when it becomes standardized.

@tom-dalton-fanduel A couple hours ago I posted https://github.com/pypa/pip/pull/6447 to permit using --use-pep517 and --no-use-pep517 inside requirements files (fixing #6438). In your case, this would let you add --no-use-pep517 before -e . inside your requirements/base.txt. Will this address your issue?

I think putting --no-use-pep517 at your top-level invocation isn’t what we’d want to get working because that would apply to all requirements in all requirements files, when really you just want it to apply to the one line -e ..

I agree with @pitrou, given that PEP 517 does not define editable installs, asking for one should imply not using PEP 517.

Which will cause problems, when a later PEP specifies how editable installs work with pep517/8. But maybe the later “editable install” should then be called something different, like “source-install”, “develope-install” or “repo-install” and not use the -e, --editable flag but rather its own.

@tom-dalton-fanduel You can test the #6447 with pip install https://github.com/cjerdonek/pip/archive/requirements-file-no-use-pep517.zip

I tried the new version and it still failed… but then I realised I forgot to add the new option to my _base requirements. I added it, and:

~/repos/sudobot pip-fix-test (sudobot) › pip install -r src/requirements/development.txt 
...
Successfully installed freezegun-0.3.11 sudobot
~/repos/sudobot pip-fix-test (sudobot) › 

🎉

@tom-dalton-fanduel You can test the #6447 with pip install https://github.com/cjerdonek/pip/archive/requirements-file-no-use-pep517.zip

Regarding https://github.com/pypa/pip/issues/6433#issuecomment-486173101, I filed #6438 which seems like a legitimate issue.

I agree with @pitrou, given that PEP 517 does not define editable installs, asking for one should imply not using PEP 517.

@tom-dalton-fanduel Have you tried adding the option inside the appropriate requirements file, instead of only at the root command?

I had great hopes but it failed spectacularly!

Running virtualenv with interpreter /usr/bin/python3.6
pip 19.1 from /var/lib/jenkins/workspace/EC2_SQS_Queue_Deletion/sqs-deletion/venv/lib/python3.6/site-packages/pip (python 3.6)
pip install -q --upgrade --requirement /var/lib/jenkins/workspace/EC2_SQS_Queue_Deletion/sqs-deletion/src/requirements/development.txt
Usage: pip [options]

ERROR: Invalid requirement: --no-use-pep517 -e .
pip: error: no such option: --no-use-pep517

make: *** [/var/lib/jenkins/workspace/EC2_SQS_Queue_Deletion/sqs-deletion/_build/requirements.txt] Error 1