serverless-python-requirements: 4.2 re-orders requirements.txt before installing, breaking extra pypi index
We have some packages in a secondary pypi server so our requirements.txt looks something like:
-i https://pypi.org/simple
--extra-index-url https://repo.fury.io/techadmin/
amazon-kclpy==1.5.0
argh==0.26.2
...
A recent change this package to pip.js seems to re-order this:
https://github.com/UnitedIncome/serverless-python-requirements/blob/master/lib/pip.js#L287
to
--extra-index-url https://repo.fury.io/techadmin/
-i https://pypi.org/simple
amazon-kclpy==1.5.0
argh==0.26.2
Now, this works fine with pip install, but when serverless tries to install with this file it can’t find anything in the repo.fury.io index.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 17
Commits related to this issue
- Bugfix for requirements.txt options ordering (#237) Fixes #236 @vickeryj Can you please try this? ``` rm -Rf node_modules/serverless-python-requirements npm i github:andrewfarley/serverless-p... — committed to serverless/serverless-python-requirements by AndrewFarley 6 years ago
- Adding -f and -i to requirements.txt options ignore list before sorting (#238) Fixes: #236 (again, properly) Problem: requirements.txt options which must be in the proper order at the start of the... — committed to serverless/serverless-python-requirements by AndrewFarley 6 years ago
PR: #481
@paddycarey Definitely the same/similar error… you can fix this right now by changing
-ito--index-urlor give me a few minutes and I’ll have another MR for you.Side-note, I’m going to go ahead and add -i and -f to the ignore list, found a good link to all header/options…
https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format
No problem, I fully expect a few tweaks/bugs here and there since it was a rather large change. I’ll do my best for quick turnaround, help keep everyone working happily.
@vickeryj Can you please try my MR?
Please note: you will need to change your requirements.txt file to be…
So, as far as I can tell, the requirement.txt format guide says that all internal options should be prefixed with – and only at the start of the file, because all “normal” lines (packages to install) may optionally use single line prefix switches (eg -e). I think this is why pip and your config is getting confused. I’m using this logic to “skip” re-ordering any lines that are prefixed with --. In fact, with the above modification to your requirements file you may not even need my branch (please try to confirm). But, I would still like to preserve their order just incase this changes in the future.
And just so you know, this re-ordering/sorting logic is in place so we can detect duplicate requirements.txt files that may contain different comments, or ordering of packages but still actually mean the same thing because the order of packages in a requirements file doesn’t matter.
It makes sense to skip trying to order these header/options type commands since pip requires them to be at the top, however, you would hope/assume that pip would not care about their order, eh? Honestly, this seems like a pip bug. Bug anyways, try my slight modification mentioned above on the branch I specified above and see if that fixes it for you and let me know.
Oh and sorry I couldn’t get around to it yesterday! 😦 Ran out of time, needed to sleep