pip: Empty and missing environment variables in requirements.txt are not substituted
Environment
- pip version: 20.1.1
- Python version: 3.7.7
- OS: Ubuntu 18.04
Description
Consider you want to install TensorFlow package that comes with 3 variants:
tensorflowtensorflow-gputensorflow-cpu
Depending on the environment, you want to install either CPU or GPU variant, or the main package by default. So you can add a line in requirements.txt like tensorflow${TF_VARIANT} and it works great if TF_VARIANT is defined and not empty.
However, when the variable is undefined or empty, I get the following error:
ERROR: Invalid requirement: 'tensorflow${TF_VARIANT}' (from line 1 of requirements.txt)
Expected behavior
An empty or missing variable should be substituted with an empty string.
About this issue
- Original URL
- State: open
- Created 4 years ago
- Comments: 16 (10 by maintainers)
My feeling (as a user mainly developing on Windows) to this is entirely the other way around: If you require Windows users to substitute an environment variable with an empty string, your setup is already broken and should be fixed. It is entirely okay to expect the ability to substitute variables with empty strings if you only develop for POSIX users (e.g. in a corporate setting where user platforms are expected), but pip should not step out of the line to “fix” your inability to develop software with cross-platform compatibility.
If this is a welcoming change, I’d be happy to create a PR for it. 😃
OTOH, I don’t think anyone would expect the Windows behavior, because Pip doesn’t even allow the Windows
%VAR%syntax 😃As pointed out in the spec: https://pip.pypa.io/en/stable/reference/pip_install/#using-environment-variables
EDIT:
So let’s not make any difference in behavior between empty and missing variables.
Bike-shedding a little: we can actually use
os.getenv(var_name, ''). IMHO this use case should be supported because the behavior expected here is the default as defined by POSIX, unless-uis set.In any case, replacing empty variables has to be done. It is arguably a bug.
What to do wrt missing variables is debatable. @adampl looking at other tools could indeed shed light on the matter. It could also be interesting to look at the pip history to see if there was a specific reason for not replacing them.
I think pip._internal.req.req_file:expand_env_variables is what processes the environment variables in requirement files.
The documentation for it is provided at https://pip.pypa.io/en/latest/reference/pip_install/#using-environment-variables