pip: Pip fails to find its own Exception file (21.1)

Also ran into this issue when restoring a ${{ env.pythonLocation }} cache in a Github Action workflow. This isn’t a pip issue, but naively looks like one, so I’ll post a potential solution for others who may end up here like I did.

While a cache key with ${{ env.pythonLocation }} captures the Python version, the setup-python action may provide upgraded versions of Python or pip. This recent pip upgrade plus an old cache seems to result in a broken pip install.

Instead of caching the whole Python dir, try just caching installed packages while excluding pip and setuptools:

    - uses: actions/cache@v2
      with:
        # Cache the Python package environment, excluding pip and setuptools installed by setup-python
        path: |
          ~/.cache/pip
          ${{ env.pythonLocation }}/bin/*
          ${{ env.pythonLocation }}/include
          ${{ env.pythonLocation }}/lib/python*/site-packages/*
          !${{ env.pythonLocation }}/bin/pip*
          !${{ env.pythonLocation }}/lib/python*/site-packages/pip*
          !${{ env.pythonLocation }}/lib/python*/site-packages/setuptools*
          !${{ env.pythonLocation }}/lib/python*/site-packages/wheel*

This “works on my ~machine~ GIthub workflow.”

Including ~/.cache/pip helps address spuriously-excluded packages like pip-tools, while still shortcutting the installation of other packages.

Hopefully this doesn’t break when pip is upgraded by setup-python, and isn’t missing any directories that could result in partially-installed packages…

Edit: Per @uranusjr, added an exclusion for wheel. It’s not currently installed by Github’s setup-python action under Ubuntu, but may be in the future.

And to reiterate the bigger idea here: regardless of Github or CI, installing Python then overwriting parts of it with a previously-saved cache can break your environment. The “solution” above works today, but may not be reliable.

In our yaml file for Github Actions, we had cached our Python environment. Commenting out the cache options entirely fixed the issue as it re-downloaded the entirety of pip. I’m leaving this issue open, as it seems like this should still be investigated.

InvalidSchemeCombination is new in 21.1, and it’s correctly declared: https://github.com/pypa/pip/blob/21.1/src/pip/_internal/exceptions.py#L70

So I’m inclined to think this is caused by a botched update. I would try viewing /opt/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/site-packages/pip/_internal/exceptions.py and see if it matches the 21.1 release. If it does not, you can try re-installing pip with get-pip.py.

I just search --help for some reinstall option and use --ignore-installed accidently, after replacing --ignore-installed with --force-reinstall this issue is fixed

My guess is that other people may have stumbled across -I in maybe the same way? (old stuff via other distro mgr, using old pip’s help, and ignoring new pip’s help).

And I see at least pip==21.1.0 echoes your sentiment:

  -I, --ignore-installed      Ignore the installed packages, overwriting them.  
                              This can break your system if the existing        
                              package is of a different version or was          
                              installed with a different package manager!

If you’re doing -I (--ignore-installed) you’re basically overriding all protections, and if things break you’re on your own. I don’t see anything actionable here. You really should not (and IMO don’t have a reason to) use -I in the script.