pip: Pip install block waiting forever for a keyring to unlock

Environment

  • pip version: 20.0.2
  • Python version: 3.7.6
  • OS: Fedora

jaraco/keyring: v20.0.1 (automatically installed as PIP dependency)

Description

Pip block during install, waiting for a keyring to unlock (at least that’s my understanding)

Expected behavior

Pip install should proceed. I’m also confused about why pypi try to use any keyring to access pypi.org (See in the verbose output Getting credentials from keyring for https://pypi.org/simple).

How to Reproduce

pip install -vvv --no-deps --no-cache-dir -U importlib-metadata==1.1.3

Workaround Disable the keyring

PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring
pip install -vvv --no-deps --no-cache-dir -U importlib-metadata==1.1.3

Likely related issue: https://github.com/pypa/pip/issues/6773

Output

Non-user install because user site-packages disabled
Created temporary directory: /tmp/pip-ephem-wheel-cache-710zevvx
Created temporary directory: /tmp/pip-req-tracker-1h_3qnhb
Initialized build tracking at /tmp/pip-req-tracker-1h_3qnhb
Created build tracker: /tmp/pip-req-tracker-1h_3qnhb
Entered build tracker: /tmp/pip-req-tracker-1h_3qnhb
Created temporary directory: /tmp/pip-install-arqqjx8s
1 location(s) to search for versions of importlib-metadata:
* https://pypi.org/simple/importlib-metadata/
Fetching project page and analyzing links: https://pypi.org/simple/importlib-metadata/
Getting page https://pypi.org/simple/importlib-metadata/
Found index url https://pypi.org/simple
Getting credentials from keyring for https://pypi.org/simple
^CCleaning up...
Removed build tracker: '/tmp/pip-req-tracker-1h_3qnhb'
ERROR: Operation cancelled by user
Exception information:
Traceback (most recent call last):
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
    status = self.run(options, args)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/commands/install.py", line 331, in run
    resolver.resolve(requirement_set)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 281, in _get_abstract_dist_for
    req.populate_link(self.finder, upgrade_allowed, require_hashes)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/req/req_install.py", line 249, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/index/package_finder.py", line 899, in find_requirement
    req.name, specifier=req.specifier, hashes=hashes,
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/index/package_finder.py", line 881, in find_best_candidate
    candidates = self.find_all_candidates(project_name)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/index/package_finder.py", line 826, in find_all_candidates
    project_url, link_evaluator=link_evaluator,
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/index/package_finder.py", line 790, in process_project_url
    html_page = self._link_collector.fetch_page(project_url)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/index/collector.py", line 497, in fetch_page
    return _get_html_page(location, session=self.session)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/index/collector.py", line 337, in _get_html_page
    resp = _get_html_response(url, session=session)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/index/collector.py", line 143, in _get_html_response
    "Cache-Control": "max-age=0",
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/network/session.py", line 405, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 519, in request
    prep = self.prepare_request(req)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 462, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_vendor/requests/models.py", line 317, in prepare
    self.prepare_auth(auth, url)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_vendor/requests/models.py", line 548, in prepare_auth
    r = auth(self)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/network/auth.py", line 202, in __call__
    url, username, password = self._get_url_and_credentials(req.url)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/network/auth.py", line 178, in _get_url_and_credentials
    username, password = self._get_new_credentials(original_url)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/network/auth.py", line 151, in _get_new_credentials
    get_keyring_auth(index_url, username) or
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/pip/_internal/network/auth.py", line 58, in get_keyring_auth
    cred = get_credential(url, username)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/keyring/core.py", line 74, in get_credential
    return _keyring_backend.get_credential(service_name, username)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/keyring/backends/SecretService.py", line 113, in get_credential
    collection = self.get_preferred_collection()
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/keyring/backends/SecretService.py", line 60, in get_preferred_collection
    collection.unlock()
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/secretstorage/collection.py", line 67, in unlock
    return unlock_objects(self.connection, [self.collection_path])
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/secretstorage/util.py", line 154, in unlock_objects
    dismissed, (signature, unlocked) = exec_prompt(connection, prompt)
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/secretstorage/util.py", line 140, in exec_prompt
    connection.recv_messages()
  File "/home/riquito/piggy_store/.env/lib64/python3.7/site-packages/jeepney/integrate/blocking.py", line 64, in recv_messages
    b = unwrap_read(self.sock.recv(4096))
KeyboardInterrupt

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 19
  • Comments: 22 (2 by maintainers)

Commits related to this issue

Most upvoted comments

Any solution to this? I have the same issue after a reboot on Ubuntu 18.04

Not a solution, but I as a workaround have added the following to my ~/.bashrc

# 1-May-2020: Fix for Keyring error with pip. Hopefully new pip will fix it
# soon https://github.com/pypa/pip/issues/7883
export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring

I had the problem on WSL2 that DISPLAY was set, but there was no X server running. This caused pip to hang on the keyring. The fix for me was simply:

unset DISPLAY

Well, I’ll add one to the mix for you all. I’m running ansible on wsl2 and just wanted to upgrade to the latest version, which requires python3 with pip3.

Pip3 fails to do anything even after minutes and I need to keyboardinterrupt it. Then went searching and entered the rabbithole of all the options to disable keyring, which all did not work (because apparently all the backend config options listed in related issues like #7883 ) are ignored on wsl2 or not processed correctly) .

I ended up doing what any man with not enough time for this would do: I monkey patched keyring on my local system:

$ sudo nano /usr/lib/python3/dist-packages/keyring/core.py Comment out the last line:

image

Result: Pip is blazing again and I can move on with my life.

Is there anyone who could explain why was this odd behavior introduced? Or link to a document explaining it? It’s really annoying to know that a tool so core to the ecosystem can be messed up this way this easily.

I had the problem on WSL2 that DISPLAY was set, but there was no X server running. This caused pip to hang on the keyring. The fix for me was simply:

unset DISPLAY

This was it for me. I was doing some display forwarding over an X server to run Qemu using WSL and forgot I had display set, but did not have the display server running!

Getting hassled for a password just to install a package when no password is needed was annoying, but thankfully only required a few stabs of the ESC key to make it keep going.

Today, it starts blocking indefinitely, waiting for something, and when I hit ^C, sure enough, it’s blocking on a passphrase to unlock a keyring it doesn’t need, to access a password that doesn’t exist.

The problem has been known of for more than a year (see issue #6773), why is this this crazy behaviour the default?

having the same issue here, can’t understand why keyring is needed on install

I’m also having this on ArchLinux under i3; I reported it at #9391 before being pointed here (thanks for correcting me!). I have screenshots:

2020-12-28-203931_1366x768_scrot

Plus, it particularly gets in the way if I’m working on a real tty (e.g. Ctrl-Alt-F3) instead of an GUI terminal emulator. There, pip hangs but doesn’t prompt because the prompt is open under X11 and I don’t see it – and pip can’t proceed – until I switch (Ctrl-Alt-F7) and cancel it there.

FYI: https://github.com/pypa/pip/pull/8687 should have made pip less eager in trying to use keyring. We’re also considering making keyring support in pip into an opt-in: https://github.com/pypa/pip/issues/8719

I ended up just removing the keyring package- it was only installed because it’s a dependency of Twine and I’m installing most of my packages via git+https://… anyway

I’m not sure if twine will work without keyring but unsetting my DISPLAY didn’t work 😕

Any solution to this? I have the same issue after a reboot on Ubuntu 18.04

Ah, I didn’t notice that #8687 didn’t close this. Thanks for the reminder of that @Cimbali! ^>^

My workaround in https://github.com/pypa/pip/issues/7883#issuecomment-753242602 has stopped working.

$ cat /etc/pam.d/lightdm
#%PAM-1.0
auth        include     system-login
#-auth       optional    pam_gnome_keyring.so
account     include     system-login
password    include     system-login
session     include     system-login
#-session    optional    pam_gnome_keyring.so auto_start

But gnome-keyring is back:

$ pstree -ap 
systemd,1
[...]
  ├─systemd,632 --user
  │   ├─(sd-pam),633
[...]
  │   ├─dbus-daemon,647 --session --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
[...]
  │   ├─gnome-keyring-d,2006 --start --foreground --components=secrets
  │   │   ├─{gnome-keyring-d},2007
  │   │   ├─{gnome-keyring-d},2008
  │   │   └─{gnome-keyring-d},2009
[...]

Chromium is not triggering the gnome-keyring prompt like it was before – which is why I tried to disable it completely instead of only solving it for people. But now the only thing I’ve found so far that triggers it is pip.

I have these:

$ pacman -Qi gnome-keyring
Name            : gnome-keyring
Version         : 1:3.36.0-3
$ pacman -Qi python-pip
Name            : python-pip
Version         : 20.3.1-1
$ # AKA:
$ pip --version 
pip 20.3.1 from /usr/lib/python3.9/site-packages/pip (python 3.9)

I’ve switched to @JohnVillalovos’s solution (https://github.com/pypa/pip/issues/7883#issuecomment-643319919) and that seems to be working for me.

Add me to the list that has wasted HOURS tracking this down. Thanks for the workaround

Same here, Debian-Testing + KDE. It is really strange (MS Windows style) behaviour, especially from pipenv. Keyring reports “Some application requires…” instead of “pip requires…”. The modal dialog causes that everything hangs, it has 2 options (Cancel/Open). With Cancel the dialog repeats immediatelly. Please prevent this.

In case it helps someone, I have a different workaround than https://github.com/pypa/pip/issues/7883#issuecomment-643319919, which also avoids other apps, like Chrome, from nagging me in the same way: my problem stems from Arch packaging lightdm (and gdm, lxdm, and ssdm) with gnome-keyring built in:

https://github.com/archlinux/svntogit-packages/blob/8d85fa6b8d07e4a82b24efc7635853110c2f93ed/trunk/lightdm.pam#L7

which is why logging in via lightdm causes it to spawn:

$ pstree -ap
systemd,1
  |-accounts-daemon,586
[...]
  |-systemd,819 --user
[...]
  |   |-gnome-keyring-d,2736 --start --foreground --components=secrets
  |   |   |-{gnome-keyring-d},2737
  |   |   |-{gnome-keyring-d},2738
  |   |   `-{gnome-keyring-d},2739

and pip to print that prompt and leave this “WARNING: keyring …”:

$ pip install markdown2
Defaulting to user installation because normal site-packages is not writeable
WARNING: Keyring is skipped due to an exception: Failed to unlock the collection!
Collecting markdown2
  Downloading markdown2-2.3.10-py2.py3-none-any.whl (34 kB)
Installing collected packages: markdown2
Successfully installed markdown2-2.3.10

Once I found this, getting rid of it was easy:

--- /etc/pam.d/lightdm.orig 2020-12-31 21:19:10.151609312 -0500
+++ /etc/pam.d/lightdm	2020-12-31 21:12:17.601523586 -0500
@@ -1,7 +1,7 @@
 #%PAM-1.0
 auth        include     system-login
--auth       optional    pam_gnome_keyring.so
+#-auth       optional    pam_gnome_keyring.so
 account     include     system-login
 password    include     system-login
 session     include     system-login
--session    optional    pam_gnome_keyring.so auto_start
+#-session    optional    pam_gnome_keyring.so auto_start

that plus rebooting successfully disabled pip bothering me:

$ pip install markdown2
Defaulting to user installation because normal site-packages is not writeable
Collecting markdown2
  Using cached markdown2-2.3.10-py2.py3-none-any.whl (34 kB)
Installing collected packages: markdown2
Successfully installed markdown2-2.3.10

And I second being very

confused about why pypi try to use any keyring to access pypi.org

I just got this on Debian Testing, gnome.

🐚 pip install --verbose poetry==1.0.9                                                                                 
Non-user install because user site-packages disabled
Created temporary directory: /tmp/pip-ephem-wheel-cache-9bx45e78
Created temporary directory: /tmp/pip-req-tracker-lu2srv1l
Initialized build tracking at /tmp/pip-req-tracker-lu2srv1l
Created build tracker: /tmp/pip-req-tracker-lu2srv1l
Entered build tracker: /tmp/pip-req-tracker-lu2srv1l
Created temporary directory: /tmp/pip-install-t3llvdta
1 location(s) to search for versions of poetry:
* https://pypi.org/simple/poetry/
Fetching project page and analyzing links: https://pypi.org/simple/poetry/
Getting page https://pypi.org/simple/poetry/
Found index url https://pypi.org/simple
Getting credentials from keyring for https://pypi.org/simple

This doesn’t happen with a fresh environment, though. Just one with poetry installed.

🐚 pip freeze                                                                                                          
attrs==20.2.0
CacheControl==0.12.6
cachy==0.3.0
certifi==2020.6.20
cffi==1.14.2
chardet==3.0.4
cleo==0.7.6
clikit==0.4.3
cryptography==3.1
html5lib==1.1
idna==2.10
jeepney==0.4.3
jsonschema==3.2.0
keyring==20.0.1
lockfile==0.12.2
msgpack==1.0.0
pastel==0.2.0
pexpect==4.8.0
pkginfo==1.5.0.1
poetry==1.0.10
ptyprocess==0.6.0
pycparser==2.20
pylev==1.3.0
pyparsing==2.4.7
pyrsistent==0.14.11
requests==2.24.0
requests-toolbelt==0.8.0
SecretStorage==3.1.2
shellingham==1.3.2
six==1.15.0
tomlkit==0.5.11
urllib3==1.25.10
webencodings==0.5.1

(Probably the addition of the non-default keyring package.)