pip: Basic auth credentials in --extra-index-urls are not used when specifying multiple under the same domain

Pip version: 8.1.2
Python version: 2.7.9
Operating System: Debian Jessie (uname -a output: Linux ip-10-254-0-170 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt25-2+deb8u3 (2016-07-02) x86_64 GNU/Linux)

Description:

I am trying to use two --extra-index-urls from packagecloud.io (we have two private indices at our office) but when I specify both I get prompted to log in to packagecloud. When I specify one or the other I do not. I think that pip is having trouble handling the basic auth for each index separately.

What I’ve run:

This is the problematic command that I wish would work but does not:

$ pip install --user --upgrade \
    --extra-index-url https://<api token>:@packagecloud.io/2rs2ts/oldrepo/pypi/simple \
    --extra-index-url https://<other api token>:@packagecloud.io/2rs2ts/newrepo/pypi/simple \
    mypackage
Collecting mypackage
User for packagecloud.io:

I can work around it by specifying PyPI as an --extra-index-url and one of my private indices as the --index, which makes me think that this is a bug with --extra-index-url:

$ pip install --user --upgrade \
    --index https://<api token>:@packagecloud.io/2rs2ts/oldrepo/pypi/simple \
    --extra-index-url https://<other api token>:@packagecloud.io/2rs2ts/newrepo/pypi/simple \
    --extra-index-url https://pypi.python.org/simple \
    mypackage
Collecting mypackage
  Downloading https://packagecloud.io/2rs2ts/newrepo/pypi/packages/mypackage-1.0.0-py2-none-any.whl (52kB)
etc. etc.

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 2
Comments: 15 (2 by maintainers)

Most upvoted comments

@2rs2ts @danriti and @kahnjw

I believe the problem is because of how Pip caches the basic auth credentials for index urls: https://github.com/pypa/pip/blob/master/pip/download.py#L160-L162.

It appears that basic auth credentials are cached on the top level domain name of the URL, not the full URL. This means if you have more than 1 repository with the same domain name (in your cases packagecloud.io) pip internally will only be able to store a single basic auth username. Pretty sure this is why you can’t authenticate to two different repositories with different basic auth parameters at the same domain name.

I believe a fix to MultiDomainBasicAuth in pip would fix this problem.

ice799 on Jul 13, 2017

See @pradyunsg’s comment. Someone needs to file a PR for this (which will be linked to this issue, so you’ll see the link here when that happens).

In the meantime, while it’s OK if people want to note their interest, more reports of people hitting the issue won’t help progress much, I’m afraid.

pfmoore on Oct 21, 2020

This is extremely annoying. I’m currently trying to do something but it is not really a simple (mostly because once you download the $repo/simple/$package page, the link to the actual tarball does not contains the creds anymore and can be totally different 😱 ).

erdnaxeli on Oct 18, 2019

Got it repo != package. Thank you.

kahnvex on Jul 13, 2017

@kahnjw The URL you’ve pasted is for an entire repository – not a single package. If you upload additional packages to user/privateRepo they will all be accessible at that URL. Please email us at support@packagecloud.io for assistance so we can keep the issues filed against pip relevant to the pip project.

ice799 on Jul 13, 2017