pip: Problems on Windows with username or hostname containing non-ASCII characters
When organizing a Python training recently, one participant failed to use pip after a fresh Python 2.7.11 installation on Windows. Quick investigation showed that the reason problem was ä
in her username. We failed to workaround that even by creating a new account and needed to use python setup.py install
instead.
I now tried to reproduce the problem on my virtual machine. My main account there has only ASCII characters in the username but I created another for testing purposes. Clearly everything is not correct:
C:\Users\Ürjö>pip
Traceback (most recent call last):
File "c:\python27\lib\runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "c:\python27\lib\runpy.py", line 72, in _run_code
exec code in run_globals
File "C:\Python27\Scripts\pip.exe\__main__.py", line 9, in <module>
File "c:\python27\lib\site-packages\pip\__init__.py", line 210, in main
cmd_name, cmd_args = parseopts(args)
File "c:\python27\lib\site-packages\pip\__init__.py", line 165, in parseopts
parser.print_help()
File "c:\python27\lib\optparse.py", line 1670, in print_help
file.write(self.format_help().encode(encoding, "replace"))
File "c:\python27\lib\optparse.py", line 1650, in format_help
result.append(self.format_option_help(formatter))
File "c:\python27\lib\optparse.py", line 1633, in format_option_help
result.append(group.format_help(formatter))
File "c:\python27\lib\optparse.py", line 1114, in format_help
result += OptionContainer.format_help(self, formatter)
File "c:\python27\lib\optparse.py", line 1085, in format_help
result.append(self.format_option_help(formatter))
File "c:\python27\lib\optparse.py", line 1074, in format_option_help
result.append(formatter.format_option(option))
File "c:\python27\lib\optparse.py", line 316, in format_option
help_text = self.expand_default(option)
File "c:\python27\lib\site-packages\pip\baseparser.py", line 112, in expand_default
return optparse.IndentedHelpFormatter.expand_default(self, option)
File "c:\python27\lib\optparse.py", line 288, in expand_default
return option.help.replace(self.default_tag, str(default_value))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xdc' in position 9: ordinal not in range(128)
Interestingly installation and uninstallation seem to work fine also with this account. I guess the difference with the problem I saw earlier could be that my main user/admin doesn’t have non-ASCII characters.
UPDATE: It later turned out that pip install
is totally broken if the hostname has non-ASCII characters. That explains why creating a new account with just ASCII characters in the username didn’t didn’t work when I encountered this first time and also why I couldn’t reproduce that more severe problem with just an account with non-ASCII username.
A workaround for both of these problems is using --no-cache-dir
. Both problems are also fixed by PR #3970 that hopefully gets merged and released at some point.
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 39 (34 by maintainers)
Commits related to this issue
- Fix pip when using Py2 on Windows with non-ASCII hostname or username. Two related problems are fixed: - Previously non-ASCII characters in hostname blew up `pip install` completely. This is rather... — committed to pekkaklarck/pip by pekkaklarck 8 years ago
- Test for issues #3970/#3463 — committed to pfmoore/pip by pfmoore 8 years ago
PR #3970 fixes both the more severe problem with
pip install
crashing if hostname has non-ASCII characters, and the less severe but annoying crash withpip --help
if username has non-ASCII characters.Yes,
--cache-dir
seems to be causing this. I debugged the problem on my machine (see the original description) and noticed that it crashes when formatting help text for exactly that option. Looking at the pip and optparse code I think I found the root cause:--cache-dir
based on whatpip.utils.appdirs.user_cache_dir
returns. It is a Unicode string and in my case it isu'C:\\Users\\\xdcrj\xf6\\AppData\\Local\\pip\\Cache'
.HelpFormatter.expand_default
(line 288) and usesstr(default_value)
there.Based on this analysis the real bug would be in optparse. Not sure what’s the best way to avoid it on pip side.