pip: Unicode decode error while writing log output
Suppose I install virtualenv 1.10.1 with pip 1.4.1 on a Debian system with Python 2.7.4, but without the python-dev library. Therefore, I will be unable to build the numpy package because I do not have the required Python development headers.
This is how you reproduce the issue:
$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.10.1.tar.gz
$ tar xzvf virtualenv-1.10.1.tar.gz
$ python virtualenv-1.10.1/virtualenv.py test
$ source test/bin/activate
$ pip install numpy
However, while testing pip with TUF, instead of seeing the appropriate failure message, we eventually see something like:
   out = check_types(*a, **kw)
  File "numpy/core/setup.py", line 271, in check_types
    "Cannot compile 'Python.h'. Perhaps you need to "\
SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel.
----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/test/build/numpy
Traceback (most recent call last):
  File "/tmp/test/bin/pip", line 9, in <module>
    load_entry_point('pip==1.4.1', 'console_scripts', 'pip')()
  File "/tmp/test/local/lib/python2.7/site-packages/pip/__init__.py", line 148, in main
    return command.main(args[1:], options)
  File "/tmp/test/local/lib/python2.7/site-packages/pip/basecommand.py", line 169, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 72: ordinal not in range(128)
Changing line 169 to this fixes the issue:
text = '\n'.join(s.decode('utf-8') for s in complete_log).encode('utf-8')
However, I am not sure that that is the appropriate fix. Furthermore, it probably does not port well into Python 3. It is probably best to fix the issue with a solution that works across all supported Python versions.
About this issue
- Original URL
 - State: closed
 - Created 11 years ago
 - Comments: 38 (19 by maintainers)
 
Links to this issue
Commits related to this issue
- Fix for unicode in the log on Python 2 This won't work on Python 3, and is probably not required there at all. It makes sure that all entries from complete_log are bytestrings when join()ing, before... — committed to remram44/pip-datelimit by remram44 11 years ago
 - Fix for unicode in the log on Python 2 This won't work on Python 3, and is probably not required there at all. It makes sure that all entries from complete_log are bytestrings when join()ing, before... — committed to remram44/pip-datelimit by remram44 11 years ago
 - Merge pull request #2012 from remram44/unicode-log_v3 Fix attempt for unicode bug #1137 (v3) — committed to pypa/pip by dstufft 10 years ago
 - Fix for unicode in the log — committed to remram44/pip-datelimit by remram44 10 years ago
 
Is it fixed already…?