WSL: Unknown encoding: 65001
I am attempting to run docker-compose.exe from Windows in BashOnWindows. Specifically I am running docker-compose.exe --version to verify that it can correctly output data. When doing this I am receiving the following error:
~ # docker-compose.exe --version
Traceback (most recent call last):
File "docker-compose", line 3, in <module>
File "compose\cli\main.py", line 66, in main
File "compose\cli\main.py", line 98, in dispatch
File "compose\cli\docopt_command.py", line 25, in parse
File "compose\cli\docopt_command.py", line 12, in docopt_full_help
File "site-packages\docopt.py", line 575, in docopt
File "site-packages\docopt.py", line 484, in extras
LookupError: unknown encoding: cp65001
Failed to execute script docker-compose
------------------------------------------------------------
I can see that codepage 65001 is the Windows codepage for UTF-8 encoding, but that this is not supported in Unix systems. A solution to this is to run the command chcp.com 437 to change to a different codepage. This solution is however not good as it removes the support for UTF-8 in the terminal.
The issue should be persistent across different Windows applications that attempt to output any text in encoding 65001.
Windows build number: Microsoft Windows [Version 10.0.15063]
About this issue
- Original URL
- State: closed
- Created 10 years ago
- Reactions: 11
- Comments: 25
Nevermind, its bash so I just made a function to pipe the output to cat. Turns out as long as stdout and stderr go to something else before they go to the console, python doesn’t complain.
This issue also causes display issues with Windows programs run from within bash.
See missing/incorrect umlauts below:
My codepage is 850, but the inner cmd.exe’s (ran from bash) is 65001.
Running
chcp 850in the inner cmd fixes it.What is strange though is the following, why is it like that?
My workaround for this: Create a file at /usr/bin/docker-compose and put this in:
now I can run docker-compose from Bash on Ubuntu on Windows without any encoding issues.
I run such thing using custom linux executables that contain code like this:
cmd.exe /c "chcp 850 >> nul && docker-compose $@"The same thing goed for multipe python based apps
I would really recommend using Linux native clients for docker and docker-compose, they are easy to install and it will keep you from many problems with executing windows natives in WSL.
OS: MS Windows 10 Pro - Version: 10.0.15063 Build 15063 - Bash on Ubuntu on Windows (WSL) Docker CE for Windows - Version 17.06.0-ce-win19 (12801) - Channel: stable c98c1c2 ConEmu console emulator: 161206 [64] {Stable}
To determine what ‘active code page’ the standard MS Windows CMD console is using, open an MS Windows CMD prompt, then type:
chcpMy machine (British English) returned the code for Multilingual (Latin I):
Active code page: 850After discovering the current ‘active code page’ is 850 in the ConEmu console emulator type:
cmd.exe /c chcpThis verified the current ‘active code page’ the ConEmu console emulator is using to be:
Active code page: 65001To set the ‘active code page’ to be 850 in the ConEmu console emulator type;
cmd.exe /c chcp 850Which responds with;
Active code page: 850Finally the command…
docker-compose -version…responds as expected with;
docker-compose version 1.14.0, build c7bdf9e3
Although, I haven’t yet worked out how to make this permanent as I expect the next time I restart I will have to repeat the previous steps?@sunjoong correct the change was not permanent.
Permanently apply this setting to all future bash sessions by adding the following to your ~/.bashrc file
It seems this causes other problems such as the output of the npm install dependency tree, so it appears the safest way is to change the code page to 850, before running docker-compose and then returning the code page back to the original value.
Does anyone know how to automate this?OK, inspired by this GitHub post: Unicode Display Error (discovered via npm STDOUT) #3819 a workaround is to pipe the output to a string, eg;
docker-compose version | while read line; do echo $line; donewill output…
This is a known bug: GitHub post: Unknown encoding error in Windows with cp65001 #2775.
Hopefully it will be fixed soon.
@sgtoj From my investigations the issue is that the Python installed in WSL does not support the code page 65001. You can see this if you dig around a bit in
/usr/lib/python2.7/encodings/.I am using cmder and the program can be run in the same terminal if I set it to use powershell. It is thus not isolated to
docker-composeand I suspect it will be like this for any program that is compiled using codepage 65001. That leaves it to whether it is a thing that should be supported in WSL. My opinion on this matter is that it should as one of the features of WSL is running Windows programs in bash.