MSYS2-packages: [BUG] Wrong charset after upgrade to msys2-runtime-3.1.4

Steps to Reproduce the Problem

python -c 'print("世界")' 荳也阜

Additional Context

locale LANG=en_US.UTF-8 LC_CTYPE=“en_US.UTF-8” LC_NUMERIC=“en_US.UTF-8” LC_TIME=“en_US.UTF-8” LC_COLLATE=“en_US.UTF-8” LC_MONETARY=“en_US.UTF-8” LC_MESSAGES=“en_US.UTF-8” LC_ALL=

The Following Methods Can All Get Correct Result

Downgrade msys2-runtime to 3.0.7
winpty python -c 'print("世界")'
python -c 'print("世界")' | iconv -f utf-8 -t utf-8
chcp.com 65001 && python -c 'print("世界")'

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 2
Comments: 49 (30 by maintainers)

Commits related to this issue

fhandler_pty_slave::setup_locale: respect charset == "UTF-8" When `LANG=en_US.UTF-8`, the detected `LCID` is 0x0409, which is correct, but after that (at least if Pseudo Console support is enabled), ... — committed to dscho/msys2-runtime by dscho 4 years ago

Most upvoted comments

Can we just skip the conversion in the non-pcon case, so that we get the old behavior without pcon while still being able to enable pcon and get the unchanged upstream behavior?

That’s exactly what reverting this commit means 😉

dscho on Sep 3, 2020

@dscho Сonfirm. Still the same error

Archer73 on Aug 30, 2020

I just tried Cygwin’s MinTTY with CYGWIN=disable_pcon and the prompt is already broken, it does not echo any command when navigating the command line history.

Okay, Cygwin v3.1.5 seems to have fixed MinTTY (it was in their release notes that a segfault was fixed). With this, I can confirm that it actually is Cygwin that broke non-pseudo console mode in MinTTY:

$ msg="υπολογιστή"; echo $msg; cmd /c echo $msg
υπολογιστή
?π?????στ?

dscho on Jun 2, 2020

$ msg="υπολογιστή"; echo $msg; cmd //c echo $msg
υπολογιστή
?π?????στ?

I can reproduce this. And MSYS=enable_pcon fixes things. So maybe cygwin broke the non conpty fallback case which we default to

lazka on May 26, 2020

@nyfair, your original post mentions various methods to get the correct result, but doesn’t say if you’re running mintty in the failing case. Other useful information would be the result of command -v python and output of chcp.com.

elieux on May 25, 2020