vscode: Unable to type in Chinese in integrated terminal

so the issue is caused by electron 11.x the event with that keyCode 229 would never fired when typing Chinese for some IME like Sogou IME and Baidu IME.

Well, not exactly. In fact, when you set a listener to keydown event, you’ll find on Electron 9 it’ll fire the event when you complete the inputs, while on newer Electron like 12, keydown event is never fired.

https://user-images.githubusercontent.com/8115912/110315282-ffefc580-8043-11eb-9816-98c79fa4147a.mp4

https://user-images.githubusercontent.com/8115912/110315303-041be300-8044-11eb-82a6-1d9c1de96d9a.mp4

So there are 3 behaviors:

• Microsoft Pinyin will fire keyup, keydown, input event when you input, and the input value is what you input right now
• Rime and QQ Pinyin will fire keyup and keydown event when you input, but input value is your previous input; it won’t fire input event until you complete it, and at this time the input value is the final input
• Sogou Pinyin and Baidu Pinyin will only fire keyup event when you input, and input value is your previous input; it won’t fire input and keydown event until you complete it, and at this time the input value is the final input; and in newer Electron 12, keydown event is never fired.

after 1.52 version, vscode behaves like this, maybe we should find out changes which cause this problem. Some input methods call special system APIs or output text to UI in a different way. but i think vscode should support them if possible.

+100000086

what’s more interesting, the tech support of sogou think that it’s vscode’s issue. but they admitted that they have released a hotfix.🤣

So, the final solution is switch to QQ pinyin or Rime, and do these changes? #115814 (comment)

@Python-37 Yes but not exactly, that’s just a solution, I don’t want to spend time on it for now since I’m still having works to do. At least, for common users, you can try these options, and for advanced users, you can build it by yourself and revert the broken commit (or just use my compiled binary, but I’m not supposed to update it).

---

BTW, I tried to run Xterm in Edge, a “real” browser, and it’s fun that at this time, Sogou Pinyin fires all the events we needed, and Xterm works well. But Baidu IME is still not working, just the same as in Electron.

https://user-images.githubusercontent.com/8115912/110411731-0888e000-80c6-11eb-9f06-9f7449e56d5c.mp4

https://user-images.githubusercontent.com/8115912/110411743-0cb4fd80-80c6-11eb-9a73-ccd60ad35035.mp4

Browser	Events	Sogou Pinyin	Baidu Pinyin	Microsoft Pinyin	Rime (Weasel)	QQ Pinyin	QQ Pinyin (SL)
Electron 9	KeyboardEvent	🔺	🔺	✔	✔	✔	✔
CompositionEvent	✖	✖	✔	✔	✔	✔
Value up-on-date	✖	✖	✔	✖	✖	✔
Xterm Working	✔	✔	✔	✖	✖	✔
Electron 12	KeyboardEvent	✖	✖	✔	✔	✔	✔
CompositionEvent	✖	✖	✔	✔	✔	✔
Value up-on-date	✖	✖	✔	✖	✖	✔
Xterm Working	✖	✖	✔	✖	✖	✔
Edge	KeyboardEvent	✔	✖	✔	✔	✔	✔
CompositionEvent	✔	✖	✔	✔	✔	✔
Value up-on-date	✔	✖	✔	✖	✖	✔
Xterm Working	✔	✖	✔	✖	✖	✔

• KeyboardEvent: ✖ Only keyup | 🔺 Only keyup on input, but will fire keydown when completed | ✔ All events
• CompositionEvent: ✖ No events | ✔ All events
• Value up-on-date: ✖ Won’t update UI input value in realtime when inputing | ✔ Will update UI input value in realtime
• QQ Pinyin (Single-line mode) behaves more like Microsoft Pinyin
• QQ Pinyin and Weasel are not working is mostly because of a css style issue

So I guess Sogou Pinyin maintains a list to determine which mode will be used for specific program, since when using Edge, its behavior is more like Microsoft Pinyin.

So at this time, I think (besides the css style problem) it’s Sogou Pinyin and Baidu IME’s fault. 🤔

@Kingwl I see some repros show ???, but some others simply do nothing.

For everyone have this issue, can you please confirm which behavior you are experiencing?

You may click the 👍 for ???, and 👎 for nothing happens

It’s not only Chinese but also Japanese, Thai, Kazakh, Russian, etc. Even emoji (paste).

I guess this may caused by unicode.

+10086

The terminal can’t input Chinese characters, but the CMD in Windows system can input Chinese normally. The installed version is 1.53. How can I input Chinese

I setup a fresh new Windows 10 VM (20H2, Enterprise, x64, with VirtualBox) and grab some versions of VSCode and IMEs. By testing these various versions of VSCode and IMEs, seems that there are at least 2 reasons that blocks terminal input.

VSCode Version	Sogou Pinyin	Baidu Pinyin	Microsoft Pinyin	Rime (Weasel)	QQ Pinyin
1.51.1	✔	✔	✔	✔	✔
1.52.0	✔	✔	✔	✖	✖
1.52.1	✔	✔	✔	✖	✖
1.53.0	✖	✖	✔	✖	✖
1.54.1	✖	✖	✔	✖	✖
1.55.0-fixed	✖	✖	✔	✔	✔

• Weasel and QQ Pinyin are broken since 1.52.0, and VSCode terminal lets them to input the first word (or in details, follow this rule)
• Sogou Pinyin and Baidu Pinyin are broken since 1.53.0, but VSCode terminal doesn’t accept their inputs.

So if you’re trying to find the broken commit for Sogou Pinyin, you can try git bisect bad 1.53.0 and git bisect good 1.52.1.

Detailed screen recording videos

https://user-images.githubusercontent.com/8115912/110276716-b9cc3f00-800e-11eb-99d0-1adcc0d0b2f9.mp4

https://user-images.githubusercontent.com/8115912/110276709-b20c9a80-800e-11eb-85f1-3e214d47c83e.mp4

https://user-images.githubusercontent.com/8115912/110276713-b5078b00-800e-11eb-87a9-53796e6ef6c7.mp4

https://user-images.githubusercontent.com/8115912/110276715-b8027b80-800e-11eb-9e2d-1cb61f6d1a3a.mp4

So for now, the following IMEs on Windows are having the same problem with integrated terminal:

• Sogou Pinyin
• QQ Pinyin (with default game compatible mode)
• Baidu Pinyin
• Rime (Weasel)

And an IME on Arch Linux (#112978, well yume-chan said it’s not the same problem, but another contributer has marked it as duplicated with this one)? I’ve tried running an Arch Linux with lxde in a VM, but all fcitx IMEs are working fine, maybe it occurs on IBus?

---

To VSCode developers or contributers, the issue has been existed for about 3 months since 1.52.0 released, and from my side I don’t see any progress on it, feeling a bit upset.

If you’re not familiar with these IMEs, at least you can try Rime, it’s an opensource IME, maybe it can helps you to find the problem.

BTW, for the working IMEs, probably they’re also having another issue that both showing input letters on VSCode UI and IME UI.

---

To VSCode end-user, if you’re still having this problem, maybe you can try the following options:

• If you’re using QQ Pinyin, switch to Single-line mode (单行模式), this should work fine.
• Try Microsoft Pinyin, it’s finally usable in Windows 10, without input lag on Windows 8.
• Rollback to any version before v1.52.0, you can download any version you like before Nov 2020.
• Input your text on another window, and copy & paste them into VSCode terminal (well I’ve been used like this for months)
• Start a terminal window directly, or try Windows Terminal, this may also speed up the command with lots of outputs, since the VSCode terminal’s stdio performance is a bit “oof”.

This issue deserves attention because it has hit a large number of Chinese developers. I thought I have to consider downgrading the version of VSCode until this problem has been fixed.

+1

I’ve merged in the fix from xterm.js, thanks all! Please try out insiders on Monday to verify it works.

I also encounter this problem. Any progress?

Sogou Pinyin input method

版本: 1.54.1 (system setup) 提交: f30a9b73e8ffc278e71575118b6bf568f04587c8 日期: 2021-03-04T22:38:31.419Z Electron: 11.3.0 Chrome: 87.0.4280.141 Node.js: 12.18.3 V8: 8.7.220.31-electron.0 OS: Windows_NT x64 10.0.19042

Is there any progress ? I’m sure it’s an urgent issue.

I met same bug, I found that Windows builtin input method can use normally, some other input methods cannot input Chinese. When I met this bug, I thought that it’s because of the input method I am using. BTW, VS Code in GNU/Linux works normally too.

Well, i have done a little more work to investigate it.

In MDN Web Docs: https://developer.mozilla.org/en-US/docs/Web/API/Element/compositionstart_event

The compositionstart event is fired when a text composition system such as an input method editor starts a new composition session. For example, this event could be fired after a user starts entering a Chinese character using a Pinyin IME.

And xterm depends on this web event totally to compose console string by CompositionHelper.

But , you see, Sogou Pinyin wouldn’t fire the compositionstart when input starting.

In Chrome (not chromium or Electron), browser work very well on firing this event with Sogou Pinyin.

What’s more important, i think that xterm will break its design if a series of hacking codes inserted into its source just for solving this problem.

So, maybe it’s necessary to ask the developers or maintainers of electron to focus on the problem,

A web stardard like compositionstart , browsers should follow what it specifies.

maybe IMEs using not good API, can’t make electron (or chromium) to fire the event, but in old versions of electron, they can handle Sogou IME well.

If the browser runtime doesn’t pass any events to xterm after user completed the input, xterm has not chances to fix the problem.

@ZhyMC Yep, so we’ve already dig in deep enough, there are at least two major issue needs to be resolved:

• A css style issue of Xterm that makes input textarea with 0 width, makes QQ Pinyin and Rime not working anymore. This may can be fixed by a simple revert.
• A runtime environment issue of Electron that doesn’t dispatch keydown event when using Sogou Pinyin, so that Xterm cannot measure IME has completed the inputs.

Until you’re willing to dive into how Xterm or Electron works, maybe we should let VSCode’s contributors to complete the rest job, I guess.

Version 1.54.1 is released, but this bug is stiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiill able to reproduced.👿

@heartacker xterm.js 的作者 @Tyriar 认为这是第三方输入法自己的问题，所以把这个 issue 移出了恢复迭代

Is it possible to make it compatible with these IMEs? As based on latest comments, it still happens on Sogou Pinyin (thought I didn’t reproduce it), and I guess Sogou Pinyin, QQ Pinyin and Rime are the top 3 using 3rd-party IMEs in China (at least Sogou Pinyin is definitely the most one), and all of them are well in previous version (1.51-).

I guess the “game compatible mode” uses Input Method Manager (IMM) API while “single line mode” uses Text Services Framework (TSF) API. IMM has been deprecated and TSF is the replacement.

Is this also apply to Linux distributions? Since the original issue I commented is on Arch Linux.

@Reilkay What’s your IME using on Arch Linux? And are you still experiencing the same problem in latest VSCode?

@Reilkay 你在 Arch Linux 上使用的输入法是什么？在最新的 VSCode 下是否仍存在无法在终端输入中文的问题？

is there any progress? I have the same problem

我暂时用了微软拼音输入法，没问题 搜狗输入法不能输入中文

Not one, but multiple third-party IMEs. They all worked fine before 1.52 and still work fine in all other programs and even in code editors after 1.52.

So definitely something has been changed in either Electron, Code or Xterm.js that affected these IMEs.

If you believe it’s those IMES’ fault, at least we need some directions to how we can “fix” them to conform the way you require.

I met same bug, I found that Windows builtin input method can use normally, some other input methods cannot input Chinese. When I met this bug, I thought that it’s because of the input method I am using. BTW, VS Code in GNU/Linux works normally too.

Thanks your reply, I try windows native input methods, it works!

me too

me too

I’ve merged in the fix from xterm.js, thanks all! Please try out insiders on Monday to verify it works.

Checked and confirmed that both QQ Pinyin and Rime (Weasel) work fine on today’s Insider build.

1.54.3 worked

hot update is nb

@ccloli Hahaha, at this time, i think the best solution is that uninstall the Sogou IME and try something else. And revert the commit which causes the css style problem.

hey guys, i think i found the reason.

in /src/browser/input/CompositionHelper.ts of xtermjs/xterm.js

we can find the code snippet:

if (ev.keyCode === 229) {
    // If the "composition character" is used but gets to this point it means a non-composition
    // character (eg. numbers and punctuation) was pressed when the IME was active.
    this._handleAnyTextareaChanges();
    return false;
}

i set up a keydown listener by using DevTools, and print the ev.keyCode when i am typing in words.

By seaching in google, i found that 229 is a special keyCode event fired with that when you typing Chinese or characters of some other languages.

so the issue is caused by electron 11.x (maybe that’s because of its dependency: chromium ) the event with that keyCode 229 would never fired when typing Chinese for some IMEs like Sogou IME and Baidu IME.

index.html

<!doctype html>
  <html>
    <head>
      <link rel="stylesheet" href="../node_modules/xterm/css/xterm.css" />
      <script src="../node_modules/xterm/lib/xterm.js"></script>
    </head>
    <body>
      <div id="terminal"></div>
      <script>
        var term = new Terminal();
        term.open(document.getElementById('terminal'));
        term.write('Hello from \x1B[1;3;31mxterm.js\x1B[0m \r\n$ ')
        term.onData(e => {
          switch (e) {
            case '\r': // Enter
            case '\u0003': // Ctrl+C
              term.write('\r\n$ ');
              break;
            case '\u007F': // Backspace (DEL)
              // Do not delete the prompt
              if (term._core.buffer.x > 2) {
                term.write('\b \b');
              }
              break;
            default: // Print all other characters for demo
              term.write(e);
          }
       });
      </script>
    </body>
  </html>

the bad commit is: 94142bd

Bumping the dependency : Electron to a new version causes the problem,

@ZhyMC If that’s true, at least for Sogou Pinyin, it would be like what’s @yume-chan said:

^ So definitely something has been changed in either Electron, Code or Xterm.js that affected these IMEs.

^ I can’t remember the exact version numbers that had these issues, but maybe Electron is currently using one of those bad ones. So hopefully these issues will go away when Electron upgrades to a newer version of Chromium.

Though it’s still weird that only integrated terminal is affected, the other modules of VSCode is fine. And it seems that it cannot be fixed by a simple downgrade, since 1.53.0 updates Electron from 9.x to 11.x, which crosses 2 major version. So hopefully it would be fixed by newer Electron, or xterm does some compatibility works.

BTW, looks like Electron upgrade only affects to Sogou Pinyin (as previous comment, QQ Pinyin and Rime are because of some css layout issue, and revert them can still works on latest build with newer Electron).

hey guys , i have made progress by using git bisect somebody can verify this.

the bad commit is: 94142bd7e0dbc073fe6abf130d7b7066a7beb1d6

if it is verified, maybe the issue comes from the upstream : Electron.

@ccloli Before your tip, i don’t even know that git has git bisect command. That’s quite a good way to find what causes the problem, i am trying to fix it in my environment. And I just spent long time to fix yarn watch problem, right now i know how to deal with it.

bad news, doesn’t work even that’s a fixed prebuilt.

@ZhyMC Well, I built it just now, can you help me to verify if it works for you? At least it works for me.

https://1drv.ms/u/s!As4fpXfOPou9ghCE3bCZXVu8u1gk?e=SeLPHO https://pan.baidu.com/s/1RrvKGtkKYWMMgVpSjy6mIQ us4x

If you’re not satisfy with prebuilt binary, you can also TIY by following these steps, just to be sure that after you clone the repo, run git revert 4b49bae.

1.52 can work normally.

This issue deserves attention because it has hit a large number of Chinese developers. I thought I have to consider downgrading the version of VSCode until this problem has been fixed.

How to downgrading? I not found old version in the VSCode home page.

You can find previous version here. https://code.visualstudio.com/updates/v1_50

搜狗输入法就没法输入中文，自带输入法就可以。。。难受

@heartacker xterm.js 的作者 @Tyriar 认为这是第三方输入法自己的问题，所以把这个 issue 移出了恢复迭代

我最初也以为是输入法问题，因为我使用 Rime 输入法出现了这个问题，但是用系统内建输入法之后就正常了，于是提了这个issue https://github.com/rime/weasel/issues/605 但是上面有其他用户提到了其他的输入法也出现了同样的问题，而且我自己使用 Rime 输入法在之前版本的VSCode中运作正常，所以原因应该不在输入法上。

I initially thought it’s because of input method’s bug, because I met this problem when I’m using Rime input method, but after I exchanged to Windows built-in input method, VSCode’s terminal works fine. However, other users above mentioned that some other input methods have the same problem, and Rime input method works fine in the previous version of VSCode, so the problem should not come from the input method.

I tested this and found the following:

• The ? issue seems specific to powershell 5, upgrading to pwsh 7 fixes the issue (plus it’s way better, you should do this if you use powershell)
• The non input problem we believe is a third party IME that has some problems.

Gathered some old comments from not-related issues here:

Well, I think #112978 is the same problem describe here. Okay, looks like it’s not a same issue, I got nothing instead of ???, but I’m in 1.52.1 instead of 1.53.0. I also tried the latest insider build (1.54.0-insider, afd102cbd2e17305a510701d7fd963ec2528e4ea), the behavior is the same as previous version and doesn’t get ???.

In detail, when you input x letters in terminal, they are not showing in terminal, then if you input y letters, then y - x of last letters will be shown.

Also this is not limited to CJK letters, when you input English letter with these CJK IME (you can press Enter to send your input Elnglish letters), they will be gone, too.

For example, if you input abc def ghi jklmn opqrst uv wxyz in sequence, the input letters will be like abcmntyz (sometimes I got abcmnopqrstyz which is a bit weird, it seems that before inputing opqrst, x is reset to 0).

BTW, pasting text with Chinese letters into terminal works fine, it only happens when inputing with some CJK IME (some IME like Microsoft Pinyin works fine here).

I have this problem too. My version is 1.53.But what I have found so far is that the QQ input method and the Microsoft input method can input Chinese, while the sogou input method cannot input Chinese.

It’s not new, I can reproduce this on both 1.52 and 1.51 (didn’t try to find a working one). xterm.js in browser works fine.