terminal: Bash command line editing gets wrong cursor position

Windows Terminal version

1.16.10262.0

Windows build number

10.0.22621.1265

Other Software

GNU bash, version 5.2.12(1)-release (x86_64-pc-msys) Oh My Posh version 14.12.0 via winget pkg JanDeDobbeleer.OhMyPosh (only indirectly relevant)

Steps to reproduce

I am using Oh My Posh to generate complex command line prompts in bash. When browsing command line history, the text input cursor often gets stuck two characters to the right of where it should go, leaving two characters stuck at the start of the command line. These characters are not in the edit buffer, but command line editing is messed up because the position on the screen and in the edit buffer apparently no longer match.

I experimented a bit and the effect seems to be triggered by recalling command lines of a certain length in the history list. I think Oh My Posh is actually irrelevant to the problem, although the prompt it generates probably is. I got a repeatable state using this PS1 prompt:

PS1=\[ESC[38;2;0;119;194m\]<U+E0B6>\[ESC[0m\]\[ESC[48;2;0;119;194m\]\[ESC[38;2;255;255;255m\] <U+E62A> \[ESC[0m\]\[ESC[48;2;0;119;194m\]\[ESC[38;2;255;255;255m\] bash \[ESC[0m\]\[ESC[0m
ESC[38;2;1;87;155;49mESC[7m\]<U+E0B0>\[ESC[0m\]\[ESC[48;2;1;87;155m\]\[ESC[38;2;255;255;255m\] ~ \[ESC[0m\]\[ESC[38;2;1;87;155m\]<U+E0B0>\[ESC[0m\]\[ESC[1000C\]\[ESC[10D\]\[ESC[38;2;73;64;79m\]<U+E0B6>\[ESC[0m\]\[ESC[48;2;73;64;79m\]\[ESC[38;2;255;255;255m\] 2.574s \[ESC[0m\]\[ESC[38;2;73;64;79m\]<U+E0B4>\[ESC[0m\] \[ESC[38;2;255;255;255m\]danm@PEGASUS \[ESC[0m\]\[ESC[0m\]\[ESC[38;2;255;255;255m\]❯ \[ESC[0m\]\[ESC]0;^G\]\[ESC]9;9;C:\Users\danmESC\\\]

I checked the prompt carefully, and all non-printing sequences seem to be properly wrapped with matching […]. I was suspicious of the OSC 9;9 sequence at the end, but removing it does not fix the problem.

Enter this command: echo 67890123678901

Now recall the command with up-arrow, then go back to an empty line with down-arrow. The characters “ec” will appear on the command line, with the text cursor after them, but if you enter a new command you’ll find that the “ec” text is not part of the edit buffer. Repeating this with an echo command one character shorter does not produce this effect.

Other possibly relevant environment variables: LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.zst=01;31:.tzst=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.wim=01;31:.swm=01;31:.dwm=01;31:.esd=01;31:.jpg=01;35:.jpeg=01;35:.mjpg=01;35:.mjpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.webp=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.m4a=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.oga=00;36:.opus=00;36:.spx=00;36:.xspf=00;36: TERM=xterm-256color

Expected Behavior

When recalling bash history, simply recalling the last command and then dismissing it should always leave the input line clear after the prompt, and the text cursor immediately after the prompt text.

Actual Behavior

Under some conditions, recalling history text and then dismissing it leaves extraneous characters on the current command line after the prompt, and the text cursor is positioned after them.

About this issue

Original URL
State: closed
Created a year ago
Comments: 36 (11 by maintainers)

Most upvoted comments

holy butts. Well. I’m glad you figured it out! LANG and LC_* have given us (and seemingly everyone) plenty of troubles in the past. I’m not sure there’s any good way to immediately mitigate that. Hopefully though, this thread should have plenty of keywords in it to help some future folks hitting this find the right fix ☺️

Thanks for following up!

zadjii-msft on Jul 5, 2023

AGH! Just today, I was going to ask you what LANG was set to. I decided that I must have already asked, and didn’t choose to.

So instead… can you share the commandline for your Git Bash profile?

If it doesn’t contain -l -i, it isn’t starting with the correct environment. 😄

DHowett on Jul 1, 2023

Holy frijole, I finally found a simple fix for this.

I was experimenting some more and was befuddled as to why this was happening:

\u2764 should be equivalent to \342\235\244, so bash clearly wasn’t interpreting the \u prefix correctly. A search for answers turned up a Stackoverflow article with an interesting top answer that pointed to locale settings.

I checked the output of locale and saw that LANG wasn’t set:

❤ locale
LANG=
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_ALL=

This appears to be the default state in Git Bash on Windows - my dotfiles don’t change anything related to locale, and I’ve seen this problem on a variety of systems, including a new laptop that I freshly set up just a few weeks ago.

Executing set LANG='C.UTF-8' fixed the \u problem and the positioning problem! So as @j4james guessed, this does seem to be a bash issue. Exporting LANG to the environment also seems to work and is visible to subordinate bash shells.

In a bash shell on WSL2, where I wasn’t seeing the problem, LANG is set to C.UTF-8 already.

Also, I’ll mention another Stackoverflow article that I ran into along the way, because it had a very clever workaround for a similar problem, using the sc and rc escape sequences. See the first answer here.

I could close this, but I’ll let the devs do that after verifying my assumption that it’s purely a bash problem. Maybe you can think of some way to defend users against this .

dwmuller on Jul 1, 2023

Further experimentation showed that putting any Unicode character in PS1 causes the problem.

Yeah. This is what I was saying in https://github.com/microsoft/terminal/issues/14982#issuecomment-1592842298. My guess is that it’s calculating the prompt length based on the number of bytes in the UTF-8 representation, rather than the actual glyph count.

j4james on Jun 30, 2023

Oh good lord. I started digging deeper into what Oh My Posh sets up using the --init option, and there’s way more going on here than is apparent from the PS1 value. It defines PS0, PS2, a shell function, and sets PROMPT_COMMAND to execute the shell function. The function sets up some context, puts the cursor at a calculated position, and defines PS1.

So there’s more going on here than is apparent from my PS1. I won’t bore you with the details just yet, but I’m analyzing it. Sorry about missing all this relevant information.

dwmuller on Apr 30, 2023

Okay, so, there’s one weird thing I couldn’t account for…

Your PS1 has a newline in it that seems like it is guarded by \[ and \]. If I do that, I get an unprintable character right after the newline (bash 5.0.3). I wonder if that’s related…

Reconstructing your actual PS1 from the version you shared was definitely challenging. 😄

DHowett on Mar 22, 2023