cmder: [Bug] Reserve-i-search return broken string

@Happin3ss thanks for the diagnostic information – the clink.log file showed what is happening.

The prompt string is incorrect.

What are you using to generate your prompt string?

It ends with ESC [ K ESC [ 0 J, and that’s causing the problem:

• ESC [ K means “clear to end of line”. That must be removed; every time the prompt is re-displayed it will wipe out whatever text is displayed on the first line of the input line.
• ESC [ 0 J means “clear from the cursor to the end of the screen”. That must be removed; every time the prompt is re-displayed it will wipe out all displayed input text.

The problem also happens in bash and Readline

If you use the same prompt string in bash, use <kbd>Ctrl-R</kbd> to search and find a history line that wraps onto 2 or more lines, and press <kbd>Right</kbd>, then the entire input line disappears.

So, those extra characters in the prompt are incorrect to have in a prompt string.

Why does it happen in Clink now, but not before?

Previously, it happened in Clink under the same circumstances where it happens in bash (such as the previous example).

But after I made Clink stop using the Readline display routines, and wrote custom display routines, the new routines are more efficient and more predictable. And they realize that they can simply shift some existing characters left, instead of reprinting stuff manually. But by that time, the prompt string has already wiped out the input line display, and so there’s nothing to shift over.

Conclusion

This is caused by an incorrect prompt string.

However, I would bet that there are a lot of incorrect prompt strings out there, because quirks in Readline’s display routines make it less common for the incorrectness to become noticeable. Quick “sanity testing” would easily lead someone to assume that it’s valid to include those escape codes in a prompt string. But it’s not, and never has been.

But I’ll make Clink smart enough to compensate for that kind of incorrectness

Technically the problem is caused by user error. But, often a user will simply have used someone else’s prompt generator program or prompt string. And in that case it is some other user’s error, causing a problem for the current user. And that can feel very frustrating, and it can make it very difficult for the current user to fix the mistake without losing their custom prompt configuration.

I’ll make Clink smart enough to automatically compensate when those escape codes are in a prompt string. And I’ll make the <kbd>Ctrl-X</kbd>,<kbd>Ctrl-Z</kbd> diagnostic command smart enough to detect the incorrectness and show an alert about it.

Jan made a fix in oh-my-posh.

The problem is reproducible in bash on Linux (as well as other shells and platforms, etc).

@DRSDavidSoft this isn’t related to Cmder. This is an issue between oh-my-posh and various shells.

It’s debatable whether the shells should be smarter, or whether oh-my-posh should be more conservatively compatible. I don’t have time to get sucked into debates like that, and so when it’s easy to compensate then I just make a change in code that I control. 🙃 (Which I did, in Clink v1.4.16)

It looks like oh-my-posh always appends ESC[K ESC[0J.

There are comments in its source code saying it needs to do that in PowerShell due to a bug in some case. I can’t see yet what is making it also show up for all the other shells as well.

I started an email conversation with Jan and shared the info about bash, Readline, Clink, etc.

Anyway, Clink v1.4.16 and newer completely compensate for the ESC[K ESC[0J.

But you should be aware that other shells will likely have edge cases where the ESC[K ESC[0J cause the input line display to become corrupted, depending on the specific text and operations that are used.

Hi @chrisant996 I’m not so sure honestly. I’m using a default cmder setup (with clink and clink completion bundled by default), theming with oh-my-posh. So I believe it should be clink, with history-search-backward and history-search-forward

Ok, you’re using oh-my-posh.

I still don’t know which theme, though.

I’ve tried to give the short version of explaining precisely what is wrong in the prompt string.

And I also explained how to reproduce the exact same problem in bash. The steps are a little different, but that’s because of ancient quirks in bash that are meant for optimizing the amount of data that would get sent over an old 1200 baud modem. Those quirks cause bash to use different drawing strategies at different times.

I can understand that you might not have familiarity with the details of prompt strings and escape codes and how prompts and input lines get drawn by programs. I can understand if you don’t believe me or my expertise or detailed analysis.

But that doesn’t change that the problem is in the theme you’re using.

In fact, if you try a bunch of other themes, you’ll probably find that most of them don’t have the problem.

I can confirm Clink 1.4.16 make it works again . Thank you a lot ! I’m also open to assist to find out the root cause of this issue, to prevent it happen in future for other users.

The only thing left is for you to share which theme you’re using with oh-my-posh. The bug is in the theme.

Then I can report it to Jan and the theme author, on your behalf. They’ll understand and recognize why it’s a problem, and they’ll know the right steps to take to fix the theme bug (the same steps I described earlier).

The bug was not in Clink. The change I made in Clink detects the theme bug, and TURNS OFF OPTIMIZED OUTPUT. Clink goes into defensive mode and becomes super inefficient, to compensate for the theme bug. The reason the precisely same cited steps don’t reproduce the problem in bash is because bash accidentally falls into an unoptimized mode. I’m leaving out a lot of specific technical details about the algorithm and escape codes chosen by bash.

Anyway, the ONLY reason I changed Clink was because most people don’t understand enough about escape codes to recognize why the problem is happening or who to report it to. I’m mainly trying to reduce the support cost that I have to pay for someone else’s bugs. 🙃