ksh: Tab completion menu causes inconsistent editor state in emacs

@McDutchie: Just hang in there. I, for one, know that you are doing a good job with this new ksh93 endeavor. Please take a break and get some rest. Perhaps take a day off and relax. Because, there is other fruit to pick from the issues tree.

I did a bit of research on this, and I think the fix to have the Emacs editing mode do the same as Vi is correct.

From RELEASE: 08-05-01 In multiline edit mode, the refresh operation will now clear the remaining portion of the last line.

Here’s a fragment from the completion.c of the venerable but dated CDE DtKsh:

                else
                        while (*com)
                        {
                                *out++  = ' ';
                                out = strcopy(out,*com++);
                        }
                *cur = (out-outbuff);
                /* restore rest of buffer */
                out = strcopy(out,stakptr(0));
                *eol = (out-outbuff);

Noticeably missing is the code to add a space after file name completions. So, it seems plausible that if multiline editing mode was added beforehand,the ep->ed->p_eol != ep->cursor-ep->screen case might never have occurred during testing.

Setting the ‘first’ parameter to -1 seems to be a pretty explicit indicator that the author(s) intended the line clearing code to run, hence the entry in RELASE.

The real issue is that if we update the cursor by calling ed_setcursor on line 1554 with old != new, the later call to setcursor on line 1583, here:

	I = (ncursor-nscreen) - ep->offset;
	setcursor(ep,i,0);

will use outdated screen information to call setcursor, which, coincidentally, calls ed_setcursor.

Well, that was quite the adventure… Could we add issue IDs/bug numbers to the regression tests, perhaps? It seems like useful information to catalog the defect delta from the last AST release, considering the effort going into this project.

Sorry, but… this is with patches 2&3 and the latest patch 1…

[12:10 PM][ttys000 +1][~/src/ksh-71][master@82d6272]
[1305] mbp13 $ bin/shtests pty                                     
#### Regression-testing /Users/mwilson/src/ksh-71/arch/darwin.i386-64/bin/ksh ####
test pty begins at 2021-02-25+12:11:27
	pty.sh[664]: emacs backslash escaping: line 673: expected "^:test-2: set -o emacs$", got ":test-2: set -o emacs em\bma\bac\bcs\bs"
	pty.sh[664]: emacs backslash escaping: line 677: expected "true \^C", got ""
test pty failed at 2021-02-25+12:11:40 with exit code 2 [ 28 tests 2 errors ]
test pty(C.UTF-8) begins at 2021-02-25+12:11:40
	pty.sh[664]: emacs backslash escaping: line 673: expected "^:test-2: set -o emacs$", got ":test-2: set -o emacs em\bma\bac\bcs\bs"
	pty.sh[664]: emacs backslash escaping: line 677: expected "true \^C", got ""
test pty(C.UTF-8) failed at 2021-02-25+12:11:54 with exit code 2 [ 28 tests 2 errors ]
test pty(shcomp) begins at 2021-02-25+12:11:54
	shcomp-pty.ksh[664]: emacs backslash escaping: line 673: expected "^:test-2: set -o emacs$", got ":test-2: set -o emacs em\bma\bac\bcs\bs"
	shcomp-pty.ksh[664]: emacs backslash escaping: line 677: expected "true \^C", got ""
test pty(shcomp) failed at 2021-02-25+12:12:08 with exit code 2 [ 28 tests 2 errors ]
Total errors: 6
CPU time       user:      system:
main:      0m00.011s    0m00.023s
tests:     0m00.738s    0m01.224s

That leaves the fourth bug in this issue, the vi one. I’ve no clue where to look for the cause yet – it may or may not even be in vi.c. Any ideas anyone? @JohnoKing @hyenias @lkujaw @hvdijk

What little I can quickly find without trying to understand the code: with the following debug patch:

--- a/src/cmd/ksh93/sh/lex.c
+++ b/src/cmd/ksh93/sh/lex.c
@@ -397,6 +397,7 @@ int sh_lex(Lex_t* lp)
                                fcseek(-LEN);
                                goto breakloop;
                        case S_EOF:
+                               write(2, "S_EOF\n", 6);
                                sp = fcfile();
                                if((n=lexfill(lp)) > 0)
                                {

After the Enter after tab completion, the S_EOF case is hit again, and the call to lexfill(lp) is what causes the prompt for more input (I think).

After the Enter without tab compltion, the S_EOF case is not hit again.

Hopefully this will speed up a deeper investigation by someone who actually understands the code.

Oh, man, number 3 is so cool, it’s like a pain you don’t know how to get rid of until it’s suddenly just… gone. What crazy person EVER thought the original behavior was a correct one?

Ahem.

Ok, number 1… yup, fixes it. behavior is correct now with multiline on or off… no extra space and no hanging characters.

For number 2, also fixes it, just get a terminal beep now with a TAB rather than a jump to the next tabstop and then a menu full of everything.

Is this one part of the same issue or is it a new issue? Found this on the old ast-users ML, apparently the user complaining about his issue got told it was normal behavior, but in testing that, I found this:

[07:14 PM][ttys000][~/test]
[135] mbp13 $ ls
JHS_REFRESH-1.txt  JHS_REFRESH-A.txt
[07:14 PM][ttys000][~/test]
[136] mbp13 $ JH 

hit TAB

[136] mbp13 $ JH      

Four spaces get inserted. Why? Hit TAB again

[136] mbp13 $ JH      JHS_REFRESH-

Huh? This time it did the filename completion. Hit TAB again

[136] mbp13 $ JH      JHS_REFRESH-
1) JHS_REFRESH-1.txt
2) JHS_REFRESH-A.txt
[136] mbp13 $ JH      JHS_REFRESH-   

And now it displays the completion list menu?

There’s some confusion going on here between what KSH does if it’s trying to complete a command (the first token on the line) and trying to complete a filename (everything else).

The original thread: https://www.mail-archive.com/ast-users@lists.research.att.com/msg00419.html The simpler testcase: https://www.mail-archive.com/ast-users@lists.research.att.com/msg00436.html