ksh: Intermittent coprocess hang on Debian/Ubuntu and Solaris

I would not enable it by default as it would cause the Github CI test runners to fail

Yes, hence my intention to make the failure a warning, i.e., it will not be counted as a failure, but still show up in the output.

So, I recommend adding in some sleeps as follows to correct whatever on Debian systems that need more time to deal with the coprocess and/or its pipes.

That would defeat the purpose of the regression test, as it would fail to expose the bug. So we might as well not bother running it at all then.

Given that it’s only reproducible on a few specific systems (which unfortunately includes Debian and derivatives, which are popular), I suspect this is a race condition in the respective kernels, not in ksh.

On my Ubuntu based systems (Intel and ARM), introducing a sleep of .01 between commands allows the coprocess script to complete without error even on my slowest box. I used head instead of cat in the following code snippet while in an interactive session to validate each test by altering the number of lines given to head and various sleeps.

Passing head -n #:

• 2: All tests pass and the head coprocess terminates by itself. [good]
• 3: head coprocess has to be terminated manually as it is still waiting on more input [expected, good]
• 0 with no sleeps: head process failed to terminate [bad, head -n 0 still running and not had a chance to exit]
• 0 and only having a sleep before 1st print: both prints fail [good]
• 1 with only sleep before 1st print: both prints pass, expected 2nd print to fail [bad]
• 1 with sleeps before each print: second print fails [good]
• 2 with sleeps before each print: head process fails to terminate [bad]
• 2 with sleeps before each print and the jobs command: all pass [good]

head -n 2 |&; pid=$!
sleep .01
print foo >&p 2> /dev/null || echo 'first write of foo to head coprocess failed'
sleep .01
print foo >&p 2> /dev/null || echo 'second write of foo to head coprocess failed'
sleep .01
jobs $pid 2> /dev/null && { echo 'head coprocess failed to terminate'; kill $pid ;}
wait $pid 2> /dev/null

So, I recommend adding in some sleeps as follows to correct whatever on Debian systems that need more time to deal with the coprocess and/or its pipes. It may even be something related to job control–I just do not know. The 3rd sleep is not needed as the wait command provides a pause for the system to catch up.

$cat |&
pid=$!
sleep .01
print foo >&p 2> /dev/null || err_exit "first write of foo to $cat coprocess failed"
sleep .01
print foo >&p 2> /dev/null || err_exit "second write of foo to coprocess failed"
kill $pid
wait $pid 2> /dev/null

Some good news. I recently got me a CentOS 8 virtual machine so I can test Red Hat’s /bin/ksh93 with all their patches. And the hang is not reproducible there. Iteration 80000 and counting…

Now that @kdudka has made sure I have access to the relevant non-public Red Hat bugs, I can simply continue my work of porting all those Red Hat patches to 93u+m, devising regression tests for them, etc. and by the time that work is complete, this bug should have gone away.