CliWrap: Console.OpenStandardInput() does not get properly released after using it as a pipe source
Using v 3.1.0:
After using .WithStandardInputPipe(PipeSource.FromStream(inputStream))), the input stream does not appear to be correctly relinquished once the command has finished executing.
Repro steps:
- Run the program below
- Once you see the
Streams disposedmessage, press a character key - You’d expect to to see the message
Intercepted key: {key}printed, but you don’t - Press a few other character keys, note that the program does not progress until you press the
enterkey - After the
enterkey has been pressed, press a single character key. Note that the messageIntercepted key: {key}is printed
using (var inputStream = Console.OpenStandardInput())
using (var errorStream = Console.OpenStandardError())
{
var commandResult =
await Cli.Wrap("git")
.WithArguments("--version")
.WithValidation(CommandResultValidation.None)
.WithStandardInputPipe(PipeSource.FromStream(inputStream))
.WithStandardOutputPipe(PipeTarget.ToStream(errorStream))
.ExecuteBufferedAsync(Encoding.UTF8);
}
Console.WriteLine("Streams disposed");
var key = Console.ReadKey(intercept: true);
Console.WriteLine($"Intercepted key: {key.KeyChar}");
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 48 (23 by maintainers)
It might, but there are a couple of issues I see with it.
First, the cancellation. Currently, CliWrap only cancels stdout/stderr pipes if the whole process is canceled. Otherwise, if the pipes are open for whatever reason, it assumes that there’s still data left to handle, even if the process itself has exited (which is a common scenario if you run a lot of short-lasting but output-heavy processes in quick succession).
https://github.com/Tyrrrz/CliWrap/blob/9b9384db16c2b759023e2ae2353c58e43fe74542/CliWrap/Command.Execution.cs#L230-L251
So that
Task.Delay(Timeout.Infinite, cancellationToken)'scancellationTokenwill never trigger in most situations.Second, if the stdout/stderr stream is open, the data has to be read. Otherwise, there’s a risk, if the process writes enough (a lot) of data to either stream, it will hit the buffer limit, and the next
Console.WriteLine(or whatever equivalent) will lock until that buffer is exhausted (or the pipe is closed).PipeTarget.Nullworks by immediately closing the pipe, preventing any deadlocks from occurring. AnyConsole.WriteLinewill just result in a no-op on the process’s side.PipeTarget.ToStream(Stream.Null)keeps the pipes open, but still reads all the data and simply discards it. Compared to the “null” target, this can create more memory overhead as you need somewhere to store the data, even temporarily (at least the ~65kb buffer used for consuming bytes from the stream), and it puts pressure on the GC to clean it up afterward.Your
PipeTarget.NoOpwill never finish executing in the current implementation, due to relying on a cancellation signal that may never trigger – but also risks creating deadlock scenarios by keeping the stdout/stderr pipes open without reading them.Ok, I see.
So basically, invoking Git is one of those cases where piping to
Stream.Nullis needed, because apparently it actively checks the pipe?I’ll do some more testing to ensure this works also on Windows and Linux, but barring any unforeseen issues, I think we are good. 👍🏻
I’d like to take this opportunity to express our appreciation and thank you for all your work maintaining this library, and for engaging with us to help solve for our scenario. It’s something we don’t take for granted.
Hmm… I think that’s unlikely, because it would never work cross-platform, in a macOS and Linux terminal environment the streams is the only thing a process has, no access to window handles and such. And it would also make it even weirder that
Console.ReadKey()does not receive anything just because we’re mucking around with the input stream, no?