runtime: BufferBlock.Completion never completes in specific scenario
Hi! While I was writing some library code, I created a small TPL Dataflow pipeline consisting of two blocks, with the completion of the first block not propagated properly to the second block. Here is a minimal example that reproduces this strange behavior:
using System;
using System.Threading;
using System.Threading.Tasks;
using System.Threading.Tasks.Dataflow;
static class Program
{
static async Task Main()
{
var block1 = new BufferBlock<int>();
var block2 = new BufferBlock<int>();
block1.LinkTo(block2, new() { PropagateCompletion = true });
block1.Post(1);
block1.Complete();
await block1.Completion;
block2.TryReceiveAll(out var items);
bool completed = block2.Completion.Wait(500);
Console.WriteLine($"block2 completed in time: {completed}");
}
}
Output:
block2 completed in time: False
The expected behavior would be for the block2 to complete immediately, since the block1 has already completed, the two blocks are linked together with the PropagateCompletion = true option, and the block2 has emitted all the messages it contains. However the block2 never completes in this scenario. Calling block2.Completion.Wait() blocks indefinitely.
Switching from BufferBlock to TransformBlock for any of the two blocks makes no difference, the issue remains.
There are several subtle changes that prevent this behavior from happening.
- Adding a
Thread.Sleep(100)after theblock1.Complete()solves the problem. - Waiting the
block1synchronously (block1.Completion.Wait()) also solves the problem. - Waiting the
block2asynchronously (await block2.Completion) solves the problem as well. - Completing the
block2manually (block2.Complete()) before waiting its completion, also fixes the problem.
My guess is that some sort of race condition is taking place in this specific scenario.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 15 (11 by maintainers)
@mayorovp and @pedoc are right: this is a deadlock caused by combining
awaitwith synchronousWait().Specifically, it happens because of this part in the TPL Dataflow code:
https://github.com/dotnet/runtime/blob/6527f540e4b50bc84eb72705f80d3f2bdd57473b/src/libraries/System.Threading.Tasks.Dataflow/src/Internal/SourceCore.cs#L962-L968
The problem is that
_completionTask.TrySetResultexecutes its continuations synchronously, which means it directly invokes the part ofMainafterawait block1.Completion;. But that blocks waiting forblock2.Completion, which means_targetRegistry.PropagateCompletion()is never called, which meansblock2.Completionis not completed, leading to a deadlock.One resolution would be to say that it’s the user’s fault for combining async and sync code in this way and close this issue. Another possible resolution would be to use
RunContinuationsAsynchronouslyon the_completionTask, which sacrifices some performance to prevent this deadlock. I think this is the way to go, so I have opened https://github.com/dotnet/runtime/pull/61140.