orleans: Microsoft.AspNetCore.Connections.ConnectionResetException: An existing connection was forcibly closed by the remote host

We have a lot of ConnectionResetException errors on localy and production environment. On production we use service fabric. On each node installed both silo as stateless service and clients as docker web applications. We use the azure table cluster provider. This error began to appear on recent versions of Orleans Localy the client and silo run on the same machine. How to get rid of them?

Thanks

Microsoft.AspNetCore.Connections.ConnectionResetException: An existing connection was forcibly closed by the remote host ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at Orleans.Networking.Shared.SocketAwaitableEventArgs.<GetResult>g__ThrowSocketException|7_0(SocketError e)
   at Orleans.Networking.Shared.SocketAwaitableEventArgs.GetResult()
   at Orleans.Networking.Shared.SocketConnection.<ProcessReceives>d__27.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Orleans.Networking.Shared.SocketConnection.<DoReceive>d__26.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
   at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
   at System.IO.Pipelines.Pipe.GetReadAsyncResult()
   at Orleans.Runtime.Messaging.Connection.<ProcessIncoming>d__55.MoveNext()

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 17 (7 by maintainers)

Most upvoted comments

Thanks, I see in this case that the error is just a normal connection reset by a client that’s connecting.

EDIT: we should avoid logging an exception/warning in these normal cases

I believe this is probably being caused by some other issue (eg, silo being killed), rather than something in the networking layer, so other logs are helpful