runtime: SslStream object disposed exception

From @timmydo on Saturday, 08 December 2018 21:55:49

I’m not sure if this is the right place to report this, but I’m seeing this exception occasionally:

kubernetes docker pod uname -a: Linux platform-prod-577cf48b4b-wrg7x 4.15.0-1030-azure dotnet/corefx#31~16.04.1-Ubuntu SMP Tue Oct 30

context: running kestrel behind nginx on linux in docker image based on dotnet:2.1-aspnetcore-runtime

19:40:01 UTC 2018 x86_64 GNU/Linux
Unhandled Exception: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'SslStream'.
   at System.Net.Security.SslState.ThrowIfExceptional()
   at System.Net.Security.SslState.CheckThrow(Boolean authSuccessCheck, Boolean shutdownCheck)
   at System.Net.Security.SslState.CheckOldKeyDecryptedData(Memory`1 buffer)
   at System.Net.Security.SslState.HandleQueuedCallback(Object& queuedStateRequest)
   at System.Net.Security.SslState.FinishHandshakeRead(Int32 newState)
   at System.Net.Security.SslState.FinishHandshake(Exception e, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.RehandshakeCompleteCallback(IAsyncResult result)
   at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
   at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
   at System.Net.AsyncProtocolRequest.CompleteUserWithError(Exception e)
   at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ThreadPoolWorkQueue.Dispatch()

pod error code:

Last State:     Terminated
      Reason:       Error
      Exit Code:    139
      Started:      Fri, 07 Dec 2018 17:06:46 -0800
      Finished:     Fri, 07 Dec 2018 23:48:04 -0800

let me know if there is more info I could provide. thanks.

Copied from original issue: aspnet/KestrelHttpServer#3111

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 27 (13 by maintainers)

Commits related to this issue

Catch exception in ssl handshake so it can be propagated to async caller Port fix from PR #34089 (.NET Core 3.0) to release/2.1 LTS branch. Fixes #34033 — committed to davidsh/corefx by davidsh 5 years ago
Catch exception in ssl handshake so it can be propagated to async caller Port fix from PR #34089 (.NET Core 3.0) to release/2.1 LTS branch. Fixes #34033 — committed to davidsh/corefx by davidsh 5 years ago
Catch exception in ssl handshake so it can be propagated to async caller (#39456) Port fix from PR #34089 (.NET Core 3.0) to release/2.1 LTS branch. Fixes #34033 — committed to dotnet/corefx by davidsh 5 years ago

Most upvoted comments

We definitely fixed the original issue where an ObjectDisposedException on SslStream was being thrown on a threadpool thread and thus crashing the process with an un-catchable exception. That was fixed in .NET Core 3.0 as I mentioned.

However, it appears based on these reports above that there is still an ObjectDisposedException surfacing from SslStream during an HTTP request cancellation/timeout. This is showing up in the callstacks above as an inner exception to the TaskCanceledException. So, if you have a repro for that, please open a new issue so that can be investigated.

Keep in mind that the top-level exception, TaskCanceledException is probably correct. Something caused the HttpClient to cancel the request, most likely due to the HttpClient.Timeout occurring. However, there is potentially a bug still in .NET Core where the cancellation logic and the closing of the SslStream and TCP connections is throwing these inner exceptions that are surfacing in the call stack. Those exceptions can probably be trapped in a better way since the ObjectDisposedException is most likely “expected” as the connection is being closed due to the cancellation.

So, to help isolate this particular issue, please open a new issue and post repro instructions so we can diagnose. Thanks!

davidsh on Nov 1, 2019

.NET Core 3.0 released today. I suggest you try out the official release. If you still have a repro using .NET Core 3.0 and see these ObjectDisposed exceptions, please open a new issue and reference this one. Thanks!

davidsh on Sep 23, 2019

The fix will ship with the next 2.2.x release which is 2.2.7. And that will be sometime next month.

It will also ship with the next 2.1.x release which is 2.1.13. That should be shipped the same time as 2.2.x.

davidsh on Aug 8, 2019

Fix for release/2.1 checked in with PR dotnet/corefx#39456

davidsh on Jul 16, 2019