extensions: Hang when calling start/stop on IHost

Yeah, we need to look at designing a real solution to this problem now.

It has nothing to do with background services. It’s all about the code in Program.cs that starts the host.

Timer log mitigation added. Backlog while working with CoreFx?

There is one more way to reproduce hanging or related weird behavior.

Environment.Exit() call in the main thread leads to permanent hang. Environment.Exit() in a background thread leads to hang for HostOptions.ShutdownTimeout if you call it in BackgroundService (IHostedService.StopAsync never completes). Also if you use a non-zero error code with the Environment.Exit(), it will be replaced to zero in the ConsoleLifetime.

I think the best solution is to create a way to detect something like SIGTERM separately and have an option to prevent immediate app termination. Then ConsoleLifetime can handle this separate event just like Console.CancelKeyPress and do not use ProcessExit event at all.

Is there some reason why the following code would not be an appropriate workaround for now. The code below does not hang and still runs your MyServiceA hosted service

	public class ProgramNoDisposeHangs
	{
		public static async Task Main(string[] args)
		{
			var host = CreateHostBuilder(args).Build();
			using (var newHost = host.Services.GetService<IHost>())
			{
				await newHost.StartAsync();
				await newHost.StopAsync();
			}
		}

		public static IHostBuilder CreateHostBuilder(string[] args) =>
			Host.CreateDefaultBuilder(args)
				.ConfigureServices(services => services.AddHostedService<MyServiceA>());
	}

I’m here to say that this was the cause of a test hang that led to lots o badness and sadness 😦

We know there are some corefx APIs needed to make this super clean and we’ll look in to that. The idea that shows the most promise is one that lets us actually just detect SIGTERM directly and tell corefx to cancel the default logic, handling it as a graceful shutdown instead. However, that’s not really feasible for 3.0 and it seems user-unfriendly to hang the application with no way of knowing why it’s hanging.

Until then, as a tactical fix to make this easier to understand, we’re going to propose this:

1. Wait a “reasonable time” (say, 1 minute?) for the _shutdownBlock to be unblocked in OnProcessExit
2. If the timeout elapses without unblocking (.WaitOne() returns false), immediately log a warning Waiting for the host to be disposed. Ensure all 'IHost' instances are wrapped in 'using' blocks. (or similar, we can wordsmith in the PR). The logger could throw ObjectDisposedException if we’re part-way through disposing though so we may need to be defensive.
3. Continue waiting indefinitely on the _shutdownBlock after logging that message
4. Ensure any samples we have for generic host (either in this repo, or in the docs) properly dispose of the host, to avoid this.

Why not just exit after the timeout? The console log is buffered so we have no guarantee that it was actually written after logging.

Why not just exit immediately? That would bypass any dispose/shutdown logic in the app. At least this way a user has some chance of understanding why the app is hanging.

cc @Tratcher @halter73