runtime: ConsoleLifetime doesn't allow full graceful shutdown for ProcessExit
Describe the bug
When ConsoleLifetime shuts down via the AppDomain.ProcessExit path, it skips a number of steps done in normal graceful program termination.
Related to dotnet/extensions#1363, but perhaps the opposite problem (in this case, it shuts down too soon rather than hanging/too late).
To Reproduce
Program.cs:
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
using System;
using System.Threading;
class Program
{
static int Main()
{
var builder = new HostBuilder()
.ConfigureLogging(l =>
{
l.AddConsole();
l.Services.AddSingleton<ILoggerProvider, CustomLoggerProvider>();
}).ConfigureServices(s =>
{
s.AddSingleton<OtherService>();
});
using (IHost host = builder.Build())
{
host.Services.GetRequiredService<OtherService>();
host.Start();
// Edit: ignore Environment.Exit; instead use CTRL+C or docker stop here, per comments below.
//Thread thread = new Thread(() => Environment.Exit(456));
//thread.Start();
host.WaitForShutdown();
Console.WriteLine("Ran cleanup code inside using host block.");
}
Console.WriteLine("Ran cleanup code outside using host block.");
Console.WriteLine("Returning exit code from Program.Main.");
return 123;
}
class CustomLoggerProvider : ILoggerProvider
{
public ILogger CreateLogger(string categoryName)
{
return NullLogger.Instance;
}
public void Dispose()
{
Console.WriteLine("Ran logger cleanup code.");
}
}
class OtherService : IDisposable
{
public void Dispose()
{
Console.WriteLine("Ran most service's cleanup code.");
}
}
}
App.csproj:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.2</TargetFramework>
<RuntimeIdentifier>win-x64</RuntimeIdentifier>
<SelfContained>false</SelfContained>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Hosting" Version="2.2.0" />
<PackageReference Include="Microsoft.Extensions.Logging.Console" Version="2.2.0" />
</ItemGroup>
</Project>
- Run the code above as-is, and terminate with Ctrl+C. See all the cleanup code that runs.
- Uncomment the two Thread lines above to let the program self-terminate via Process.Exit.
Expected behavior
The same cleanup code runs when exiting via ProcessExit as it did with Ctrl+C:
Ran cleanup code inside using host block.
Ran most service's cleanup code.
Ran logger cleanup code.
Ran cleanup code outside using host block.
Returning exit code from Program.Main.
X:\path\to\App.exe (process nnn) exited with code 123.
Actual behavior
Only the following cleanup code runs:
Ran cleanup code inside using host block.
Ran most service's cleanup code.
X:\path\to\App.exe (process nnn) exited with code 456.
Note that the return code for a normal ProcessExit case (rather than one simulated by Environment.Exit) would likely be 3221225786 (STATUS_CONTROL_C_EXIT) on Windows for v2.2.0 of this package and 0 for the latest (I think; based on ConsoleLifetime source).
Additional context
Trapping ProcessExit and then deciding how long to wait overall is rather problematic; it could wait too long/block, as in dotnet/extensions#1363, or too short, as here. The main options I can think of are:
- Only have ProcessExit trigger the start of graceful shutdown (like Ctrl+C does), and give something else the responsibility to keep the shutdown from completing until Program.Main finishes.
- Have some automatic way to know when Program.Main returns. Not sure if this is possible - I tried doing a thread.Join on the main thread, but that doesn’t quite work. Polling it for when IsBackground flips to false looks like it could work, but that seems not exactly ideal.
- Push the control up to the user for when to have the shutdown handler terminate (have the user pass in an event signaled from Program.Main maybe or something roughly like that; more complicated API to make that work).
Overall, the host itself doesn’t own the entirety of Program.Main, so having it decide when to stop running is tricky.
I did find that at least the logger cleanup can likely be done better by changing the order of the disposables in the service container, including by adding a first constructor parameter to Host that gets registered first - currently, the IHostApplicationLifetime is after loggers in the list of things to dispose, and the logic to dispose in reverse order on the service provider/scope means loggers don’t get to cleanup.
For application insights, for example, having the logger not get to have Dispose called means it can’t call Flush, so logs currently get lost on shutdown.
The full list of things that currently get lost are:
- Earlier registered service provider disposables, including logger providers (likely could be fixed with some registration order/constructor chaining/dependency order changes).
- Code after the using host block.
- Program.Main’s exit code.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 3
- Comments: 34 (16 by maintainers)
Commits related to this issue
- Handle SIGTERM in Hosting and handle just like SIGINT (CTRL+C) Don't listen to ProcessExit on net6.0+ in Hosting anymore. This allows for Environment.Exit to not hang the app. Don't clobber ExitCode ... — committed to eerhardt/runtime by eerhardt 3 years ago
- Consume PosixSignal in Hosting's ConsoleLifetime (#56057) * Add NetCoreAppCurrent target to Microsoft.Extensions.Hosting * Handle SIGTERM in Hosting and handle just like SIGINT (CTRL+C) Don't l... — committed to dotnet/runtime by eerhardt 3 years ago
Excellent. Thanks, @eerhardt!
Not exactly.
Environment.Exit
is low-level API that is not designed to be overridable.Ctrl-C and similar signals exit the process by default, but that behavior can be overriden. #50527 is about making the APIs for these signals better.
This issue looks like duplicate of #50527 and can be closed. @davidfowl Do you agree?
Unhandled exception is abnormal exit. It does not follow the same path as
Environment.Exit
Here’s a zip of the source I’m using as a workaround. The usage is as described above. I’m using it with v2.2.0 of the hosting extensions package (it may work with a later version but I haven’t tested that).
ProcessLifetimeWorkaround.zip
Disclaimer: as-is; just a workaround; not an official solution; not offering support for this code; etc.
A couple other things that likely currently get lost: full disposing of custom service providers, full disposing of custom hosts.
I ended up working around this with a wrapper around the entry point for now. Here’s what it looks like:
When run with the original set of things hooked up above, all of the cleanup code runs (including code outside the using block and setting the exit code).
The IProcessLifetime interface looks like this:
If you’re interested in going down this route further, let me know and I can provide the other details as well (UseProcessLifetime extension method and ProcessLifetimeHost.Run method).