runtime: IHost.RunAsync() never completes

Describe the bug

I’m building a .NET Core 3.1 app that will run a BackgroundService in a Docker container. While I’ve implemented the startup and shutdown tasks for the BackgroundService and the service is definitely shutting down when triggered via SIGTERM, I’m finding that the await host.RunAsync() call never completes - meaning the remaining code in my Main() block isn’t executed.

To Reproduce

Use latest Microsoft.Extensions.Hosting package

Program.cs:

using System;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;

namespace BackgroundServiceTest
{
    class Program
    {
        static async Task Main(string[] args)
        {
            Console.WriteLine("Main: starting");
            try
            {
                using var host = CreateHostBuilder(args).Build();

                Console.WriteLine("Main: Waiting for RunAsync to complete");

                await host.RunAsync();

                Console.WriteLine("Main: RunAsync has completed");
            }
            finally
            {
                Console.WriteLine("Main: stopping");
            }
        }

        public static IHostBuilder CreateHostBuilder(string[] args) =>
            Host.CreateDefaultBuilder(args)
                .UseConsoleLifetime()
                .ConfigureServices((hostContext, services) =>
                {
                    services.AddHostedService<Worker>();

                    // give the service 120 seconds to shut down gracefully before whacking it forcefully
                    services.Configure<HostOptions>(options => options.ShutdownTimeout = TimeSpan.FromSeconds(120));
                });

    }

    class Worker : BackgroundService
    {
        protected override async Task ExecuteAsync(CancellationToken stoppingToken)
        {
            Console.WriteLine("Worker: ExecuteAsync called...");
            try
            {
                while (!stoppingToken.IsCancellationRequested)
                {
                    await Task.Delay(TimeSpan.FromSeconds(10), stoppingToken);
                    Console.WriteLine("Worker: ExecuteAsync is still running...");
                }
            }
            catch (OperationCanceledException) // will get thrown if TaskDelay() gets cancelled by stoppingToken
            {
                Console.WriteLine("Worker: OperationCanceledException caught...");
            }
            finally
            {
                Console.WriteLine("Worker: ExecuteAsync is terminating...");
            }
        }

        public override Task StartAsync(CancellationToken cancellationToken)
        {
            Console.WriteLine("Worker: StartAsync called...");
            return base.StartAsync(cancellationToken);
        }

        public override async Task StopAsync(CancellationToken cancellationToken)
        {
            Console.WriteLine("Worker: StopAsync called...");
            await base.StopAsync(cancellationToken);
        }

        public override void Dispose()
        {
            Console.WriteLine("Worker: Dispose called...");
            base.Dispose();
        }
    }
}

Dockerfile:

#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.

FROM mcr.microsoft.com/dotnet/core/runtime:3.1-buster-slim AS base
WORKDIR /app

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-buster AS build
WORKDIR /src
COPY ["BackgroundServiceTest.csproj", "./"]
RUN dotnet restore "BackgroundServiceTest.csproj"
COPY . .
WORKDIR "/src/"
RUN dotnet build "BackgroundServiceTest.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "BackgroundServiceTest.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "BackgroundServiceTest.dll"]

docker-compose.yml:

version: '3.4'

services:
  backgroundservicetest:
    image: ${DOCKER_REGISTRY-}backgroundservicetest
    build:
      context: .
      dockerfile: Dockerfile

Start the app with docker-compose up --build and see the console logging from the application indicating that the service is running.

Run a second command prompt window and execute docker stop -t 90 backgroundservicetest_backgroundservicetest_1 and see that the app shuts down gracefully but never writes any of the console messages after the call to await host.RunAsync()

Expected behavior

After the BackgroundService completes shutting down, control should be returned to the method that called .RunAsync()

Screenshots

Console output from docker-compose

    Successfully built 3aa605d4798f
    Successfully tagged backgroundservicetest:latest
    Recreating backgroundservicetest_backgroundservicetest_1 ... done
    Attaching to backgroundservicetest_backgroundservicetest_1
    backgroundservicetest_1  | Main: starting
    backgroundservicetest_1  | Main: Waiting for RunAsync to complete
    backgroundservicetest_1  | Worker: StartAsync called...
    backgroundservicetest_1  | Worker: ExecuteAsync called...
    backgroundservicetest_1  | info: Microsoft.Hosting.Lifetime[0]
    backgroundservicetest_1  |       Application started. Press Ctrl+C to shut down.
    backgroundservicetest_1  | info: Microsoft.Hosting.Lifetime[0]
    backgroundservicetest_1  |       Hosting environment: Production
    backgroundservicetest_1  | info: Microsoft.Hosting.Lifetime[0]
    backgroundservicetest_1  |       Content root path: /app
    backgroundservicetest_1  | Worker: ExecuteAsync is still running...
    backgroundservicetest_1  | Worker: ExecuteAsync is still running...
    backgroundservicetest_1  | info: Microsoft.Hosting.Lifetime[0]
    backgroundservicetest_1  |       Application is shutting down...
    backgroundservicetest_1  | Worker: StopAsync called...
    backgroundservicetest_1  | Worker: OperationCanceledException caught...
    backgroundservicetest_1  | Worker: ExecuteAsync is terminating...
    backgroundservicetest_1  | Worker: Dispose called...
    backgroundservicetest_backgroundservicetest_1 exited with code 0

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 22 (14 by maintainers)

Commits related to this issue

Most upvoted comments

Unfortunately this is by design, at least currently without other bigger changes to the runtime. When the process is being torn down we block the event handler until the host is dispose. RunAsync and Run automatically unblock the waiting so code will not execute after it (https://github.com/dotnet/runtime/blob/82ca681cbac89d813a3ce397e0c665e6c051ed67/src/libraries/Microsoft.Extensions.Hosting.Abstractions/src/HostingAbstractionsHostExtensions.cs#L75). That’s what disposing the host does.

@ericstj That issue you mentioned is similar but windows specific. With this one, SIGTERM does trigger process exit but once the host is disposed, it will unwind.

@martyt

To work around this you’d need to write code like this:

static async Task Main(string[] args)
{
    Console.WriteLine("Main: starting");
    IHost host = null;
    try
    {
        host = CreateHostBuilder(args).Build();

        Console.WriteLine("Main: Waiting for RunAsync to complete");

        await host.WaitForShutdownAsync();

        Console.WriteLine("Main: RunAsync has completed");
    }
    finally
    {
        Console.WriteLine("Main: stopping");

        if (host is IAsyncDisposable d) await d.DisposeAsync();
    }
}