azure-webjobs-sdk: WebJobs Shutdown signal not working

WebJobsShutdownWatcher is designed to monitor a sentinel file that the Kudu continuous webjobs infrastructure uses to signal that the WebJob is shutting down. In v2 this shutdown signal would cause RunAndBlock to complete, code here stopping the WebJob exe and allowing any cleanup code to run.

In v3, we’re not honoring this - causing a shutdown signal that is picked up by the WebJobsShutdownWatcher does not cause the await host.RunAsync() to complete, allowing any subsequent cleanup code to run. While we do handle the shutdown file notification and trigger token cancellation, it appears that currently this shutdown doesn’t cause IHostedService.StopAsync to be called, meaning any hosted service registered by the SDK or users are not stopped. Likely WebJobsShutdownWatcher needs to use IHostLifetime to call StopAsync when triggered.

We shouldn’t be ignoring the shutdown signal - it’s part of graceful shutdown. As it is now the WebJob will continue running until Kudu kills the process.

About this issue

Original URL
State: open
Created 5 years ago
Reactions: 4
Comments: 16 (1 by maintainers)

Commits related to this issue

Ensure graceful host shutdown when shutdown file is created (#2088) — committed to Azure/azure-webjobs-sdk by mathewc 3 years ago
Ensure graceful host shutdown when shutdown file is created (#2088) — committed to Azure/azure-webjobs-sdk by mathewc 3 years ago
Ensure graceful host shutdown when shutdown file is created (#2088) — committed to Azure/azure-webjobs-sdk by mathewc 3 years ago
Ensure graceful host shutdown when shutdown file is created (#2088) — committed to Azure/azure-webjobs-sdk by mathewc 3 years ago
Ensure graceful host shutdown when shutdown file is created (#2088) — committed to Azure/azure-webjobs-sdk by mathewc 3 years ago
Ensure graceful host shutdown when shutdown file is created (#2088) — committed to Azure/azure-webjobs-sdk by mathewc 3 years ago
Ensure graceful host shutdown when shutdown file is created (#2088) (#2730) — committed to Azure/azure-webjobs-sdk by mathewc 3 years ago

Most upvoted comments

Fix is checked in (see https://github.com/Azure/azure-webjobs-sdk/commit/01786831f084c5e7a5afb4301464e2d91bfe17af). Will be in the next package release.

mathewc on Jun 10, 2021

Any update, please?

Thanks

joshidp on Oct 21, 2020

Having the same issue

alhardy on Jan 24, 2020

This doesn’t seem to work.

var host = builder.Build();
var cancellationToken = new WebJobsShutdownWatcher().Token;
cancellationToken.Register(async () => await host.StopAsync());

using (host)
{
    await host.RunAsync(cancellationToken);
}

After some debugging, this code allows for graceful shutdown in my local environment when I introduce the environment variable WEBJOBS_SHUTDOWN_FILE and manually create the file. But when deployed to Azure, something means it will not work 😕

divinci on Jun 11, 2019

This doesn’t seem to work.

var host = builder.Build();
var cancellationToken = new WebJobsShutdownWatcher().Token;
cancellationToken.Register(async () => await host.StopAsync());

using (host)
{
    await host.RunAsync(cancellationToken);
}

divinci on Jun 10, 2019

At last some information regarding why my shutdown code is not working 😃

Thanks @mathewc do you have information for a workaround in the mean time?

divinci on Jun 10, 2019