azure-functions-host: DryIoc.ContainerException causes lots of azure function to fail few times every day

We see few DryIoc.ContainerException failures throughout the day in multiple azure functions in both our test and prod bed. So, I am hoping to get some guidance on figuring out why these failures are happening and what can we do to stop/reduce them.

Investigative information

  • Timestamp: 2020-09-08T22:13:12.0028507Z
  • Invocation ID: 5f7fe852-f2f8-45ae-b6bf-84478fd64b71
  • Region: West US 2

Repro steps

These functions are either timer triggered or queue triggered.

Actual behavior

We are seeing lots of azure functions failing in our test code few times a day with DryIoc.ContainerException.

Message:

Container is disposed and should not be used: Container is disposed. You may include Dispose stack-trace into the message via: container.With(rules => rules.WithCaptureContainerDisposeStackTrace())

Call Stack:

DryIoc.ContainerException:
   at DryIoc.Throw.It (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 8990)
   at DryIoc.Container.ThrowIfContainerDisposed (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 410)
   at DryIoc.Container.WithCurrentScope (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 436)
   at DryIoc.ResolverContext.OpenScope (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 2699)
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.WebHostServiceProvider.CreateScope (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\WebHostServiceProvider.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 51)
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.ScopedResolver.CreateChildScope (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\ScopedResolver.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 42)
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.CreateScope (Microsoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=nullMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\WebJobs.Script.WebHost\DependencyInjection\JobHostServiceProvider.csMicrosoft.Azure.WebJobs.Script.WebHost, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null: 104)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionInstanceWrapper.get_InstanceServices (Microsoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35Microsoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInstanceWrapper.csMicrosoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: 44)
   at Microsoft.Azure.WebJobs.Host.Executors.DefaultJobActivator.CreateInstance (Microsoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35Microsoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\DefaultJobActivator.csMicrosoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: 32)
   at Microsoft.Azure.WebJobs.Host.Executors.ActivatorInstanceFactory`1+<>c__DisplayClass1_1.<.ctor>b__0 (Microsoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35Microsoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\ActivatorInstanceFactory.csMicrosoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: 20)
   at Microsoft.Azure.WebJobs.Host.Executors.ActivatorInstanceFactory`1.Create (Microsoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35Microsoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\ActivatorInstanceFactory.csMicrosoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: 26)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2.CreateInstance (Microsoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35Microsoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.csMicrosoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: 44)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+ParameterHelper.Initialize (Microsoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35Microsoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.csMicrosoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: 846)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<TryExecuteAsyncCore>d__18.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35Microsoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.csMicrosoft.Azure.WebJobs.Host, Version=3.0.19.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: 122)

Related information

  • Programming language used: C#, .net core

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Reactions: 1
  • Comments: 18 (1 by maintainers)

Most upvoted comments

@fabiocav We do use DI singletons for cosmos db clients, http client factory, etc. as per recommendations in the Azure Functions documentation. What information do you need to debug this further?

+3
zmarty on Sep 14, 2020

Update 2022

We faced this problem and found out that AzureFunctionsWebHost__hostid was missing in this particular function and we have added it manually using a new guid and the function was working as expected

Thanks for verifying and confirming it.

+1
vsha041 on Dec 27, 2022

Update 2022

We faced this problem and found out that AzureFunctionsWebHost__hostid was missing in this particular function and we have added it manually using a new guid and the function was working as expected

+1
gethari on Dec 27, 2022

@vsha041 Just to be clear this is how my startup file looks like:

        public override void Configure(IFunctionsHostBuilder builder)
        {
            base.Configure(builder);

            builder.Services.AddSingleton<IGraphServiceClient>((services) =>
            {
                return new GraphServiceClient(FunctionAppDI.CreateAuthProviderFromSecret(services.GetService<IOptions<GraphCredentials>>().Value));
            })
            .AddScoped<IGraphGroupRepository, GraphGroupRepository>()
            .AddScoped<Calculator>()
            .AddSingleton<IBlobStorageRepository, BlobStorageRepository>((s) =>
            {
                var configuration = s.GetService<IConfiguration>();
                var storageAccountName = configuration["storageAccountName"];
                var containerName = configuration["containerName"];

                return new BlobStorageRepository($"https://{storageAccountName}.blob.core.windows.net/{containerName}");
            });
        }

Could you please let me know how to update this?

You can try below code.

public override void Configure(IFunctionsHostBuilder builder)
        {
            base.Configure(builder);
	// 1) get graph credentials here i.e. read from appSettings here
            builder.Services.AddSingleton<IGraphServiceClient>((services) =>
            {
                return new GraphServiceClient(FunctionAppDI.CreateAuthProviderFromSecret(graphCredentials));
            })
            .AddScoped<IGraphGroupRepository, GraphGroupRepository>()
            .AddScoped<Calculator>()
			
	// 2) get configuration object here 
	// var configuration = builder.GetContext().Configuration;
            .AddSingleton<IBlobStorageRepository, BlobStorageRepository>((s) =>
            {                
                var storageAccountName = configuration["storageAccountName"];
                var containerName = configuration["containerName"];

                return new BlobStorageRepository($"https://{storageAccountName}.blob.core.windows.net/{containerName}");
            });
        }
+1
vsha041 on Aug 15, 2022

Ok let me try that. Thank you! Also, while swapping slots, should these values be swapped?

It’s ok if they are swapped as long as they are different. Or you can put slotSettings: true if you don’t want them to get swapped.

+1
vsha041 on Aug 15, 2022

@ilantom - yes but GUID should be different for both slots unless staging one is deleted immediately after production deployment. So if you have multiple active slots at any given time then they all should have different GUID.

+1
vsha041 on Aug 15, 2022

@vsha041 I see the same exception how do I update my code below:

public override void Configure(IFunctionsHostBuilder builder)
{
            base.Configure(builder);
            builder.Services.AddSingleton<IServiceClient>((services) =>
            {
                return new ServiceClient(CreateAuthProviderFromSecret(value));
            })
            .AddScoped<IGraphRepository, GraphRepository>()            
            .AddSingleton<IBlobRepository, BlobRepository>((s) =>
            {
                var configuration = s.GetService<IConfiguration>();
                var a = configuration["a"];                
                return new BlobRepository(a);
            });
}
+1
lipalath-ms on Aug 12, 2022

@Shuvajyoti - it was something along these lines but I will re-check again and let you know next week. By the way make sure the azure function configuration variable AzureFunctionsWebHost__hostId is set to lowercase GUID without the dashes. E.G. = 34c63f7c44ee4271ba7ffa02c8c883a9

Before

services.AddScoped(x => new P());
services.AddScoped(x => 
{
  x.propA = x.getService<P> (); // this line was failing to resolve after upgrading to V4.
});

After

var p = new P();
services.AddScoped(x => p);
services.AddScoped(x => 
{
  x.propA = p;
});
+1
vsha041 on May 13, 2022
Read more comments on GitHub
← azure-functions-host: DryIoc.ContainerException : Scope disposed{no name, Parent=disposed{no name, Parent=disposed{no name}}} is disposed and scoped instances are disposed and no longer available
azure-functions-host: Error on build for in-portal functions with NuGet references →