aspnetcore: ConfigureTestContainer not working with GenericHost

OK waking up and re-reading this issue. IT does seem like a bug that should be fixed and potentially patched though it is risky (maybe it’ll need to be quirked). I will follow up with some people.

All due respect, I am sorry but the provided answers to this issues are simply not satisfactory.

When we moved to generic host, we changed that behavior and now the callbacks are queued in the order in which you register them, so when you do UseStartup the callback gets immediately queued. For that reason, if you are using generic host you shouldn’t need the ConfigureTestContainer overload, you can simply call hostBuilder.ConfigureContainer after the call to UseStartup to tweak the container configuration.

This does not work, as demonstrated in Tratcher’s post here: https://github.com/aspnet/AspNetCore/issues/14907#issuecomment-542388201

@javiercn 's description in https://github.com/aspnet/AspNetCore/issues/14907#issuecomment-541011786 implies to be the intended behaviour is that Startup.ConfigureContainer should be called after Program.ConfigureContainer if the later is called after UseStarup. This is not the case which means it is a plain bug.

None of the technique above are working, meaning that there is no apparent nor easy workaround. I’m excluding @cdibbs 's techniques which prevents testing in parallel.

This issue prevents from having a proper way for tests to override dependencies, which is a huge issue for us (and I suspect many other). We have tons of tests doing exactly that and this bug prevents us from moving from 2.1 to 3.1

This is a big show-stopper for anyone who wants to properly test his application.

I have the exact same issue. Is there a workaround that would allow us to bypass this issue?

@cdibbs no changes are planned for 3.1.

Here is a workaround for Autofac that will call any ConfigureTestContainer(...) method, also in .WithWebHostBuilder(...) calls.

It builds on @ssunkari’s workaround

public class CustomWebApplicationFactory<TStartup> : WebApplicationFactory<TStartup>
        where TStartup : class
{
	protected override void ConfigureWebHost(IWebHostBuilder webHostBuilder)
	{
		webHostBuilder.ConfigureTestContainer<Autofac.ContainerBuilder>(builder =>
		{
			// called after Startup.ConfigureContainer
		});
	}

	protected override IHost CreateHost(IHostBuilder builder)
	{
		builder.UseServiceProviderFactory(new CustomServiceProviderFactory());
		return base.CreateHost(builder);
	}
}

public class CustomServiceProviderFactory : IServiceProviderFactory<CustomContainerBuilder>
{
	public CustomContainerBuilder CreateBuilder(IServiceCollection services) => new CustomContainerBuilder(services);

	public IServiceProvider CreateServiceProvider(CustomContainerBuilder containerBuilder) =>
		new AutofacServiceProvider(containerBuilder.CustomBuild());
}

public class CustomContainerBuilder : Autofac.ContainerBuilder
{
	private readonly IServiceCollection services;

	public CustomContainerBuilder(IServiceCollection services)
	{
		this.services = services;
		this.Populate(services);
	}

	public Autofac.IContainer CustomBuild()
	{
		var sp = this.services.BuildServiceProvider();
#pragma warning disable CS0612 // Type or member is obsolete
		var filters = sp.GetRequiredService<IEnumerable<IStartupConfigureContainerFilter<Autofac.ContainerBuilder>>>();
#pragma warning restore CS0612 // Type or member is obsolete

		foreach (var filter in filters)
		{
			filter.ConfigureContainer(b => { })(this);
		}

		return this.Build();
	}
}

Usage

var factory = new CustomWebApplicationFactory<Startup>();

var client1 = factory.CreateClient();


var client2 = factory.WithWebHostBuilder(b =>
{
	b.ConfigureTestContainer<Autofac.ContainerBuilder>(builder =>
	{
		// Called after Startup.ConfigureContainer and after CustomWebApplicationFactory's ConfigureTestContainer
	});
}).CreateClient();

This workaround* of creating a derived TestStartup from @RehanSaeed is working for us: https://rehansaeed.com/asp-net-core-integration-testing-mocking-using-moq/

*I describe it as a workaround as it would be nicer not to have to make Configure and ConfigureServices on Startup virtual.

You can override container registrations in the following way, using Autofac as example:

Configure your container in Program instead of Startup.ConfigureContainer(ContainerBuilder):

public static IHostBuilder CreateHostBuilder (string[] args)
{
  return Host.CreateDefaultBuilder (args)
    .UseServiceProviderFactory (new AutofacServiceProviderFactory())
    .ConfigureContainer<ContainerBuilder>(b => { /* configure container here */ })
    .ConfigureWebHostDefaults(builder => builder.UseStartup<Startup>());
}

Override container registrations using ContainerBuilder in a derived WebApplicationFactory:

public class TestWebApplicationFactory : WebApplicationFactory<Startup>
{
  protected override IHost CreateHost(IHostBuilder builder)
  {
    builder.ConfigureContainer<ContainerBuilder>(b => { /* test overrides here */ });
    return base.CreateHost(builder);
  }
}

@GODBS I ended up going the super-hacky route, for now. I hope this is fixed in 3.1.

internal static Action<ContainerBuilder> IntegTestOverrides = null;
public void ConfigureContainer(ContainerBuilder cb)
{
    cb.RegisterModule(new ApiRootModule());
    cb.RegisterType<StartupHttpPipeline>().As<IStartupHTTPPipeline>();
    IntegTestOverrides?.Invoke(cb);
}

@anurse this is really unfortunate… We have a lot of narrow integration tests that are redefining some of the dependencies and swapping them with mocks - calls to services. This bug prevents us from migrating as-is and implies a lot of refactoring, starting with something clean and ending with something ugly. I’m puzzled that something like that was overlooked, after all DI and testing are supposed to be an integral part of this product.

This is a pickle, I agree with @forktrucka that nobody seems to have this working. On the other hand I can see why you don’t want to patch and possible break hypothetical implementations, but here we are with all the 2.x implementations irremediably breaking when going to 3.x

I got it working by implementing custom factory for my container setup. In my case its Autofac. I have tweaked the IServiceProviderFacotry<ContainerBuilder> implementation of Autofac library.

Code

public class CustomAutofacServiceProviderFactory : IServiceProviderFactory<ContainerBuilder>
{
    private readonly Action<ContainerBuilder> _configurationOverride;
    private readonly Action<ContainerBuilder> _configurationAction;
    /// <summary>
    /// Initializes a new instance of the <see cref="AutofacServiceProviderFactory"/> class.
    /// </summary>
    /// <param name="testStartup"></param>
    /// <param name="configurationAction">Action on a <see cref="ContainerBuilder"/> that adds component registrations to the conatiner.</param>
    public CustomAutofacServiceProviderFactory(Action<ContainerBuilder> configurationAction = null, Action<ContainerBuilder> configurationOverride = null)
    {
        _configurationOverride = configurationOverride;
        _configurationAction = configurationAction ?? (builder => { });
    }



    /// <summary>
    /// Creates a container builder from an <see cref="IServiceCollection" />.
    /// </summary>
    /// <param name="services">The collection of services.</param>
    /// <returns>A container builder that can be used to create an <see cref="IServiceProvider" />.</returns>
    public ContainerBuilder CreateBuilder(IServiceCollection services)
    {
        var builder = new ContainerBuilder();

        builder.Populate(services);

        _configurationAction(builder);

        return builder;
    }

    /// <summary>
    /// Creates an <see cref="IServiceProvider" /> from the container builder.
    /// </summary>
    /// <param name="containerBuilder">The container builder.</param>
    /// <returns>An <see cref="IServiceProvider" />.</returns>
    public IServiceProvider CreateServiceProvider(ContainerBuilder containerBuilder)
    {
        if (containerBuilder == null) throw new ArgumentNullException(nameof(containerBuilder));

        _configurationOverride(containerBuilder); // Added this to override container overrides
        var container = containerBuilder.Build();

        return new AutofacServiceProvider(container);
    }
}

Test Setup

public class CustomWebApplicationFactory<TStartup> : WebApplicationFactory<TStartup> where TStartup:class {

    protected override IHost CreateHost(IHostBuilder builder)
    {
        builder.ConfigureContainer<ContainerBuilder>(cb =>
        {
        }).UseServiceProviderFactory(
            new CustomAutofacServiceProviderFactory(
                b => { },
                OverrideContainer));
 

        return base.CreateHost(builder);
    }

} Refer to Autofac docs https://autofaccn.readthedocs.io/en/latest/integration/aspnetcore.html for service registration

The resolution I am using is below.

          public class TestApplicationFactory : WebApplicationFactory<Startup>
        {
            protected override IHost CreateHost(IHostBuilder builder)
            {
                builder.ConfigureContainer<ContainerBuilder>(containerBuilder =>
                    {
           
                    });

                return base.CreateHost(builder);
            }
        }

The ConfigureContainer registered inside WebApplicationFactory is called after ConfigureContainer by the generic host builder inside Program.cs.

I tried this. Seemed to face the same problem in my implementation… What am i missing?

Using 3.1, we’re using autofac and calling ConfigureContainer(ContainerBuilder builder) from startup. This seems to execute in the same order…

You can override container registrations in the following way, using Autofac as example: …

I would replace overriding TestWebApplicationFactory class with an action for added flexibility, like this:

public class TestWebApplicationFactory : WebApplicationFactory<Startup>
{
    ....
    private Action<ContainerBuilder> setupMockServicesAction;

    public TestWebApplicationFactory WithMockServices(Action<ContainerBuilder> setupMockServices)
    {
        setupMockServicesAction = setupMockServices;
        return this;
    }

    protected override IHost CreateHost(IHostBuilder builder)
    {
        builder.ConfigureContainer<ContainerBuilder>(containerBuilder =>
        {
            setupMockServicesAction?.Invoke(containerBuilder);
        });

        return base.CreateHost(builder);
   }
    ....
}

Then, my test method which resides inside an NUnit test fixture which needs to instantiate TestWebApplicationFactory class looks like this:

[TestFixture]
public class CustomExceptionHandlerTests
{
    [Test]
    public async Task HandleException_WhenApiThrowsException_MustConvertExceptionToInstanceOfProblemDetailsClass()
    {
         // Arrange phase
         ...
         using var testWebApplicationFactory =
                new TestWebApplicationFactory(...)
                    .WithMockServices(containerBuilder =>
                    {
                        // Ensure a mock implementation will be injected whenever a service requires an instance of the
                        // IGenerateJwtFlow interface.
                        containerBuilder
                            .RegisterType<GenerateJwtFlowWhichThrowsException>()
                            .As<IGenerateJwtFlow>()
                            .InstancePerLifetimeScope();
                    });

         using HttpClient httpClient = testWebApplicationFactory.CreateClient();
         ...
    }
}

One can employ WithMockServices only when needing to replace a registered service with something else, e.g. a class which will throw an exception whenever one of its methods are called, like in my above example.

The full source code can be found here .

@jr01’s workaround prevents update to Autofac 6.0.0 because they sealed their ContainerBuilder. Therefore a fix would still be appreciated here.

Cleanest solution I could achieve based on previous answers:

Create overrides:

public class TestAutofacModule : Module
{
    protected override void Load(ContainerBuilder builder)
    {
        builder.RegisterType<TestService>().As<IService>();
    }
}

Create new ServiceProviderFactory:

public class TestServiceProviderFactory<T> : IServiceProviderFactory<ContainerBuilder> where T: Module, new()
{
    private readonly AutofacServiceProviderFactory _factory = new();

    public ContainerBuilder CreateBuilder(IServiceCollection services)
    {
        return _factory.CreateBuilder(services);
    }

    public IServiceProvider CreateServiceProvider(ContainerBuilder containerBuilder)
    {
        containerBuilder.RegisterModule(new T());
        return _factory.CreateServiceProvider(containerBuilder);
    }
}

Use new Factory:

var builder = new HostBuilder()
                .UseServiceProviderFactory(new TestServiceProviderFactory<TestAutofacModule>())
                ...

To wrap this up, Alistair tweaked the workaround provided by @jr01 to also work with the sealed ContainerBuilder of Autofac >= 6.0.0. So we can await .NET 5.0 without having to worry that it would break our integration tests.

This issue always makes me facepalm… love ASP.NET Core but I can’t figure out why this is such a painful experience. Feel like we’re so close to being the number one web framework. Especially with Blazor RTM.

It’s been twice in three weeks that I’ve found myself coming back to this post to try and remember the workaround for this issue. I’ll try and turn it into a blog post so people can have a reference for how to remedy the situation until it’s fixed. I wouldn’t want people to not write tests 😉

I agree with @Pvlerick that my workaround doesn’t work when you run the tests in parallel (which is the norm). So, is my workaround really “viable?” Maybe not.

It sounds like any fix would have side effects, but perhaps that just builds a case for doubling up on the normal amount of testing done for a patch like this.