spring-boot: Context not being reused in tests when MockBeans are used

Hi!

I would like to start a discussion about the mechanism of caching application contexts that are using @MockBeans across spring boot tests.

I was following this guide about testing the web layer: https://spring.io/guides/gs/testing-web/ and I was surprised that application context was build independently for every test class. I was hoping it will be cached to decrease test time, but I was wrong.

I’ve created small sample app to show this. Here is pull request: https://github.com/spring-projects/spring-boot-issues/pull/56

There’s one @Service (FooBarService) used by controller (FooBarApplication) in two methods. And two tests that have some expectations about the web layer (as indicated by the @WebMvcTest annotation). Those two tests work on the same context - the only difference is distinct mocks. But if you do mvn clean test you’ll see the spring banner twice, which means that context for the web layer is created two times.

I’ve done some research and debugging and found that this is caused by including the @MockBean and @SpyBean definitions in the key for context cache (through the MockitoContextCustomizer that is a part of MergedContextConfiguration).

Well… After rethinking all of this I can understand that from technical point of view those contexts are not the same, because mock@1 and mock@2 are not equal. But still I feel that my assumption that context should be reused is what users could expect. Especially if we realize that @WebMvcTest is used to reduce the test time by not starting tomcat. Having this goal in mind it’s hard to accept the extra time of repeated context building.

I’m not sure if such a change is a bug fixing or enhancement. I feel that this behaviour is bug but you can discuss it.

For now my workaround (or maybe it’s official way to do this?) is to create an abstract test class with all @MockBeans definitions, something like:

@RunWith(SpringRunner.class)
@WebMvcTest
public abstract class AbstractTest {
    protected @MockBean FooBarService service;
}

public class FooTest extends AbstractTest {...}

But maybe it’s better to consider an approach in which context is build once and mocks are replaced in beforeTestClass and afterTestClass methods of TestExecutionListener? And to avoid refreshing dependencies in every class referencing those mocked beans maybe it’s ok to generate some beanToMockProxy object, that will be autowired in controllers and will hold a reference to mockito mock (created per test class), proxing all the methods to that mock? I don’t know… I’m just thinking loud.

Cheers, Alek

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 18
Comments: 26 (13 by maintainers)

Commits related to this issue

Start building against Spring Framework 4.3.4 snapshots Closes gh-7174 See gh-7213 — committed to spring-projects/spring-boot by wilkinsona 8 years ago
Fix web integration tests not reusing application context Each controller integration test class would start a new application context. This is because of the MockBean annotation. This would cause th... — committed to OpenLMIS/openlmis-referencedata by chunky56 7 years ago
Korjattu Spring Context vuotoja testeissä. - https://github.com/spring-projects/spring-boot/issues/7174 - https://github.com/spring-projects/spring-boot/issues/9282 - Korjattu Spring Boot sovelluksen ... — committed to tmfg/digitraffic-marine by sirkkalap 6 years ago

Most upvoted comments

Thank you all very much for taking care of that 😃

I know that the issue is closed but I’d like to follow up on the topic of context caching. You’ve fixed the mechanism of calculating cache’s keys based on mock definitions. But I’m not convinced for the general idea of including mocks in that key 😦

Please consider one more example that I’ve provided in issues repo. The PR is here: https://github.com/spring-projects/spring-boot-issues/pull/59

In that example I have two tests, that are testing separate features and are independent. And they are using the same WebMvc context - the only difference is in mocks used in both. Despite that, spring is creating two contexts (which may take long time) instead of one (run mvn test to see banner twice).

My feeling is that developer’s decision of what to mock and when shouldn’t have impact on context creation. I’d expect in that case one application context, shared between tests and with @MockBeans created and injected into context only for one test class.

What do you think?

+13

alebar on Oct 25, 2016

@philwebb In reality for any project there would be at least of 40 integration tests (bare minimum) and its may not be possible for having integration tests without mocking. Or maintaining same mocks for all tests. Reloading context for 40 times Will easily make test time 1hr that very bad for testing For engineering perspective, It does not mandatory required for rebuilding context on configuration change. If proxies are maintained - this can be handled very easily. Like proxy holder contains service holder and that pointing address can be changed on mocking. We don’t have to recreate all beans or change objects of each bean where mocked DI is there. And Unit FW will take care of copy cache and modify that proxy bean so scope can only to be on that test I suggest or feel like this should have to be new feature of spring boot test - which will be required for many people who are using spring.

surapuramakhil on Jul 27, 2020

Any test that uses org.springframework.boot.test.mock.mockito.MockBean reloads the context, for example for example

@MockBean
private RestTemplate template;

and this prevents me from testing events, when it reloads the context, spring loses itself in counting parallel processes.

eutiagocosta on Aug 15, 2017

This is fixed in the latest Spring Framework 4.3.4 snapshots. Overriding spring.version in the project that reproduced the problem results in this output:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running example.BarTest

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.4.1.RELEASE)

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.47 sec - in example.BarTest
Running example.FooTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in example.FooTest

Results :

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

Note that the banner only appears once, i.e. the context has been reused.

wilkinsona on Oct 24, 2016

@brolinuk Spring context will be rebuild even if you have different mock in beans.

Like , for example we have 2 test classes and 26 beans(a … z) testClassA -> mocks beanA testClassB -> mocks beanB

in above case spring context will be initialized twice

surapuramakhil on Jan 21, 2021

At the time when we’re getting the context cache key to decide whether a context can be reused or we need to create a new one, we have no knowledge at all of how the context’s beans will be wired together. We also have no knowledge of the functionality that your test is going to exercise. This means that it’s impossible for us to know that your FooTest can happily run with a mocked BarService and that your BarTest can happily run with a mocked FooService.

wilkinsona on Oct 26, 2016