quarkus: OOM with QuarkusTestResourceConfigurableLifecycleManager
Describe the bug
I replaced our wiremock integration tests with a QuarkusTestResourceLifecycleManager approach. Instead of starting und stopping the wiremock server in the junit BeforeAll and AfterAll methods, I provide a class the looks like the following:
class WiremockServerTestResource: QuarkusTestResourceConfigurableLifecycleManager<WithWiremockedRemoteApis> {
val server = WireMockServer(WireMockConfiguration.options().dynamicPort())
val name = AtomicReference<String>()
override fun init(annotation: WithWiremockedRemoteApis) {
name.set(annotation.name)
}
override fun start(): MutableMap<String, String> {
val mappingSource = JsonFileMappingsSource(ClasspathFileSource("wiremock"))
mappingSource.loadMappingsInto(InMemoryStubMappings())
server.loadMappingsUsing(mappingSource)
server.start()
return mutableMapOf(
"org.acme.UserClient/mp-rest/url" to server.baseUrl()
)
}
@Synchronized
override fun stop() {
server.stop()
}
}
@QuarkusTestResource(WiremockServerTestResourceSlow::class)
@Retention(AnnotationRetention.RUNTIME)
@Target(AnnotationTarget.CLASS)
annotation class WithWiremockedRemoteApis(
val name: String = "Frufru"
)
I annotated all integration tests with @WithWiremockedRemoteApis and basically this approach looks pretty charming. My problem is though, that my tests ran into an OOM with this approach. On my local macbook this does not happen all the time but on our gitlab runners (with lower resources), the tests do not work at all and fail with an OOM.
I also created a sample project that at least shows, that this approach is much slower than the static initialisation: https://github.com/u6f6o/slow_tests I could not reproduce the OOM though 😦 I think there might be an issue with the QuarkusTestResourceConfigurableLifecycleManager as the tests, even for this simple project are much slower in the test setup phase:

Expected behavior
Tests should be almost as fast as with static wiremock initialisation and should not run in to an OOM.
Actual behavior
No response
How to Reproduce?
1.) Check out https://github.com/u6f6o/slow_tests 2.) Run tests in org.acme.static 3.) Run tests in org.acme.quarkuslifecycle 4.) Compare results (OOM does not occur in this example though)
Output of uname -a or ver
No response
Output of java -version
No response
GraalVM version (if different from Java)
No response
Quarkus version or git rev
No response
Build tool (ie. output of mvnw --version or gradlew --version)
No response
Additional information
No response
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 19 (10 by maintainers)
Commits related to this issue
- CI: Prevent OOM/GCLocker-issue in tests / disable Micrometer JVM-Metrics for tests TL;DR Quarkus restarts (due to profile/configuration changes) causes memory leaks with Micrometer's JVM GC Metrics. ... — committed to snazy/nessie by snazy a year ago
- CI: Prevent OOM/GCLocker-issue in tests / disable Micrometer JVM-Metrics for tests TL;DR Quarkus restarts (due to profile/configuration changes) causes memory leaks with Micrometer's JVM GC Metrics. ... — committed to snazy/nessie by snazy a year ago
- CI: Prevent OOM/GCLocker-issue in tests / disable Micrometer JVM-Metrics for tests (#6050) TL;DR Quarkus restarts (due to profile/configuration changes) causes memory leaks with Micrometer's JVM GC ... — committed to projectnessie/nessie by snazy a year ago
For tests, yes
There seems to be some deep leak in JMX that I don’t see how we can work around. The good news is though that if you exclude the micrometer extension, the OOM in the tests goes away. So I am going to close this as there isn’t much we can do on our side
Creating a heap dump is no problem but for the real life service, this dump may also contain sensitive data and I am most likely not allowed to share the dump. Have to figure out if there is any way to obfuscate the data etc.
In the meantime here is a screenshot of the leak suspects that MAT found:

The thing is that as soon as you add
@QuarkusTestResourceto another annotation (a stereotype), then this resource is treated as ifrestrictToAnnotatedClasswas set totrue. So, each of your annotated test classes will start and stop the wiremock server, instead of starting one server once and keeping it up until the last test class is done. I cannot say I like this behaviour, but this is just how it works currently. There is a hint here: https://quarkus.io/guides/getting-started-testing#annotation-based-test-resourcesI all you tests need that resource, you’re better off adding that
@QuarkusTestResourceonce to some class, e.g.ModuleWideTestResourcesor so. It will be picked up automatically. If you need more flexibility then I’d suggest@TestProfile.As for the OOM, there might a leak due to the number of (stopped) wiremock servers that might still be around, but we’d need a memory dump to check that.
/cc @geoand