quarkus: OOM with QuarkusTestResourceConfigurableLifecycleManager

Describe the bug

I replaced our wiremock integration tests with a QuarkusTestResourceLifecycleManager approach. Instead of starting und stopping the wiremock server in the junit BeforeAll and AfterAll methods, I provide a class the looks like the following:

class WiremockServerTestResource: QuarkusTestResourceConfigurableLifecycleManager<WithWiremockedRemoteApis> {
    val server = WireMockServer(WireMockConfiguration.options().dynamicPort())
    val name = AtomicReference<String>()

    override fun init(annotation: WithWiremockedRemoteApis) {
        name.set(annotation.name)
    }

    override fun start(): MutableMap<String, String> {
        val mappingSource = JsonFileMappingsSource(ClasspathFileSource("wiremock"))
        mappingSource.loadMappingsInto(InMemoryStubMappings())

        server.loadMappingsUsing(mappingSource)
        server.start()

        return mutableMapOf(
            "org.acme.UserClient/mp-rest/url" to server.baseUrl()
        )
    }

    @Synchronized
    override fun stop() {
        server.stop()
    }
}


@QuarkusTestResource(WiremockServerTestResourceSlow::class)
@Retention(AnnotationRetention.RUNTIME)
@Target(AnnotationTarget.CLASS)
annotation class WithWiremockedRemoteApis(
    val name: String = "Frufru"
)

I annotated all integration tests with @WithWiremockedRemoteApis and basically this approach looks pretty charming. My problem is though, that my tests ran into an OOM with this approach. On my local macbook this does not happen all the time but on our gitlab runners (with lower resources), the tests do not work at all and fail with an OOM.

I also created a sample project that at least shows, that this approach is much slower than the static initialisation: https://github.com/u6f6o/slow_tests I could not reproduce the OOM though 😦 I think there might be an issue with the QuarkusTestResourceConfigurableLifecycleManager as the tests, even for this simple project are much slower in the test setup phase: image

Expected behavior

Tests should be almost as fast as with static wiremock initialisation and should not run in to an OOM.

Actual behavior

No response

How to Reproduce?

1.) Check out https://github.com/u6f6o/slow_tests 2.) Run tests in org.acme.static 3.) Run tests in org.acme.quarkuslifecycle 4.) Compare results (OOM does not occur in this example though)

Output of uname -a or ver

No response

Output of java -version

No response

GraalVM version (if different from Java)

No response

Quarkus version or git rev

No response

Build tool (ie. output of mvnw --version or gradlew --version)

No response

Additional information

No response

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 19 (10 by maintainers)

Commits related to this issue

Most upvoted comments

For tests, yes

There seems to be some deep leak in JMX that I don’t see how we can work around. The good news is though that if you exclude the micrometer extension, the OOM in the tests goes away. So I am going to close this as there isn’t much we can do on our side

Thanks for reporting.

As @famod mentions, a memory dump will go a long way to figuring out the OOM error

Creating a heap dump is no problem but for the real life service, this dump may also contain sensitive data and I am most likely not allowed to share the dump. Have to figure out if there is any way to obfuscate the data etc.

In the meantime here is a screenshot of the leak suspects that MAT found: image image

The thing is that as soon as you add @QuarkusTestResource to another annotation (a stereotype), then this resource is treated as if restrictToAnnotatedClass was set to true. So, each of your annotated test classes will start and stop the wiremock server, instead of starting one server once and keeping it up until the last test class is done. I cannot say I like this behaviour, but this is just how it works currently. There is a hint here: https://quarkus.io/guides/getting-started-testing#annotation-based-test-resources

For example, this defines the @WithKubernetesTestServer annotation, which you can use on your tests to activate the KubernetesServerTestResource, but only for the annotated test class.

I all you tests need that resource, you’re better off adding that @QuarkusTestResource once to some class, e.g. ModuleWideTestResources or so. It will be picked up automatically. If you need more flexibility then I’d suggest @TestProfile.

As for the OOM, there might a leak due to the number of (stopped) wiremock servers that might still be around, but we’d need a memory dump to check that.

/cc @geoand