resilience4j: JVM running out of threads when I use a Time Limiter (circuit breaker + thread pool bulkhead + time limiter)

Resilience4j version: 1.7

Java version: 8

I’m new to resilience4j (though I have moderate experience with Hystrix) and am trying to learn how to use it to optimize performance under load. Currently the code I’ve written uses: a circuit breaker, a thread pool bulkhead, and a TimeLimiter. I’ve observed that when I decorate the supplier using a Time Limiter, under medium load (around 90 requests per second) the JVM runs out of threads and throws the following exception.

! java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached

This effectively kills the service and causes every request to fail until the service is restarted. When I remove the Time Limiter, this issue goes away. Here is my code:

final CheckedFunction0<? extends HttpResponse> supplierWithCircuitBreaker = Decorators .ofCheckedSupplier(supplier).withCircuitBreaker(circuitBreaker).decorate();

ScheduledExecutorService scheduler = Executors.newSingleThreadScheduledExecutor();

final CompletableFuture<? extends HttpResponse> async = Decorators.ofSupplier(() -> {
			try {
				return supplierWithCircuitBreaker.apply();
			} catch (Throwable e) {
				return handleException(e);
			}
		}).withThreadPoolBulkhead(threadPoolBulkhead).withTimeLimiter(timeLimiter, scheduler).get().toCompletableFuture();

async.thenAccept(result -> {
			LOGGER.info(String.format("Returning value: %s", result.getHttpStatus()));
			response.resume(Response.status(result.getHttpStatus()).entity(result).build());
		}).exceptionally(e -> {
			final StandardResponse errorResponse = handleException(e);
			response.resume(Response.status(errorResponse.getHttpStatus()).entity(errorResponse).build());

			// without this the compiler complains
			return null;
		});

Can anyone let me know what I’m doing wrong here?