netty: io.netty.util.internal.InternalThreadLocalMap,memory leak

Expected behavior

No memory leaks occur

Actual behavior

undeploy my web application.

Steps to reproduce

I use elasticsearch jar and it contains netty. when I close the ESClient in the contextDestroyed, the Glassfish logs show severe error, as follows.

Minimal yet complete reproducer code (or URL to code)

The web application [/service_generic_log] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@5eeeaf73]) and a value of type [io.netty.util.internal.InternalThreadLocalMap] (value [io.netty.util.internal.InternalThreadLocalMap@6304ac92]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

Netty version

netty-3.10.6 Final.jar
netty-buffer,codec,codec-http,common,handler,transport,handler-4.1.9.Final.jar

JVM version (e.g. `java -version`)

openjdk version “1.8.0_111” OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-2ubuntu0.16.04.2-b14) OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)

OS version (e.g. `uname -a`)

Linux lyk 4.6.2-040602-generic #201606100516 SMP Fri Jun 10 09:18:34 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

About this issue

Original URL
State: open
Created 7 years ago
Reactions: 6
Comments: 33 (13 by maintainers)

Commits related to this issue

netty: use singleton for the custom allocator. The allocator has a circular reference that prevents it from GC'ed, thus causes memory leak if gRPC Channels are created and shutdown (even cleanly) on ... — committed to zhangkun83/grpc-java by zhangkun83 5 years ago
netty: use singleton for the custom allocator. (#6526) The allocator has a circular reference that prevents it from GC'ed, thus causes memory leak if gRPC Channels are created and shutdown (even cl... — committed to grpc/grpc-java by zhangkun83 5 years ago
netty: use singleton for the custom allocator. (#6526) The allocator has a circular reference that prevents it from GC'ed, thus causes memory leak if gRPC Channels are created and shutdown (even cl... — committed to dfawley/grpc-java by zhangkun83 5 years ago

Most upvoted comments

Iam also having the same issue

SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [apprity] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1994ff12]) and a value of type [io.netty.util.internal.InternalThreadLocalMap] (value [io.netty.util.internal.InternalThreadLocalMap@7d6a4e1e]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

is there any solution for this ?

vidhyasriramulu on Aug 18, 2018

Hi, I uploaded a repository with steps for reproducing the issue: https://github.com/scruz-denodo/netty-thread-local-issue

It contains two ways for checking the problem.

Only with Netty.
Netty + Tomcat for seeing the log error.

Some comments:

It seems that InternalThreadLocalMap is attached as ThreadLocal to the thread that executes the connect on the client side:

   bootstrap = new Bootstrap();
   b.connect(host, port);   //Here

The threadlocal remains at the thread when netty connections are closed.
Thread used for opening the connection remains with that threadLocal value, so:
- it could be a problem reusing the same thread for opening a different connection?. At the Only wtih Netty sample the thread are from a ExecutorService, so in a real application those threads could be used for any other task. Also, when problem happens at Netty + Tomcat, those threads are the tomcat executor threads “http-nio-exec”, so they are shared between different applications.

Thanks!

scruz-denodo on Dec 1, 2020

@skonx try adding in this…

@WebListener
public class MyListener implements ServletContextListener {
	@Override
	public void contextInitialized(ServletContextEvent sce) {
	}
	@Override
	public void contextDestroyed(ServletContextEvent sce) {
		InternalThreadLocalMap.destroy();
	}
}

useless

Fsstyle on Apr 12, 2019