nacos: The web application [ROOT] appears to have started a thread named [com.alibaba.nacos.naming.*] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
2020-11-03 16:59:46.088 [main] WARN o.a.c.loader.WebappClassLoaderBase [173] - The web application [ROOT] appears to have started a thread named [com.alibaba.nacos.naming.beat.sender] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) 2020-11-03 16:59:46.089 [main] WARN o.a.c.loader.WebappClassLoaderBase [173] - The web application [ROOT] appears to have started a thread named [com.alibaba.nacos.naming.failover] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) 2020-11-03 16:59:46.090 [main] WARN o.a.c.loader.WebappClassLoaderBase [173] - The web application [ROOT] appears to have started a thread named [com.alibaba.nacos.naming.push.receiver] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: java.net.PlainDatagramSocketImpl.receive0(Native Method) java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:143) java.net.DatagramSocket.receive(DatagramSocket.java:812) com.alibaba.nacos.client.naming.core.PushReceiver.run(PushReceiver.java:73) java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745)
and then the java process exit!
p.s. “-XX:+HeapDumpOnOutOfMemoryError”,but there is no dump output.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 20
我本地项目成功复现该问题。 mave依赖中包含spring-boot-starter-actuator.
application配置:
启动类:
本地的nacos-server没有进行启动,8848端口不进行监听。项目启动时,会去注册实例到nacos-server, 但是nacos-server没有启动,就会出现connection refused的报错,导致spring初始化失败,关闭tomcat容器。而警告日志就是在关闭tomcat容器的时候打出来的。
tomcat中打警告日志的逻辑见:
org.apache.catalina.loader.WebappClassLoaderBase#clearReferencesThreads
. 这个org.apache.catalina.loader.WebappClassLoaderBase
的实现类其实就是org.springframework.boot.web.embedded.tomcat.TomcatEmbeddedWebappClassLoader
.下面是clearReferencesThreads方法的部分逻辑。
Springboot程序在启动失败过后,会去关闭Tomcat容器,在关闭Tomcat容器的时候会扫描线程,如果对应的线程满足一下几个点,就会打警告日志。
1.判断线程的ContextClassLoader是否和是自己, 也就是
org.springframework.boot.web.embedded.tomcat.TomcatEmbeddedWebappClassLoader
. 2.判断该线程不是当前线程。 满足两点过后,再判断当前线程的栈帧中是否存在org.apache.catalina.connector.CoyoteAdapter
,如果包含这个类,说明当前线程正在处理请求,打警告日志:如果不包含这个类的话,打警告日志:
这个就是打警告日志的原因。
创建线程时,线程的contextClassLoader和创建线程的线程的contextClassLoader是一致的。 以下来自Thread的构造函数的部分相关逻辑:
所以,之前的判断打警告日志时,对应线程的classLoader只要是
org.springframework.boot.web.embedded.tomcat.TomcatEmbeddedWebappClassLoader
就会打相关的警告日志。如果创建相关的线程时,Thread.currentThread()的contextClassLoader对应的是
TomcatEmbeddedWebappClassLoader
的话, 就会发生之前打警告日志的问题。 这涉及到org.springframework.boot.web.embedded.tomcat.TomcatEmbeddedContext
初始化的一个逻辑,见方法org.apache.catalina.core.StandardContext#startInternal
.相关逻辑如下:其中bindThread()方法会把当前线程的contextClassLoader,也就是AppClassLoader替换成TomcatEmbeddedWebappClassLoader. 后续在finnaly再执行 unbindThread(oldCCL), 再把AppClassLoader进行还原。 在bindThread和unbindThread之间,就会做一些listener的相关start. 关键在于
// Call ServletContainerInitializers
, 这里会把org.springframework.boot.web.servlet.ServletContextInitializer
类型的bean进行获取,org.springframework.boot.actuate.endpoint.web.ServletEndpointRegistrar
这个类是spring-boot-starter-actuator
中的,它实现了ServletContextInitializer
接口,也会在// Call ServletContainerInitializers
进行加载。在加载过程中,就会进行ServletEndpointRegistrar的相关实现类加载,而在sca中,存在类com.alibaba.cloud.nacos.endpoint.NacosDiscoveryEndpointAutoConfiguration
,其中有nacos相关的endpoint的实现,这之中会进行NacosNamingService的初始化,这个时候初始化时,contextClassLoader还没有进行unBind, 所以这时创建的线程的contextClassLoader都是TomcatEmbeddedWebappClassLoader。如果不引入
spring-boot-starter-actuator
的情况下,NacosNamingService初始化已经是unbindThread之后了,这个时候当前线程的contextClassLoader已经还原成了AppClassLoader。因此后续如果在spring的初始化周期抛出了没有catch的异常,进行tomcat销毁时,就会判断线程的contextClassLoader是否是TomcatEmbeddedWebappClassLoader,在不引入
spring-boot-starter-actuator
的情况下,线程的contextClassLoader是AppClassLoader, 不会打警告日志。引入之后,contextClassLoader是TomcatEmbeddedWebappClassLoader,会打警告日志