skywalking: apm-spring-cloud-gateway-2.1.x-plugin-8.1.0 plug-in, an error "TracingContext: More than 300 spans required to create" is reported during stress testing

Please answer these questions before submitting your issue.

• Why do you submit this issue?
• Question or discussion
• • [✔️ ] Bug
• Requirement
• Feature or performance improvement

---

Bug

• Which version of SkyWalking, OS and JRE? 1.spring-cloud-starter-gateway-2.1.5.RELEASE 2.java version “1.8.0_141” 3.skywalking 8.1.0

• What happened?

Use an interface’/user-center/query’ of the jmeter pressure measurement gateway, and the function is provided by the service’user-center’. Stress test effect: 8 times of ‘More than 300 spans required to create’ error occurred at 10,000 times. As an additional note, the abnormal rate reached 89% due to too frequent calls. jmeter report:

The Error stack detail : WARN 2020-08-13 11:13:12:730 reactor-http-nio-5 TracingContext : More than 300 spans required to create java.lang.RuntimeException: Shadow tracing context. Thread dump at org.apache.skywalking.apm.agent.core.context.TracingContext.isLimitMechanismWorking(TracingContext.java:562) at org.apache.skywalking.apm.agent.core.context.TracingContext.createLocalSpan(TracingContext.java:297) at org.apache.skywalking.apm.agent.core.context.ContextManager.createLocalSpan(ContextManager.java:102) at org.apache.skywalking.apm.plugin.spring.cloud.gateway.v21x.NettyRoutingFilterInterceptor.beforeMethod(NettyRoutingFilterInterceptor.java:40) at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInterWithOverrideArgs.intercept(InstMethodsInterWithOverrideArgs.java:75) at org.springframework.cloud.gateway.filter.NettyRoutingFilter.filter(NettyRoutingFilter.java) at org.springframework.cloud.gateway.handler.FilteringWebHandler$GatewayFilterAdapter.filter(FilteringWebHandler.java:138) at org.springframework.cloud.gateway.filter.OrderedGatewayFilter.filter(OrderedGatewayFilter.java:44) at org.springframework.cloud.gateway.handler.FilteringWebHandler$DefaultGatewayFilterChain.lambda$filter$0(FilteringWebHandler.java:118) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.MonoPeekTerminal.subscribe(MonoPeekTerminal.java:61) at reactor.core.publisher.Mono.subscribe(Mono.java:3879) at reactor.core.publisher.FluxRetryWhen.subscribe(FluxRetryWhen.java:86) at reactor.core.publisher.MonoRetryWhen.subscribe(MonoRetryWhen.java:50) at reactor.core.publisher.MonoRepeatWhen.subscribe(MonoRepeatWhen.java:79) at reactor.core.publisher.MonoSourceFlux.subscribe(MonoSourceFlux.java:47) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.MonoPeekTerminal.subscribe(MonoPeekTerminal.java:61) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.Mono.subscribe(Mono.java:3879) at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.drain(MonoIgnoreThen.java:172) at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:56) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.MonoPeek.subscribe(MonoPeek.java:71) at reactor.core.publisher.Mono.subscribe(Mono.java:3879) at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.drain(MonoIgnoreThen.java:172) at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:56) at reactor.core.publisher.MonoPeekFuseable.subscribe(MonoPeekFuseable.java:74) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.drain(MonoIgnoreThen.java:153) at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:56) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:150) at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:67) at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:121) at reactor.core.publisher.FluxContextStart$ContextStartSubscriber.onNext(FluxContextStart.java:103) at reactor.core.publisher.FluxContextStart$ContextStartSubscriber.onNext(FluxContextStart.java:103) at reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onNext(FluxMapFuseable.java:287) at reactor.core.publisher.FluxFilterFuseable$FilterFuseableConditionalSubscriber.onNext(FluxFilterFuseable.java:331) at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1623) at reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:160) at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:136) at reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:252) at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:136) at reactor.netty.channel.FluxReceive.terminateReceiver(FluxReceive.java:427) at reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:211) at reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:369) at reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:369) at reactor.netty.http.server.HttpServerOperations.onInboundNext(HttpServerOperations.java:496) at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:92) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at reactor.netty.http.server.HttpTrafficHandler.channelRead(HttpTrafficHandler.java:214) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Unknown Source)

Checked the ui trace, it feels a bit strange。

Comparing several red entries, we found: The endpoint information is consistent: “VNode: da87aad68dfd479292e868af12e61647.99.15972883737272806”; The component information is different. As shown