lettuce-core: lettuce cluster multiGet RedisCommandTimeoutException cause memory leak

I used lettuce 5.0.5.RELEASE and Spring Boot 2.0

and I write the code to multiGet value

  @Autowired
   private RedisTemplate<String, byte[]> redisTemplate;

        redisTemplate.opsForValue().multiGet(
            ids.stream()
                .map(RedisVideoProfileCache::genCacheKey)
                .collect(Collectors.toSet())
        )

Whenever I start the service for a long time, some app’s memory is leaked I download a jmap to debug it

 num     #instances         #bytes  class name
----------------------------------------------
   1:       5668369     3397294608  [B
   2:       9985354      455769512  [Ljava.lang.Object;
   3:       9797176      313509632  java.util.HashMap$Node
   4:       9700404      232809696  java.util.concurrent.CompletableFuture
   5:       8891675      213400200  java.util.ArrayList
   6:       3189943      127597720  io.lettuce.core.output.KeyValueListOutput
   7:       2802968      112118720  java.util.concurrent.CompletableFuture$BiRelay
   8:       3190015      102080480  io.lettuce.core.protocol.AsyncCommand
   9:       3190015      102080480  io.lettuce.core.protocol.Command
  10:       2358668       94346720  java.util.concurrent.CompletableFuture$UniWhenComplete
  11:       1077001       92868160  [C
  12:       3218728       77249472  io.lettuce.core.protocol.CommandArgs$KeyArgument
  13:       2399019       76768608  java.util.ArrayList$Itr
  14:       3190015       76560360  io.lettuce.core.protocol.CommandArgs
  15:        107450       76403064  [Ljava.util.HashMap$Node;
  16:       1572519       62900760  java.util.concurrent.CompletableFuture$UniApply
  17:       2411845       57884280  io.lettuce.core.KeyValue
  18:       3224435       51590960  java.lang.Integer
  19:        801185       32047400  io.lettuce.core.cluster.ClusterCommand
  20:        786213       31448520  java.util.concurrent.CompletableFuture$UniHandle
  21:       1251582       30037968  java.util.concurrent.CompletableFuture$CoCompletion
  22:        225886       29931880  [I
  23:        457000       25592000  org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation
  24:        786283       25161056  io.lettuce.core.DefaultConnectionFuture
  25:        826014       23765088  [Ljava.util.concurrent.CompletableFuture;
  26:        853506       20484144  java.lang.String
  27:        786652       18879648  io.lettuce.core.cluster.PooledClusterConnectionProvider$$Lambda$1028/1326396018
  28:        786652       18879648  io.lettuce.core.cluster.PooledClusterConnectionProvider$$Lambda$1030/1117636922
  29:        786652       18879648  io.lettuce.core.cluster.PooledClusterConnectionProvider$$Lambda$1031/481259366

the byte[] used large than normal case

When I checked this app error log, I found large count RedisCommandTimeoutException

 less c67003c59bbb854f176124c48d4c172a0d7aff1b1d14725705a7d3e57aad9e4a.log  | grep ERROR | grep  "io.lettuce.core.RedisCommandTimeoutException: Command timed out" | wc
  12804  320100 3111372

less c67003c59bbb854f176124c48d4c172a0d7aff1b1d14725705a7d3e57aad9e4a.log  | grep ERROR | grep -v "io.lettuce.core.RedisCommandTimeoutException: Command timed out" | wc
     20     467    4835

I suspect this exception caused a memory leak, why ?


About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 19 (4 by maintainers)

Most upvoted comments

Watch out for the channel handlers (in particular CommandHandler) and the attached DefaultEndpoint. It’s interesting to inspect the state of the connection, whether it’s connected and how many instances of connections you have. If you don’t have any concerns regarding data privacy issues with your heap dump, feel free to share it with me.