quarkus: IllegalReferenceCountException logged with enabled compression

Hi

Describe the bug I’m seeing this in 1.11.1.Final after enabling compression via quarkus.http.enable-compression:

2021-01-29 07:19:42,608 WARN  [io.net.cha.ChannelOutboundBuffer] (vert.x-eventloop-thread-4) Failed to mark a promise as success because it has failed already: DefaultChannelPromise@3d236c66(failure: io.netty.handler.codec.EncoderException: io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1), unnotified cause:: io.netty.handler.codec.EncoderException: io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
        at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:104)
        at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717)
        at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:709)
        at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:792)
        at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:702)
        at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:697)
        at io.netty.handler.codec.MessageToMessageEncoder.writePromiseCombiner(MessageToMessageEncoder.java:137)
        at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:117)
        at io.netty.handler.codec.MessageToMessageCodec.write(MessageToMessageCodec.java:116)
        at io.vertx.core.http.impl.HttpChunkContentCompressor.write(HttpChunkContentCompressor.java:38)
        at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717)
        at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:709)
        at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:792)
        at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:702)
        at io.netty.handler.stream.ChunkedWriteHandler.doFlush(ChunkedWriteHandler.java:300)
        at io.netty.handler.stream.ChunkedWriteHandler.flush(ChunkedWriteHandler.java:132)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:750)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:742)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:728)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:127)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:750)
        at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:765)
        at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:790)
        at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:758)
        at io.vertx.core.net.impl.ConnectionBase.write(ConnectionBase.java:124)
        at io.vertx.core.net.impl.ConnectionBase.lambda$queueForWrite$2(ConnectionBase.java:215)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
        at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74)
        at io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100)
        at io.vertx.core.http.impl.AssembledHttpResponse.release(AssembledHttpResponse.java:159)
        at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:88)
        at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:91)
        ... 32 more

Expected behavior There should be no exception logged as the content is transfered without any problem.

To Reproduce

Steps to reproduce the behavior:

  1. Enable compression
  2. Access a (reactive [Mutiny]?) endpoint

I’ll try to create a reproducer if needed be.

Configuration

quarkus:
  http:
    auth:
      proactive: true
    cors:
      ~: true
      origins: http://localhost:3000
      methods: DELETE, GET, POST, PUT, OPTIONS
    root-path: /
    limits:
      max-body-size: 20480K
    enable-compression: true
  resteasy:
    gzip:
      enabled: true
      max-input: 20480K

Environment (please complete the following information):

  • Output of uname -a or ver: Linux voyager 5.10.11-arch1-1 #1 SMP PREEMPT Wed, 27 Jan 2021 13:53:16 +0000 x86_64 GNU/Linux
  • Output of java -version:
openjdk version "11.0.10" 2021-01-19
OpenJDK Runtime Environment (build 11.0.10+8)
OpenJDK 64-Bit Server VM (build 11.0.10+8, mixed mode)
  • GraalVM version (if different from Java):
  • Quarkus version or git rev: 1.11.1.Final
  • Build tool (ie. output of mvnw --version or gradlew --version):
❯ ./gradlew --version

------------------------------------------------------------
Gradle 6.8.1
------------------------------------------------------------

Build time:   2021-01-22 13:20:08 UTC
Revision:     31f14a87d93945024ab7a78de84102a3400fa5b2

Kotlin:       1.4.20
Groovy:       2.5.12
Ant:          Apache Ant(TM) version 1.10.9 compiled on September 27 2020
JVM:          11.0.10 (Oracle Corporation 11.0.10+8)
OS:           Linux 5.10.11-arch1-1 amd64

Additional context

  • Adding “quarkus-undertow” dependency like stated in #9194 does not remove the exception.
  • Disabling compression removes the exception.

/edit: fixed the statement about the tried workaround – it’s not working.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 3
  • Comments: 17 (9 by maintainers)

Commits related to this issue

Most upvoted comments

Turns out my fix was incomplete: https://github.com/netty/netty/pull/11358

It fixes it if compression is actually used, but it fails if no accept-encoding header is present.

The fix is in Netty 4.1.61 which has not been included yet.