vertx-web: Changed network communication in case of HTTP response code 413 after upgrade from Vert.x 4.2.4 to 4.3.4

Version

The issue was introduced by upgrading Vert.x from previously used version 4.2.4 to version 4.3.4.

Context

I am running a Vert.x core HTTP server behind a Nginx load balancer and limiting the body size via a BodyHandler. Before the upgrade from Vert.x 4.2.4 everything worked as expected, i.e., the Vert.x core HTTP responded with HTTP 413 Payload Too Large response status in case of hitting the body limit and Nginx simply passed this response to the client. After the upgrade to Vert.x 4.3.4 we are always getting HTTP 502 Bad Gateway response status from Nginx when hitting the body limit. After enabling request and network activity logging and checking the Nginx error log I found that with Vert.x 4.3.4 Nginx changes HTTP 413 Payload Too Large response status to HTTP 502 Bad Gateway response status due to an error

2022/10/28 21:08:12 [error] 9072#9072: *3583678 upstream prematurely closed connection while reading response header from upstream, client: 172.156.0.99, server: management.datahub5.stage.c8y.io, request: "POST /service/datahub/scheduler/gettriggerstatus HTTP/1.1", upstream: "http://172.156.0.197:8303/service/datahub/scheduler/gettriggerstatus", host: "cdh-testing2.datahub5.stage.c8y.io:443"

Therefore, I had a look at the network communication using Wireshark and it turns out that network communication differs between Vert.x 4.2.4 and 4.3.4. Screenshot With Vert.x 4.2.4 (left window) the response is sent after consuming the whole TCP stream. With Vert.x 4.3.4 (right window) the response is sent after consuming the first TCP segment which contains the Content-Length header. It looks like this leads to a connection close which interrupts response processing in Nginx.

Do you have a reproducer?

Unfortunately not since this is not a trivial setup and it took me already a lot of time to pin it down.

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 29 (11 by maintainers)

Most upvoted comments

After some additional testing it looks like the changed network communication was introduced by commit 4e1c42aaa66d3b1b11f86a0784a2c4b07a623d9a or more precisely by the following piece of code which fails the request directly after reading the headers:

      // before parsing the body we can already discard a bad request just by inspecting the content-length against
      // the body limit, this will reduce load, on the server by totally skipping parsing the request body
      if (bodyLimit != -1 && parsedContentLength != -1) {
        if (parsedContentLength > bodyLimit) {
          context.fail(413);
          return;
        }
      }

riemenschneider on Nov 24, 2022