request: Getting HPE_UNEXPECTED_CONTENT_LENGTH trying to proxy HTTPS request

I experienced this issue last week, in my case the problem was the server was adding a Content-Length header even when Transfer-Encoding was set to chunked. As @felicienfrancois said, this goes against the HTTP spec and in my case was a bug in the server. Node.js used to tolerate this incorrect server behaviour and parse the response anyway, but has now become more strict.

I don’t think request is doing anything wrong, you should fix the server if possible, or perhaps raise a request in https://github.com/nodejs/node and ask if they would consider adding a flag to ignore the error and provide compatibility with buggy servers.

hi, if you has this problem yet, you should check the server resopnse has had both ofcontent-length、transfer-encoding . The little tip, the koa server will automatic add the content-length in the response headers

@gajus as I said before, this happen when both Content-Length and Transfer-encoding: chunked headers are set, which is invalid according to HTTP spec.

To make it go through, just remove one of the 2 headers before making the request

if (headers['transfer-encoding'] === 'chunked') {
    delete headers['content-length'];
}
request({url: url, headers: headers}, callback);

As a workaround for the web-crawler usage, AFAIK, there’s no way to get the Node HTTP parser to accept this, but you can continue using request and other high-level modules and just monkey-patch in a different low-level HTTP parser:

// Monkey patch before you require http or request for the first time.
process.binding('http_parser').HTTPParser = require('http-parser-js').HTTPParser;

Note: monkey patching does not work in certain kinds of child processes starting with Node v10.x, so if you’re doing something tricky, you might need to use an older Node version until that is resolved.

http-parser-js was just updated to better deal with chunked responses that erroneously have a Content-Length header, but if you run into the opposite (non-chunked responses that erroneously have a Transfer-Encoding: chunked header), open an issue with http-parser-js and we can probably get a work-around or add an option to control that particular behavior.

Is there any workaround or fix for this? I’m trying to set up a web crawler, and the external server is sending down Content-Length twice (as the same value). It’d be nice if we didn’t error out just because the server was a little strange.

Any workarounds for situations where the API is not available? Using the --security-revert=CVE-2016-2216 does not ignore this.