undici: When Content-Length is specified, fetch() raises RequestContentLengthMismatchError on redirection

Bug Description

When the Content-Length request header field is manually specified (instead of automatically generated), fetch() raises a RequestContentLengthMismatchError in case of redirection.

Reproducible By

Send a POST request to the resource identified by the /sirene/public/recherche target URI to search for a non-existing company whose name foobarbaz is provided in the request body:

const uri = 'https://www.sirene.fr/sirene/public/recherche';
const method = 'POST';
const body = 'recherche.sirenSiret=&recherche.raisonSociale=foobarbaz&recherche.adresse=&recherche.commune=&recherche.excludeClosed=true&__checkbox_recherche.excludeClosed=true&recherche.captcha=';
const headers = {'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': Buffer.byteLength(body)};
const response = await fetch(uri, {method, headers, body});

The resource replies with a 302 response with a Location: /sirene/error/autre.action header field. So fetch automatically (by default) redirects the original request to the target URI /sirene/error/autre.action and raises a RequestContentLengthMismatchError:

Uncaught TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11413:11)
    at async REPL5:1:47 {
  cause: RequestContentLengthMismatchError: Request body length does not match content-length header
      at write (node:internal/deps/undici/undici:9907:41)
      at _resume (node:internal/deps/undici/undici:9885:33)
      at resume (node:internal/deps/undici/undici:9787:7)
      at connect (node:internal/deps/undici/undici:9776:7) {
    code: 'UND_ERR_REQ_CONTENT_LENGTH_MISMATCH'
  }
}

Expected Behavior

The specified Content-Length request header field is correct in the original request so I don’t expect a RequestContentLengthMismatchError after automatic redirection.

Logs & Screenshots

I assume that during redirection, fetch() resends the original request but with the following modifications:

  • the target URI is changed from /sirene/public/recherche to /sirene/error/autre.action;
  • the POST request method is changed to GET;
  • the request body is removed.

But it doesn’t remove the content-specific header fields (Content-Type and Content-Length here), contrary to what the latest HTTP specification RFC 9110 recommends:

When automatically following a redirected request, the user agent SHOULD resend the original request message with the following modifications:

  1. Replace the target URI with the URI referenced by the redirection response’s Location header field value after resolving it relative to the original request’s target URI.
  2. […]
  3. […]
  4. Change the request method according to the redirecting status code’s semantics, if applicable.
  5. If the request method has been changed to GET or HEAD, remove content-specific header fields, including (but not limited to) Content-Encoding, Content-Language, Content-Location, Content-Type, Content-Length, Digest, Last-Modified.

The non-compliance of Undici to point 5 likely causes the RequestContentLengthMismatchError. Indeed, sending a GET request without a body but with a Content-Length header field raises the same RequestContentLengthMismatchError:

fetch('http://example.com/', {headers: {'Content-Length': 0}})

Output:

Uncaught TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11413:11) {
  cause: RequestContentLengthMismatchError: Request body length does not match content-length header
      at write (node:internal/deps/undici/undici:9907:41)
      at _resume (node:internal/deps/undici/undici:9885:33)
      at resume (node:internal/deps/undici/undici:9787:7)
      at connect (node:internal/deps/undici/undici:9776:7) {
    code: 'UND_ERR_REQ_CONTENT_LENGTH_MISMATCH'
  }
}

Environment

MacOS 11.6.7, Node 19.7.0.

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 25 (21 by maintainers)

Commits related to this issue

Most upvoted comments

digest isn’t a filtered header name and last-modified is only a CORS safelisted header (not something undici implements)

I have a branch that makes it spec compliant (see: https://github.com/KhafraDev/undici/tree/change-headerslist-to-array), but after benchmarking, the performance was so bad that I decided against it. I actually published that branch for this exact reason - to show how badly it performs compared to our current impl.

Rather than allow everything I think it’s better to deviate on specific header names where it makes sens.

I don’t think we should filter any headers in fetch. Adding content-length to requestBodyHeader fixes the issue in a spec-compliant & rfc compliant way.

Maybe time to have another iteration?

I’ve purposefully kept it close to Rafael’s version, as it makes headers faster by tenfold (using a map instead of an array). Unfortunately that decision snowballed to where the implementation is completely different from what the spec says to do.

Oh nvm, adding content-length to that list fixes it lol

The headers are being removed in fetch, this is a bug elsewhere: https://github.com/nodejs/undici/blob/e6fc80f809d1217814c044f52ed40ef13f21e43c/lib/fetch/index.js#L1177-L1195

where requestBodyHeader is

const requestBodyHeader = [
  'content-encoding',
  'content-language',
  'content-location',
  'content-type'
]

Here’s a small repro

import { once } from 'events'
import { createServer } from 'http'
import { fetch } from './index.js'

const server = createServer((req, res) => {
  if (req.url === '/redirect') {
    res.writeHead(302, { Location: '/redirect2' })
    res.end()
    return
  }

  console.log('hello', req.url)
  res.end()
}).listen(0)

await once(server, 'listening')

await fetch(`http://localhost:${server.address().port}/redirect`, {
  method: 'POST',
  body: 'a+b+c',
  headers: {
    'content-length': Buffer.byteLength('a+b+c')
  }
})

The RFC mentioned in the OP recommends removing some headers on redirect, that’s likely what we should do.

No, the content-length header is a forbidden header, but we don’t filter headers.