undici: When Content-Length is specified, fetch() raises RequestContentLengthMismatchError on redirection
Bug Description
When the Content-Length
request header field is manually specified (instead of automatically generated), fetch()
raises a RequestContentLengthMismatchError
in case of redirection.
Reproducible By
Send a POST
request to the resource identified by the /sirene/public/recherche
target URI to search for a non-existing company whose name foobarbaz
is provided in the request body:
const uri = 'https://www.sirene.fr/sirene/public/recherche';
const method = 'POST';
const body = 'recherche.sirenSiret=&recherche.raisonSociale=foobarbaz&recherche.adresse=&recherche.commune=&recherche.excludeClosed=true&__checkbox_recherche.excludeClosed=true&recherche.captcha=';
const headers = {'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': Buffer.byteLength(body)};
const response = await fetch(uri, {method, headers, body});
The resource replies with a 302
response with a Location: /sirene/error/autre.action
header field. So fetch
automatically (by default) redirects the original request to the target URI /sirene/error/autre.action
and raises a RequestContentLengthMismatchError
:
Uncaught TypeError: fetch failed
at Object.fetch (node:internal/deps/undici/undici:11413:11)
at async REPL5:1:47 {
cause: RequestContentLengthMismatchError: Request body length does not match content-length header
at write (node:internal/deps/undici/undici:9907:41)
at _resume (node:internal/deps/undici/undici:9885:33)
at resume (node:internal/deps/undici/undici:9787:7)
at connect (node:internal/deps/undici/undici:9776:7) {
code: 'UND_ERR_REQ_CONTENT_LENGTH_MISMATCH'
}
}
Expected Behavior
The specified Content-Length
request header field is correct in the original request so I don’t expect a RequestContentLengthMismatchError
after automatic redirection.
Logs & Screenshots
I assume that during redirection, fetch()
resends the original request but with the following modifications:
- the target URI is changed from
/sirene/public/recherche
to/sirene/error/autre.action
; - the
POST
request method is changed toGET
; - the request body is removed.
But it doesn’t remove the content-specific header fields (Content-Type
and Content-Length
here), contrary to what the latest HTTP specification RFC 9110 recommends:
When automatically following a redirected request, the user agent SHOULD resend the original request message with the following modifications:
- Replace the target URI with the URI referenced by the redirection response’s Location header field value after resolving it relative to the original request’s target URI.
- […]
- […]
- Change the request method according to the redirecting status code’s semantics, if applicable.
- If the request method has been changed to GET or HEAD, remove content-specific header fields, including (but not limited to) Content-Encoding, Content-Language, Content-Location, Content-Type, Content-Length, Digest, Last-Modified.
The non-compliance of Undici to point 5 likely causes the RequestContentLengthMismatchError
. Indeed, sending a GET
request without a body but with a Content-Length
header field raises the same RequestContentLengthMismatchError
:
fetch('http://example.com/', {headers: {'Content-Length': 0}})
Output:
Uncaught TypeError: fetch failed
at Object.fetch (node:internal/deps/undici/undici:11413:11) {
cause: RequestContentLengthMismatchError: Request body length does not match content-length header
at write (node:internal/deps/undici/undici:9907:41)
at _resume (node:internal/deps/undici/undici:9885:33)
at resume (node:internal/deps/undici/undici:9787:7)
at connect (node:internal/deps/undici/undici:9776:7) {
code: 'UND_ERR_REQ_CONTENT_LENGTH_MISMATCH'
}
}
Environment
MacOS 11.6.7, Node 19.7.0.
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 25 (21 by maintainers)
Commits related to this issue
- fix(fetch): remove content-length header on redirect Fixes https://github.com/nodejs/undici/issues/2021 — committed to KhafraDev/undici by KhafraDev a year ago
- fix(fetch): remove content-length header on redirect (#2022) Fixes https://github.com/nodejs/undici/issues/2021 — committed to nodejs/undici by KhafraDev a year ago
- fix(fetch): remove content-length header on redirect (#2022) Fixes https://github.com/nodejs/undici/issues/2021 — committed to metcoder95/undici by KhafraDev a year ago
- fix(fetch): remove content-length header on redirect (#2022) Fixes https://github.com/nodejs/undici/issues/2021 — committed to crysmags/undici by KhafraDev a year ago
digest isn’t a filtered header name and last-modified is only a CORS safelisted header (not something undici implements)
I have a branch that makes it spec compliant (see: https://github.com/KhafraDev/undici/tree/change-headerslist-to-array), but after benchmarking, the performance was so bad that I decided against it. I actually published that branch for this exact reason - to show how badly it performs compared to our current impl.
I don’t think we should filter any headers in fetch. Adding content-length to requestBodyHeader fixes the issue in a spec-compliant & rfc compliant way.
I’ve purposefully kept it close to Rafael’s version, as it makes headers faster by tenfold (using a map instead of an array). Unfortunately that decision snowballed to where the implementation is completely different from what the spec says to do.
Oh nvm, adding
content-length
to that list fixes it lolThe headers are being removed in fetch, this is a bug elsewhere: https://github.com/nodejs/undici/blob/e6fc80f809d1217814c044f52ed40ef13f21e43c/lib/fetch/index.js#L1177-L1195
where requestBodyHeader is
Here’s a small repro
The RFC mentioned in the OP recommends removing some headers on redirect, that’s likely what we should do.
No, the
content-length
header is a forbidden header, but we don’t filter headers.