envoy: Update to oghttp2=true causes cookie-related failures
With updating from v1.28.0 to v1.29.1 came runtime flag envoy.reloadable_features.http2_use_oghttp2
defaulting to true.
This caused failures in some of our integration tests, which, after several days of trying to track down what the problem was, I finally isolated to the fact that while envoy was still reporting in its debug log router decoding headers
message “cookie: a; b; c”, the headers at the upstream were getting interpreted as “cookie: c”.
I think that what’s probably happening is the codec is restructuring the headers as
cookie: a
cookie: b
cookie: c
and some library in the upstream is (in violation of http spec) interpreting this as three “set” operations that each overwrite the previous cookie value in a dictionary/map type structure.
Flipping the flag back to false fixes the immediate problem; I’m not sure if this is an issue that should be addressed in envoy, or if it’s appropriate in these circumstances for envoy to say “the spec says it’s okay to restructure that kind of header so the problem is at the upstream.”
Note: I’m also not certain this is what’s happening, but it’s an interpretation that fits the data I have; that the upstream sees only the last cookie, and that envoy’s router filter debug logs all the cookies, and that flipping this flag changes that. From this data it could be that the codec is where the lost-cookie flattening is happening, in which case that’s a much bigger bug. 😃
About this issue
- Original URL
- State: open
- Created 4 months ago
- Comments: 16 (14 by maintainers)
Commits related to this issue
- Adds the ability to skip `Cookie` crumbling in `HpackEncoder`. This is preparation for fixing the issue described in https://github.com/envoyproxy/envoy/issues/32611. Protected by new code path, not... — committed to google/quiche by birenroy 4 months ago
- Disables Cookie header field crumbling in OgHttp2Session by default. Addresses https://github.com/envoyproxy/envoy/issues/32611. PiperOrigin-RevId: 613371306 — committed to google/quiche by birenroy 4 months ago
- http2: Switch the value of envoy.reloadable_features.http2_use_oghttp2 to false (#32751) http2: Switch the value of envoy.reloadable_features.http2_use_oghttp2 to false A number of users have report... — committed to envoyproxy/envoy by RyanTheOptimist 4 months ago
- http2: Switch the value of envoy.reloadable_features.http2_use_oghttp2 to false (#32751) http2: Switch the value of envoy.reloadable_features.http2_use_oghttp2 to false A number of users have reporte... — committed to phlax/envoy by RyanTheOptimist 4 months ago
- http2: Switch the value of envoy.reloadable_features.http2_use_oghttp2 to false (#32751) http2: Switch the value of envoy.reloadable_features.http2_use_oghttp2 to false A number of users have reporte... — committed to envoyproxy/envoy by RyanTheOptimist 4 months ago
- repo: Release v1.29.2 **Summary of changes**: - Revert default codec to nghttp2 as a number of users have reported issues with oghttp2 including #32611 and #32401 - Assorted minor fixes **Docker ... — committed to phlax/envoy by phlax 4 months ago
- repo: Release v1.29.2 **Summary of changes**: - Revert default codec to nghttp2 as a number of users have reported issues with oghttp2 including #32611 and #32401 - Assorted minor fixes **Docker ... — committed to envoyproxy/envoy by phlax 4 months ago
- 1.29.2 (#18) * build(deps): bump distroless/base-nossl-debian12 from `51ab103` to `49edf70` in /ci (#32348) build(deps): bump distroless/base-nossl-debian12 in /ci Bumps distroless/base-nossl-d... — committed to solo-io/envoy-fork by jbohanon 4 months ago
/assign @birenroy