envoy: TcpProxy improperly sets `:protocol=bytestream` for HTTP2

TcpProxy improperly sets :protocol=bytestream for HTTP2: When using TunnelingConfig for an HTTP/2 tunnel, it send an out-of-spec psuedo header.

Description:

https://github.com/envoyproxy/envoy/blob/main/source/common/tcp_proxy/upstream.cc#L282-L285

Envoy will always set :protocol, but it’s not part of the HTTP/2 spec:

https://datatracker.ietf.org/doc/html/rfc7540#section-8.1.2.3

An extended CONNECT spec does mention it however:

https://datatracker.ietf.org/doc/html/rfc8441#section-3

This becomes a problem for picky implementations, like Go/Golang’s net/http:

Repro steps:

Run. a Go HTTP/2 server with GODEBUG="http2debug=2"
Create a listener with TunnelingConfig (UsePost=false)
Send traffic from the listener to a cluster with Http2ProtocolOptions AllowConnect and a static endpoint which is a Golang HTTP2 server.
You will see http2: invalid pseudo headers: invalid pseudo-header ":protocol"

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 41 (35 by maintainers)

Commits related to this issue

http: fixing up CONNECT to be RFC compliant (#21440) Risk Level: Medium (data plane changes) Testing: updated tests Docs Changes: n/a Release Notes: inline Runtime guard: envoy.reloadable_feature... — committed to envoyproxy/envoy by alyssawilk 2 years ago
http: fixing up CONNECT to be RFC compliant (#21440) Risk Level: Medium (data plane changes) Testing: updated tests Docs Changes: n/a Release Notes: inline Runtime guard: envoy.reloadable_features.us... — committed to tyxia/envoy by alyssawilk 2 years ago
http: fixing up CONNECT to be RFC compliant (#21440) Risk Level: Medium (data plane changes) Testing: updated tests Docs Changes: n/a Release Notes: inline Runtime guard: envoy.reloadable_features.us... — committed to Amila-Rukshan/envoy by alyssawilk 2 years ago

Most upvoted comments

any workaround to apply?

the websocket Request Envoy send to a golang upstream http server always be rejected now with below:

“2022/05/31 21:18:40 http2: invalid pseudo headers: invalid pseudo-header “:protocol””

This is specifically about non-websocket CONNECT usage. For websockets, Envoy is correct - :protocol must be used for HTTP2 WS. Go doesn’t support this, so you should use HTTP1.

howardjohn on Jun 1, 2022

@moderation RFC9113 still refers to “TCP streams”, so it doesn’t clarify things. The draft RFC that David mentioned is an updated HTTP semantics which is decoupled from the underlying transport. I think the concern is that this new HTTP semantics may not be universally supported, but same can be said about Extended CONNECT.

kyessenov on Jun 7, 2022

Or from a different perspective: “extended CONNECT” extends the normal CONNECT, which is intended for proxying HTTPS/TCP with extra functionality, like WS or UDP. It is not deprecating or replacing the original CONNECT, just extends it.

On Tue, May 31, 2022 at 8:21 AM David Schinazi @.***> wrote:

Sorry for the delay, I was on vacation last week. My apologies, but @PiotrSikora https://github.com/PiotrSikora your understanding of CONNECT is incorrect. Here is the relevant text from the latest spec https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-semantics-19#section-9.3.6 :

CONNECT is intended for use in requests to a proxy. The recipient can establish a tunnel either by directly connecting to the server identified by the request target or, if configured to use another proxy, by forwarding the CONNECT request to the next inbound proxy. An origin server MAY accept a CONNECT request, but most origin servers do not implement CONNECT.

The only correct and standardized way to tunnel TCP in HTTP is regular CONNECT. The is no Extended CONNECT :protocol that is defined for proxying TCP.

— Reply to this email directly, view it on GitHub https://github.com/envoyproxy/envoy/issues/20378#issuecomment-1142277475, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAUR2VOSQPLLUKH2MKAEODVMYU5ZANCNFSM5Q42HXRA . You are receiving this because you commented.Message ID: @.***>

costinm on May 31, 2022

Also was the first link supposed to be https://datatracker.ietf.org/doc/html/rfc8441 ?

alyssawilk on May 19, 2022