envoy: proxying CONNECT does not seem to work as expected

Title: proxying CONNECT does not seem to work as expected

Description: I am trying to proxy CONNECT requests upstream to another proxy server. The configuration I am using is similar to https://github.com/envoyproxy/envoy/blob/3883a94c0d66fc9c58ca5c11cadf4007b5ee3287/configs/proxy_connect.yaml except I am using HTTP 1.1 instead of HTTP2 for the upstream connect. It looks like Envoy is connecting upstream using TLS instead of a plain HTTP CONNECT. It seems Envoy is terminating the CONNECT even though connect_config is not set.

Config:

static_resources:
      listeners:
      - name: listener_0
        address:
          socket_address:
            protocol: TCP
            address: 0.0.0.0
            port_value: 10000
        filter_chains:
        - filters:
          - name: envoy.filters.network.http_connection_manager
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
              stat_prefix: ingress_http
              route_config:
                name: local_route
                virtual_hosts:
                - name: local_service
                  domains: ["*"]
                  routes:
                  - match:
                      prefix: "/"
                    route:
                      cluster: policy
                  - match:
                      connect_matcher:
                        {}
                    route:
                      cluster: policy
                      upgrade_configs:
                        - upgrade_type: CONNECT
              http_filters:
              - name: envoy.filters.http.router
                typed_config:
                  "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
              http_protocol_options: {}
              upgrade_configs:
                - upgrade_type: CONNECT
      clusters:
     - name: policy
        connect_timeout: 30s
        type: LOGICAL_DNS
        dns_lookup_family: V4_ONLY
        typed_extension_protocol_options:
          envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
            "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
            explicit_http_config:
              http_protocol_options: {}
        load_assignment:
          cluster_name: policy
          endpoints:
          - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: policy
                    port_value: 9080

Logs:

[2022-04-27 09:31:12.663][16][debug][conn_handler] [source/server/active_tcp_listener.cc:140] [C0] new connection from 10.42.0.59:55990
[2022-04-27 09:31:12.663][16][debug][http] [source/common/http/conn_manager_impl.cc:283] [C0] new stream
[2022-04-27 09:31:12.664][16][debug][http] [source/common/http/conn_manager_impl.cc:873] [C0][S3421268804210892329] request headers complete (end_stream=false):
':authority', 'www.example.com:443'
':method', 'CONNECT'
'user-agent', 'curl/7.58.0'
'proxy-connection', 'Keep-Alive'

[2022-04-27 09:31:12.664][16][debug][router] [source/common/router/router.cc:486] [C0][S3421268804210892329] cluster 'policy' match for URL ''
[2022-04-27 09:31:12.665][16][debug][misc] [source/common/upstream/cluster_manager_impl.cc:1744] Allocating TCP conn pool
[2022-04-27 09:31:12.666][16][debug][router] [source/common/router/router.cc:702] [C0][S3421268804210892329] router decoding headers:
':authority', 'www.example.com:443'
':method', 'CONNECT'
':scheme', 'http'
'user-agent', 'curl/7.58.0'
'x-forwarded-proto', 'http'
'x-request-id', '1e0ff772-b055-4f69-b8f5-2d59097d99d5'
'x-envoy-expected-rq-timeout-ms', '15000'

[2022-04-27 09:31:12.666][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:267] trying to create new connection
[2022-04-27 09:31:12.666][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:144] creating a new connection
[2022-04-27 09:31:12.667][16][debug][connection] [source/common/network/connection_impl.cc:896] [C1] connecting to 10.43.110.23:9080
[2022-04-27 09:31:12.668][16][debug][connection] [source/common/network/connection_impl.cc:916] [C1] connection in progress
[2022-04-27 09:31:12.668][16][debug][connection] [source/common/network/connection_impl.cc:680] [C1] connected
[2022-04-27 09:31:12.669][16][debug][connection] [source/common/network/connection_impl.cc:936] [C1] connected on local interface 'eth0'
[2022-04-27 09:31:12.669][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:305] [C1] attaching to next stream
[2022-04-27 09:31:12.670][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:177] [C1] creating stream
[2022-04-27 09:31:12.670][16][debug][router] [source/common/router/upstream_request.cc:418] [C0][S3421268804210892329] pool ready
[2022-04-27 09:31:12.671][16][debug][router] [source/common/router/router.cc:1320] [C0][S3421268804210892329] upstream headers complete: end_stream=false
[2022-04-27 09:31:12.672][16][debug][http] [source/common/http/conn_manager_impl.cc:1472] [C0][S3421268804210892329] encoding headers via codec (end_stream=false):
':status', '200'
'date', 'Wed, 27 Apr 2022 09:31:12 GMT'
'server', 'envoy'

[2022-04-27 09:31:12.681][16][debug][router] [source/common/router/upstream_request.cc:341] [C0][S3421268804210892329] resetting pool request
[2022-04-27 09:31:12.681][16][debug][connection] [source/common/network/connection_impl.cc:138] [C1] closing data_to_write=0 type=1
[2022-04-27 09:31:12.681][16][debug][connection] [source/common/network/connection_impl.cc:249] [C1] closing socket: 1
[2022-04-27 09:31:12.682][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:443] [C1] client disconnected, failure reason: 
[2022-04-27 09:31:12.683][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:410] invoking idle callbacks - is_draining_for_deletion_=false
[2022-04-27 09:31:12.683][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:210] [C1] destroying stream: 0 remaining
[2022-04-27 09:31:12.684][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:410] invoking idle callbacks - is_draining_for_deletion_=false
[2022-04-27 09:31:12.685][16][debug][http] [source/common/http/conn_manager_impl.cc:205] [C0][S3421268804210892329] doEndStream() resetting stream
[2022-04-27 09:31:12.685][16][debug][http] [source/common/http/conn_manager_impl.cc:1523] [C0][S3421268804210892329] stream reset
[2022-04-27 09:31:12.685][16][debug][connection] [source/common/network/connection_impl.cc:138] [C0] closing data_to_write=103 type=2
[2022-04-27 09:31:12.685][16][debug][connection] [source/common/network/connection_impl_base.cc:47] [C0] setting delayed close timer with timeout 1000 ms
[2022-04-27 09:31:12.685][16][debug][connection] [source/common/network/connection_impl.cc:712] [C0] write flush complete
[2022-04-27 09:31:12.688][16][debug][connection] [source/common/network/connection_impl.cc:712] [C0] write flush complete
[2022-04-27 09:31:13.684][16][debug][connection] [source/common/network/connection_impl_base.cc:63] [C0] triggered delayed close
[2022-04-27 09:31:13.684][16][debug][connection] [source/common/network/connection_impl.cc:249] [C0] closing socket: 1

Using curl to access the https://www.example.com

Through Envoy curl --> Envoy --> upstream Proxy --> origin server

$ curl -v -x envoy_ip:10000 https://www.example.com
* Rebuilt URL to: https://www.example.com/
*   Trying 192.168.1.104...
* TCP_NODELAY set
* Connected to(192.168.1.104) port 10000 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.example.com:443
> CONNECT www.example.com:443 HTTP/1.1
> Host: www.example.com:443
> User-Agent: curl/7.58.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 OK
< date: Wed, 27 Apr 2022 10:25:45 GMT
< server: envoy
< 
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* CONNECT phase completed!
* CONNECT phase completed!
* error:1408F10B:SSL routines:ssl3_get_record:wrong version number
* stopped the pause stream!
* Closing connection 0
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number

Direct to proxy curl --> upstream Proxy --> origin server

$ curl -v -x proxy_ip:9080 https://www.example.com
* Rebuilt URL to: https://www.example.com/
*   Trying 10.43.110.23...
* TCP_NODELAY set
* Connected to 10.43.110.23 (10.43.110.23) port 9080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.example.com:443
> CONNECT www.example.com:443 HTTP/1.1
> Host: www.example.com:443
> User-Agent: curl/7.58.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 OK
< Date: Wed, 27 Apr 2022 10:27:13 GMT
< Transfer-Encoding: chunked
* Ignoring Transfer-Encoding in CONNECT 200 response
< 
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* CONNECT phase completed!
* CONNECT phase completed!
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Los Angeles; O=Internet�Corporation�for�Assigned�Names�and�Numbers; CN=www.example.org
*  start date: Mar 14 00:00:00 2022 GMT
*  expire date: Mar 14 23:59:59 2023 GMT
*  subjectAltName: host "www.example.com" matched cert's "www.example.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert TLS RSA SHA256 2020 CA1
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* Using Stream ID: 1 (easy handle 0x55d27f79c620)
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
> GET / HTTP/2
> Host: www.example.com
> User-Agent: curl/7.58.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
< HTTP/2 200 
< accept-ranges: bytes
< age: 286149
< cache-control: max-age=604800
< content-type: text/html; charset=UTF-8
< date: Wed, 27 Apr 2022 10:27:14 GMT
< etag: "3147526947+gzip"
< expires: Wed, 04 May 2022 10:27:14 GMT
< last-modified: Thu, 17 Oct 2019 07:18:26 GMT
< server: ECS (nyb/1D10)
< vary: Accept-Encoding
< x-cache: HIT
< content-length: 1256
< 
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
<!doctype html>
<html>
<head>
    <title>Example Domain</title>

    <meta charset="utf-8" />
   ...
</html>