contour: Envoy segfaults and errors when ExternalName service is defined
What steps did you take and what happened: When executing the below code to define an externalName service all Envoy pods segfault and error.
apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
name: externaltest
namespace: default
spec:
routes:
- services:
- name: externaltest
port: 443
requestHeadersPolicy:
set:
- name: Host
value: projectcontour.io
---
apiVersion: v1
kind: Service
metadata:
labels:
run: externaltest
name: externaltest
namespace: default
annotations:
projectcontour.io/upstream-protocol.tls: 443,https
spec:
externalName: projectcontour.io
ports:
- name: https
port: 443
protocol: TCP
targetPort: 443
type: ExternalName
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: externaltest
namespace: projectcontour
spec:
secretName: externaltest
commonName: "externaltest.mydomain.com"
dnsNames:
- "externaltest.mydomain.com"
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
---
apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
name: externaltest
namespace: projectcontour
spec:
virtualhost:
fqdn: externaltest.mydomain.com
tls:
secretName: externaltest
includes:
- name: externaltest
namespace: default
Logs from an Envoy pod:
[2021-01-10 01:55:23.463][1][critical][main] [source/exe/terminate_handler.cc:13] std::terminate called! (possible uncaught exception, see trace)
[2021-01-10 01:55:23.463][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:91] Backtrace (use tools/stack_decode.py to get line numbers):
[2021-01-10 01:55:23.463][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:92] Envoy version: 8fb3cb86082b17144a80402f5367ae65f06083bd/1.16.0/Clean/RELEASE/BoringSSL
[2021-01-10 01:55:23.463][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #0: [0x55608e88fe0b]
[2021-01-10 01:55:23.463][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #1: [0x55608e88fc79]
[2021-01-10 01:55:23.463][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #2: [0x55608ee9ef93]
[2021-01-10 01:55:23.463][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #3: [0x55608e748542]
[2021-01-10 01:55:23.463][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #4: [0x55608e2dbcfd]
[2021-01-10 01:55:23.463][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #5: [0x55608e0c31d5]
[2021-01-10 01:55:23.464][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #6: [0x55608e0c6773]
[2021-01-10 01:55:23.464][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #7: [0x55608e0c5b15]
[2021-01-10 01:55:23.464][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #8: [0x55608e0c634b]
[2021-01-10 01:55:23.464][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #9: [0x55608e0c4d50]
[2021-01-10 01:55:23.464][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #10: [0x55608e2de8a5]
[2021-01-10 01:55:23.464][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #11: [0x55608e2d8366]
[2021-01-10 01:55:23.464][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #12: [0x55608e710fe8]
[2021-01-10 01:55:23.464][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #13: [0x55608e70f9be]
[2021-01-10 01:55:23.464][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #14: [0x55608e2bb79c]
[2021-01-10 01:55:23.464][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #15: [0x55608d619008]
[2021-01-10 01:55:23.464][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #16: [0x55608d619807]
[2021-01-10 01:55:23.464][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #17: [0x55608d617bdc]
[2021-01-10 01:55:23.464][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #18: __libc_start_main [0x7f1979649b97]
[2021-01-10 01:55:23.464][1][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] Caught Aborted, suspect faulting address 0x1
Pods:
kubectl get pods -n projectcontour 389ms Sun 10 Jan 13:15:25 2021
NAME READY STATUS RESTARTS AGE
contour-8556b76b45-h2g5k 1/1 Running 0 8m56s
contour-8556b76b45-xm5ht 1/1 Running 0 8m56s
contour-certgen-v1.11.0-f5zs9 0/1 Completed 0 8m56s
envoy-7vbvh 1/2 Error 1 8m56s
envoy-9bdzh 1/2 Error 1 8m56s
envoy-xmnf8 1/2 Error 1 8m56s
What did you expect to happen: Service is created successfully
Environment:
- Contour version: -1.11.0
- Kubernetes version: (use
kubectl version
): Server Version: version.Info{Major:“1”, Minor:“17”, GitVersion:“v1.17.14”, GitCommit:“f238f5142728be4033c37aa0ad69bf806090beae”, GitTreeState:“clean”, BuildDate:“2020-11-11T13:03:54Z”, GoVersion:“go1.13.15”, Compiler:“gc”, Platform:“linux/amd64”} - Kubernetes installer & version: Rancher K8S latest stable
- Cloud provider or hardware configuration: self hosted, 3 node vmware cluster
- OS (e.g. from
/etc/os-release
): Ubuntu 20.04
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 15 (11 by maintainers)
Right you are; I missed that. I worked this out and put together an article about it so hopefully others don’t get stuck. https://alexpturner.medium.com/exposing-external-services-in-contour-5a4adcd3dfbd
Thanks for the help
Upstream image. I just ran the quick start manifest here: https://projectcontour.io/getting-started/#option-1-quickstart
Thanks for logging this @alexanderturner. I’ll ping @stevesloka as he has the most experience with ExternalName services. Any thoughts Steve? Something about TLS ExternalNames maybe?
I’ve marked this as “Needs investigation” because we haven’t seen anything like this before.