aws-load-balancer-controller: Internal load balancer throws curl: (52) Empty reply from server
I exposed a k8s service using internal load balancer using annotation
kubectl patch svc argo-server -n argo -p '{"metadata": {"annotations": {"service.beta.kubernetes.io/aws-load-balancer-internal": "true"}}}'
It worked for past 6 months with EKS k8s 1.17 but stopped working with EKS K8s upgrade to 1.19 and giving me (52) Empty reply from server for curl.
I tried
service.beta.kubernetes.io/aws-load-balancer-internal: "0.0.0.0/0"
and
service.beta.kubernetes.io/aws-load-balancer-scheme: "internal"
both gives same error.
I see external ip is exposed with service.beta.kubernetes.io/aws-load-balancer-internal: "true" but unable to access it
kubectl get svc argo-server -n argo
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
argo-server LoadBalancer 172.20.135.242 internal-ae99b272671214b38bba978529ccc81f-1242061336.us-east-1.elb.amazonaws.com 2746:32105/TCP 11h
❯ curl http://internal-ae99b272671214b38bba978529ccc81f-1242061336.us-east-1.elb.amazonaws.com:2746/
curl: (52) Empty reply from server
P.S - EKS k8s version - 1.19
Any suggestion?
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 20
@mikeinton Glad to hear you were able to narrow down to the root cause of your issue.
Yes, there is indeed limits on number of NLBs per cluster due to worker security group rule limitation. We are aware of this and work is underway to address this limitation, stay tuned 😄.