aws-load-balancer-controller: Controller can't attach ALB security group to eks cluster security group

Hi, The alb controller provides two ingress annotations to work with AWS security groups to restrict traffic access.

alb.ingress.kubernetes.io/security-groups

alb.ingress.kubernetes.io/inbound-cidrs

In my scenario, alb.ingress.kubernetes.io/security-groups is applied.

However, while trying to navigate to the ingress URL, the website application was not accessible with a 504 error. So I checked the target group health check and found that all the backend were in unhealthy status.

I noticed that the cause of those phenomena is that the security group of the EKS cluster did not include the alb security group so that the health check was unable to work normally.

On the other hand, if applying alb.ingress.kubernetes.io/inbound-cidrs, ALB would work well since the alb security is covered by the security group of the EKS cluster.

Is there any roadmap for our ALB controller project to fix the problem?

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 12
Comments: 22 (2 by maintainers)

Most upvoted comments

Starring v2.3 release, controller supports the annotation alb.ingress.kubernetes.io/manage-backend-security-group-rules. When you set the annotation to “true”, controller attaches the shared backend security group to the ALB, and create ingress rules on the Node/Pod SG to allow traffic from the shared SG.

Closing the issue.

kishorj on Jan 28, 2022

Hi Is there any news on this matter?

rdeteix on Jan 4, 2022

also, i think we can change the default behavior, so that we reuse a single LB-securityGroup shared for all ALBs in the cluster, and create a another additional LB-securityGroup per ALB. the shared LB-securityGroup will be granted access to worker nodes at any port number in your cluster. while the additional LB-securityGroup per ALB is used to encode the inbound client CIDRs that is allowed to access ALB(the client CIDR might differs per ALB). The benefit of this approach is we’ll only use a single worker node securityGroup rule no matter how many ALBs we have. The only downside of approach is we have a securityGroup that allowed to access worker node at any port number, some customers might treat this as a security threat. however, it looks secure to me as we’ll only attach this security group to LoadBalancers.

We can use the same approach for NLBs too and share the same shared LB-securityGroup for both ALB/NLB.

What do you guys think? @schollii @skerava

M00nF1sh on Feb 10, 2021

I have same issue. The docs provide an explanation for this:

When this annotation is not present, the controller will automatically create one security groups: the security group will be attached to the LoadBalancer and allow access from inbound-cidrs to the listen-ports. Also, the securityGroups for Node/Pod will be modified to allow inbound traffic from this securityGroup.

The annotation supports multiple security groups being on ALB, so I can see why the controller does not automatically apply them to the nodes’ security group. However the controller should automatically add the specified SG if there is only one in the annotation, AND it should support having some SG that are added automatically, and some that are not. This could be done with a second annotation such as

alb.ingress.kubernetes.io/security-groups-auto-add: [list]

If auto-add is not given, all security group in alb.ingress.kubernetes.io/security-groups list would be auto added.

Then the following annotations on an ingress would cause ALB to have SG A, B, C, but only B to be added. Ingress groups still work with this approach.

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: your-ingress
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/group.name: group-name
    alb.ingress.kubernetes.io/security-groups: sg-A, sg-B, sg-C
    alb.ingress.kubernetes.io/security-groups-auto-add: sg-B

For now it seems that the workaround is to manually (or via terraform or such automation tool) add the ALB SG to the cluster SG.

schollii on Feb 2, 2021