aws-load-balancer-controller: Error adding targets to target group,InvalidInstanceID: There are multiple interfaces attached to instance

Deployed alb controller on AWS EKS. Attached policy to worker nodes. Used 2048-game example to test. Below are the config files:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: "2048-ingress"
  namespace: "2048-game"
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/target-type: ip
    #alb.ingress.kubernetes.io/security-groups: sg-0941bddfa4d5a39a5
    alb.ingress.kubernetes.io/subnets: subnet-xxxx, subnet-xxxx
  labels:
    app: 2048-ingress
spec:
  rules:
  - http:
      paths:
      - path: /
        backend:
          serviceName: "service-2048"
          servicePort: 80

apiVersion: v1
kind: Service
metadata:
  name: "service-2048"
  namespace: "2048-game"
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
  selector:
    app: "2048"

Getting below error:

Event(v1.ObjectReference{Kind:“Ingress”, Namespace:“2048-game”, Name:“2048-ingress”, UID:“7c48f85e-a0ad-11e8-99ae-061cda1a6546”, APIVersion:“extensions/v1beta1”, ResourceVersion:“133013”, FieldPath:“”}): type: ‘Warning’ reason: ‘ERROR’ Error adding targets to target group xxxxxxxxxx: InvalidInstanceID: There are multiple interfaces attached to instance ‘i-xxxxxxx’. Please specify an interface ID for the operation instead.


And instance security groups are not attached to nodes.

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 27

Most upvoted comments

This root cause is alb-ingress controller calls ModifyInstanceAttribute when attaching security groups to nodes (which allows alb to route traffic into it). According to https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_ModifyInstanceAttribute.html, it can cause errors when nodes already have multiple ENIs attached. I’ll change it to use ModifyNetworkInterfaceAttribute instead, and submit a PR for this

M00nF1sh on Aug 30, 2018

BTW, for the multiple-interface issue, an more robust change have been merged into master, and available as quay.io/coreos/alb-ingress-controller:latest, so no need to use docker.io/m00nf1sh/alb-ingress-controller:1.0-beta.7 anymore 😸

The fix in master works in both instance mode and ip mode

M00nF1sh on Sep 20, 2018

Hi Sashanka, The issue of 2048 can be resolved by https://github.com/kubernetes-sigs/aws-alb-ingress-controller/pull/636, (only need to update path:/ to path:/*)

I’ll resolve this issue 😸

M00nF1sh on Sep 21, 2018

I’m able to reproduce this issue by creating an alb ingress after multiple ENI attach to the node (can happen when creating multiple pods as each pod will consume one secondary ip)

Will start to investigate the root cause and solve this

M00nF1sh on Aug 30, 2018

A full fix for this issue should be ready by this week(works on both ip mode & instance mode)

For now to unblock you, you can update the deployment of ingress controller to use docker.io/m00nf1sh/alb-ingress-controller:1.0-beta.7, which is based on https://github.com/M00nF1sh/aws-alb-ingress-controller/tree/sg-instance-temp-fix This temporary fix only works for instance mode(alb.ingress.kubernetes.io/target-type: instance)

M00nF1sh on Sep 12, 2018