kubernetes: Masters should not be excluded from service load balancers

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug

What happened: I’m running a single node cluster on AWS EC2, ie. scheduling work on the master as well. Now that I tried to add an ELB for my service, the EC2 instance is not associated with the ELB. The ELB gets created but there are 0 EC2 instances associated with it. I’m using ELB for SSL termination, if you wonder why I’d like to load balance a single node cluster.

The service controller logs this message:

I0628 17:54:48.853175       1 event.go:221] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"nginx", UID:"59980933-7afc-11e8-9a8c-0621b993d602", APIVersion:"v1", ResourceVersion:"2052", FieldPath:""}): type: 'Warning' reason: 'UnAvailableLoadBalancer' There are no available nodes for LoadBalancer service default/nginx

What you expected to happen: The EC2 instance is associated with the ELB.

How to reproduce it (as minimally and precisely as possible):

  • Boot an EC2 instance with correct IAM permissions
  • Install a single node cluster using kubeadm and cloud-provider=aws
  • Untaint the node
  • Schedule eg. nginx
  • Create an external load balancer for nginx
  • View the ELB in AWS console and note the Status: 0 of 0 instances in service

Anything else we need to know?: The reason for this seems to be the node-role.kubernetes.io/master label. This blocks associating a load balancer with the node. On the other hand this changed what is included, because includeNodeFromNodeList did not check if a node is a master. I’m not sure what would be correct fix. I could try submit a PR, if you guide me how this should behave. Is my scenario even a supported one?

I think this bug should be reproducible on other clouds as well.

Environment:

  • Kubernetes version (use kubectl version):
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:17:28Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:08:34Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
  • Cloud provider or hardware configuration: AWS
  • OS (e.g. from /etc/os-release):
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
  • Kernel (e.g. uname -a):
Linux ip-x.eu-central-1.compute.internal 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 (2018-05-07) x86_64 GNU/Linux
  • Install tools: kubeadm

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 51 (38 by maintainers)

Commits related to this issue

Most upvoted comments

The implementation of Services and LB needs to be distinct from the API around it. Not every implementation uses the same techniques or even uses kube-proxy. This is why I have historically been against such API changes, but I admit we have not come up with a workable compromise.

In other words please don’t do dev work on this because there’s not a clear design.

I’ll pick up this soon @andrewsykim