kubernetes: kubelet --hostname-override is ignored in v1.2.0-beta.1

Running kubectl get nodes shows the master node name in the default aws format.

NAME                                       LABELS                                                                                                                                                                             STATUS                     AGE
254.1.161.115                              kubernetes.io/hostname=254.1.161.115                                                                                                                                               Ready                      14h
254.1.35.13                                kubernetes.io/hostname=254.1.35.13                                                                                                                                                 Ready                      14h
ip-254-1-0-10.eu-west-1.compute.internal   beta.kubernetes.io/instance-type=m4.large,failure-domain.beta.kubernetes.io/region=eu-west-1,failure-domain.beta.kubernetes.io/zone=eu-west-1a,kubernetes.io/hostname=254.1.0.10   Ready,SchedulingDisabled   14h

The kubelet is started using the following options. I only started seeing this when I used the --register-schedulable=false (kubectl get nodes wouldn’t list the master node previously).

kubelet \
  --address=$private_ipv4 \
  --allow-privileged=true \
  --api-servers=http://127.0.0.1:8080 \
  --cadvisor-port=4194 \
  --cloud-provider=aws \
  --cluster-dns=10.100.0.10 \
  --cluster-domain=cluster.local \
  --config=/etc/kubernetes/manifests \
  --hairpin-mode=promiscuous-bridge \
  --healthz-bind-address=0.0.0.0 \
  --healthz-port=10248 \
  --hostname-override=$private_ipv4 \
  --port=10250 \
  --register-node=true \
  --register-schedulable=false

I believe this is causing new nodes to not be able to register, as I am getting the following errors on a newly added node.

Mar 15 12:23:14 k8s-node kubelet[1009]: E0315 12:23:14.565088    1009 kubelet.go:2697] Error updating node status, will retry: error getting node "254.1.106.101": nodes "254.1.106.101" not found
Mar 15 12:23:14 k8s-node kubelet[1009]: E0315 12:23:14.566390    1009 kubelet.go:2697] Error updating node status, will retry: error getting node "254.1.106.101": nodes "254.1.106.101" not found
Mar 15 12:23:14 k8s-node kubelet[1009]: E0315 12:23:14.567792    1009 kubelet.go:2697] Error updating node status, will retry: error getting node "254.1.106.101": nodes "254.1.106.101" not found
Mar 15 12:23:14 k8s-node kubelet[1009]: E0315 12:23:14.567805    1009 kubelet.go:1173] Unable to update node status: update node status exceeds retry count
Mar 15 12:23:24 k8s-node kubelet[1009]: E0315 12:23:24.569728    1009 kubelet.go:2697] Error updating node status, will retry: error getting node "254.1.106.101": nodes "254.1.106.101" not found
Mar 15 12:23:24 k8s-node kubelet[1009]: E0315 12:23:24.576356    1009 kubelet.go:2697] Error updating node status, will retry: error getting node "254.1.106.101": nodes "254.1.106.101" not found
Mar 15 12:23:24 k8s-node kubelet[1009]: E0315 12:23:24.577641    1009 kubelet.go:2697] Error updating node status, will retry: error getting node "254.1.106.101": nodes "254.1.106.101" not found
Mar 15 12:23:24 k8s-node kubelet[1009]: E0315 12:23:24.578926    1009 kubelet.go:2697] Error updating node status, will retry: error getting node "254.1.106.101": nodes "254.1.106.101" not found
Mar 15 12:23:24 k8s-node kubelet[1009]: E0315 12:23:24.580302    1009 kubelet.go:2697] Error updating node status, will retry: error getting node "254.1.106.101": nodes "254.1.106.101" not found
Mar 15 12:23:24 k8s-node kubelet[1009]: E0315 12:23:24.580314    1009 kubelet.go:1173] Unable to update node status: update node status exceeds retry count

Related: #8195, #15108.

Thanks in advance!

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 3
Comments: 19 (7 by maintainers)

Most upvoted comments

To necro this thread, nodename-override would be wonderful.

gtaylor on May 2, 2017

Initially this is how we handle hostname-override in kubelet:

kcfg.Hostname = nodeutil.GetHostname(kcfg.HostnameOverride)

    if len(kcfg.NodeName) == 0 {
        // Query the cloud provider for our node name, default to Hostname
        nodeName := kcfg.Hostname
        if kcfg.Cloud != nil {
            var err error
            instances, ok := kcfg.Cloud.Instances()
            if !ok {
                return fmt.Errorf("failed to get instances from cloud provider")
            }

            nodeName, err = instances.CurrentNodeName(kcfg.Hostname)
            if err != nil {
                return fmt.Errorf("error fetching current instance name from cloud provider: %v", err)
            }

            glog.V(2).Infof("cloud provider determined current node name to be %s", nodeName)
        }

        kcfg.NodeName = nodeName
    }

there are nodeName and hostName, they are different. For nodeName, hostname-override will be ignored if you are using cloud provider, we will use the node name determined by your cloud provider to register the node with apiserver. Perhaps we need some flag like nodename-override thus we can override node name even using cloud provider. @vishh @alexkappa @developerinlondon @mzupan wdyt?

adohe-zz on Jun 19, 2016

@AdoHe indeed when --cloud-provider is set, --hostname-override is ignored. Should that be documented maybe? I could have saved me some trouble.

My particular issue was that my nodes kubelet didn’t have --cloud-provider set so --hostname-override was in effect thus resulting in two different host naming conventions.

Thanks!

alexkappa on Mar 15, 2016