cluster-api: kubeadm bootstrap provider does not take into account Cluster APIServerPort

What steps did you take and what happened:

Create a cluster with a non-default value for cluster.Spec.ClusterNetwork.APIServerPort (e.g. 6444 instead of 6443)
Create a controller Node using the kubeadm bootstrap provider
The generated cloud-init ignores the APIServerPort setting and uses the default 6443 (or whatever is specified in the controller KubeadmConfig).

What did you expect to happen:

For the specified apiServer port to be used.

Anything else you would like to add:

I am not sure this is a bug, just seems like strange behaviour - as an infrastructure provider I would need to parse the resulting cloud-init or otherwise to work out which port to forward connections to or health check, so I would expect the kubeadm bootstrap provider to override the default or whatever is configured in the KubeadmConfig with the value given by the cluster.

Environment:

Cluster-api version: 0.2.8
kubeadm bootstrap provider version: 0.1.5
Minikube/KIND version: None (using kubernetes infrastructure provider on GKE)
Kubernetes version: (use kubectl version): 1.16.3
OS (e.g. from /etc/os-release): Container Optimised Linux for GKE cluster (kind images for cluster-api Nodes)

/kind bug

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 26 (23 by maintainers)

Most upvoted comments

This should also take into account the spec.livenessProbe.port field as well. It defaults to 6443 and if you change APIServerPort you need to ensure spec.livenessProbe.port is updated as well or your container will just get endlessly rebooted.

In fact, I just hit this bug and have had to resort to the hackiest of hacks to fix this:

spec:
  kubeadmConfigSpec:
    postKubeadmCommands:
    - chmod +x /tmp/kube-apiserver-quickfix.sh && /tmp/kube-apiserver-quickfix.sh
    files:
    - path: /tmp/kube-apiserver-overlay.yaml
      content: |
        apiVersion: v1
        kind: Pod
        spec:
          livenessProbe:
            httpGet:
              port: 443
    - path: /tmp/kube-apiserver-quickfix.sh
      content: |
        # Stop kube-apiserver and edit YAML
        mv /etc/kubernetes/manifests/kube-apiserver.yaml /tmp/kube-apiserver.yaml
        # Patch the YAML, super hacky :).
        echo "Started patching kube-apiserver"
       cd /tmp && docker run --rm -v "${PWD}":/workdir mikefarah/yq yq m -i --overwrite --autocreate=false /tmp/kube-apiserver.yaml /tmp/kube-apiserver-overlay.yaml
        # Sleep to let container download and run
        sleep 30 && mv /tmp/kube-apiserver.yaml /etc/kubernetes/manifests/kube-apiserver.yaml
        echo "Finished patching kube-apiserver"

stgarf on May 5, 2020

We should probably take a closer look at this after the holidays. /cc @ncdc @vincepri

detiber on Dec 30, 2019