kubernetes: Cluster dns stops functioning after 1.6 -> 1.5 downgrade
I have been able to reproduce this twice.
Steps to reproduce:
- Download kubernetes 1.5.5 tar, extract and cd into the directory
- run cluster/get-kube-binaries.sh
- run cluster/kube-up.sh
- Run the steps defined here here
- You don’t need to wait for the HPA to react, just verify that the script says “OK!” a bunch of times to ensure it can find the servier
- Download the
v1.6.0-rc.1release tar, extract, and cd into the directory - run cluster/get-kube-binaries.sh
- Warning: Set the following env or you will not be able to downgrade later
export TARGET_STORAGE=etcd3
export ETCD_IMAGE=3.0.17
export TARGET_VERSION=3.0.17
export STORAGE_MEDIA_TYPE=application/json
- run
./cluster/gce/upgrade.sh -M v1.6.0-rc.1 - Run the steps defined here here to verify the Pod can see the service
- run
./cluster/gce/upgrade.sh -N -o v1.6.0-rc.1to upgrade the nodes - Run the steps defined here here to verify the Pod can see the service
- run
./cluster/gce/upgrade.sh -N -o v1.5.5to downgrade the nodes - Run the steps defined here here to verify the Pod can see the service
- Warning: Set the following env to downgrade
export TARGET_STORAGE=etcd2
export ETCD_IMAGE=3.0.17
export TARGET_VERSION=2.2.1
export STORAGE_MEDIA_TYPE=application/json
- run
./cluster/gce/upgrade.sh -M v1.5.5to downgrade the master - Run the steps defined here here to verify the Pod can see the service
- This is where things fail for me. I am not longer able to connect to the service through the DNS address.
wget php-apache.default.svc.cluster.local -q -O -
wget: bad address 'php-apache.default.svc.cluster.local'
- I am able to connect to the service using its IP address
kubectl get services
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes 10.0.0.1 <none> 443/TCP 1h
php-apache 10.0.52.190 <none> 80/TCP 1h
wget -q -O - 10.0.52.190
OK!
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 27 (27 by maintainers)
fixed in #43676
successfully upgraded v1.5.5 -> ci/latest-1.6 -> ci/latest-1.5 and ensured kube-proxy and kube-dns were functioning correctly after upgrade and downgrade, and tokens file contained correct control plane users after each step