kubernetes: Unable to perform initial IP allocation check: unable to refresh the service IP block
What happened:
Hello, I have a k8s 1.14.3, I found a phenomenon: my apiserver often restart failed when I restart apiserver manually. The apiserver’s restart fail log as follows:
[root@master1 ~]# kubectl logs kube-apiserver-master2 -nkube-system --previous
I0826 11:13:42.389894 1 feature_gate.go:226] feature gates: &{map[SupportPodPidsLimit:true TTLAfterFinished:true VolumeSubpathEnvExpansion:true]}
Flag --insecure-bind-address has been deprecated, This flag will be removed in a future version.
Flag --insecure-port has been deprecated, This flag will be removed in a future version.
I0826 11:13:42.410210 1 flags.go:33] FLAG: --address="127.0.0.1"
I0826 11:13:42.410327 1 flags.go:33] FLAG: --admission-control="[]"
I0826 11:13:42.410404 1 flags.go:33] FLAG: --admission-control-config-file=""
I0826 11:13:42.410447 1 flags.go:33] FLAG: --advertise-address="172.16.3.2"
I0826 11:13:42.410474 1 flags.go:33] FLAG: --allow-privileged="true"
I0826 11:13:42.410516 1 flags.go:33] FLAG: --alsologtostderr="false"
I0826 11:13:42.410547 1 flags.go:33] FLAG: --anonymous-auth="true"
I0826 11:13:42.410582 1 flags.go:33] FLAG: --api-audiences="[]"
I0826 11:13:42.410609 1 flags.go:33] FLAG: --apiserver-count="3"
I0826 11:13:42.410655 1 flags.go:33] FLAG: --application-metrics-count-limit="100"
I0826 11:13:42.410681 1 flags.go:33] FLAG: --audit-dynamic-configuration="false"
I0826 11:13:42.410703 1 flags.go:33] FLAG: --audit-log-batch-buffer-size="10000"
I0826 11:13:42.410741 1 flags.go:33] FLAG: --audit-log-batch-max-size="1"
I0826 11:13:42.410763 1 flags.go:33] FLAG: --audit-log-batch-max-wait="0s"
I0826 11:13:42.410806 1 flags.go:33] FLAG: --audit-log-batch-throttle-burst="0"
I0826 11:13:42.410829 1 flags.go:33] FLAG: --audit-log-batch-throttle-enable="false"
I0826 11:13:42.410850 1 flags.go:33] FLAG: --audit-log-batch-throttle-qps="0"
I0826 11:13:42.410898 1 flags.go:33] FLAG: --audit-log-format="json"
I0826 11:13:42.410920 1 flags.go:33] FLAG: --audit-log-maxage="0"
I0826 11:13:42.410954 1 flags.go:33] FLAG: --audit-log-maxbackup="0"
I0826 11:13:42.410976 1 flags.go:33] FLAG: --audit-log-maxsize="0"
I0826 11:13:42.410996 1 flags.go:33] FLAG: --audit-log-mode="blocking"
I0826 11:13:42.411032 1 flags.go:33] FLAG: --audit-log-path=""
I0826 11:13:42.411052 1 flags.go:33] FLAG: --audit-log-truncate-enabled="false"
I0826 11:13:42.411074 1 flags.go:33] FLAG: --audit-log-truncate-max-batch-size="10485760"
I0826 11:13:42.411119 1 flags.go:33] FLAG: --audit-log-truncate-max-event-size="102400"
I0826 11:13:42.411142 1 flags.go:33] FLAG: --audit-log-version="audit.k8s.io/v1"
I0826 11:13:42.411165 1 flags.go:33] FLAG: --audit-policy-file=""
I0826 11:13:42.411199 1 flags.go:33] FLAG: --audit-webhook-batch-buffer-size="10000"
I0826 11:13:42.411222 1 flags.go:33] FLAG: --audit-webhook-batch-initial-backoff="10s"
I0826 11:13:42.411258 1 flags.go:33] FLAG: --audit-webhook-batch-max-size="400"
I0826 11:13:42.411281 1 flags.go:33] FLAG: --audit-webhook-batch-max-wait="30s"
I0826 11:13:42.411306 1 flags.go:33] FLAG: --audit-webhook-batch-throttle-burst="15"
I0826 11:13:42.411341 1 flags.go:33] FLAG: --audit-webhook-batch-throttle-enable="true"
I0826 11:13:42.411363 1 flags.go:33] FLAG: --audit-webhook-batch-throttle-qps="10"
I0826 11:13:42.411387 1 flags.go:33] FLAG: --audit-webhook-config-file=""
I0826 11:13:42.411425 1 flags.go:33] FLAG: --audit-webhook-initial-backoff="10s"
I0826 11:13:42.411448 1 flags.go:33] FLAG: --audit-webhook-mode="batch"
I0826 11:13:42.411482 1 flags.go:33] FLAG: --audit-webhook-truncate-enabled="false"
I0826 11:13:42.411504 1 flags.go:33] FLAG: --audit-webhook-truncate-max-batch-size="10485760"
I0826 11:13:42.411527 1 flags.go:33] FLAG: --audit-webhook-truncate-max-event-size="102400"
I0826 11:13:42.411562 1 flags.go:33] FLAG: --audit-webhook-version="audit.k8s.io/v1"
I0826 11:13:42.411586 1 flags.go:33] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
I0826 11:13:42.411608 1 flags.go:33] FLAG: --authentication-token-webhook-config-file=""
I0826 11:13:42.411645 1 flags.go:33] FLAG: --authorization-mode="[Node,RBAC]"
I0826 11:13:42.411687 1 flags.go:33] FLAG: --authorization-policy-file=""
I0826 11:13:42.411724 1 flags.go:33] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
I0826 11:13:42.411748 1 flags.go:33] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
I0826 11:13:42.411783 1 flags.go:33] FLAG: --authorization-webhook-config-file=""
I0826 11:13:42.411804 1 flags.go:33] FLAG: --azure-container-registry-config=""
I0826 11:13:42.411825 1 flags.go:33] FLAG: --basic-auth-file=""
I0826 11:13:42.411859 1 flags.go:33] FLAG: --bind-address="0.0.0.0"
I0826 11:13:42.411882 1 flags.go:33] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
I0826 11:13:42.411910 1 flags.go:33] FLAG: --cert-dir="/var/run/kubernetes"
I0826 11:13:42.411951 1 flags.go:33] FLAG: --client-ca-file="/etc/kubernetes/ssl/ca.pem"
I0826 11:13:42.411978 1 flags.go:33] FLAG: --cloud-config=""
I0826 11:13:42.412012 1 flags.go:33] FLAG: --cloud-provider=""
I0826 11:13:42.412032 1 flags.go:33] FLAG: --cloud-provider-gce-lb-src-cidrs="130.211.0.0/22,209.85.152.0/22,209.85.204.0/22,35.191.0.0/16"
I0826 11:13:42.412091 1 flags.go:33] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
I0826 11:13:42.412128 1 flags.go:33] FLAG: --containerd="unix:///var/run/containerd.sock"
I0826 11:13:42.412170 1 flags.go:33] FLAG: --contention-profiling="false"
I0826 11:13:42.412192 1 flags.go:33] FLAG: --cors-allowed-origins="[]"
I0826 11:13:42.412216 1 flags.go:33] FLAG: --default-not-ready-toleration-seconds="300"
I0826 11:13:42.412253 1 flags.go:33] FLAG: --default-unreachable-toleration-seconds="300"
I0826 11:13:42.412275 1 flags.go:33] FLAG: --default-watch-cache-size="100"
I0826 11:13:42.412311 1 flags.go:33] FLAG: --delete-collection-workers="1"
I0826 11:13:42.412332 1 flags.go:33] FLAG: --deserialization-cache-size="0"
I0826 11:13:42.412353 1 flags.go:33] FLAG: --disable-admission-plugins="[DefaultStorageClass]"
I0826 11:13:42.412405 1 flags.go:33] FLAG: --docker="unix:///var/run/docker.sock"
I0826 11:13:42.412432 1 flags.go:33] FLAG: --docker-env-metadata-whitelist=""
I0826 11:13:42.412467 1 flags.go:33] FLAG: --docker-only="false"
I0826 11:13:42.412489 1 flags.go:33] FLAG: --docker-root="/var/lib/docker"
I0826 11:13:42.412512 1 flags.go:33] FLAG: --docker-tls="false"
I0826 11:13:42.412548 1 flags.go:33] FLAG: --docker-tls-ca="ca.pem"
I0826 11:13:42.412570 1 flags.go:33] FLAG: --docker-tls-cert="cert.pem"
I0826 11:13:42.412591 1 flags.go:33] FLAG: --docker-tls-key="key.pem"
I0826 11:13:42.412627 1 flags.go:33] FLAG: --enable-admission-plugins="[]"
I0826 11:13:42.412651 1 flags.go:33] FLAG: --enable-aggregator-routing="false"
I0826 11:13:42.412672 1 flags.go:33] FLAG: --enable-bootstrap-token-auth="false"
I0826 11:13:42.412707 1 flags.go:33] FLAG: --enable-garbage-collector="true"
I0826 11:13:42.412728 1 flags.go:33] FLAG: --enable-load-reader="false"
I0826 11:13:42.412762 1 flags.go:33] FLAG: --enable-logs-handler="true"
I0826 11:13:42.412784 1 flags.go:33] FLAG: --enable-swagger-ui="false"
I0826 11:13:42.412805 1 flags.go:33] FLAG: --encryption-provider-config=""
I0826 11:13:42.412838 1 flags.go:33] FLAG: --endpoint-reconciler-type="lease"
I0826 11:13:42.412860 1 flags.go:33] FLAG: --etcd-cafile="/etc/ssl/etcd/ssl/ca.pem"
I0826 11:13:42.412885 1 flags.go:33] FLAG: --etcd-certfile="/etc/ssl/etcd/ssl/node-master2.pem"
I0826 11:13:42.412926 1 flags.go:33] FLAG: --etcd-compaction-interval="5m0s"
I0826 11:13:42.412948 1 flags.go:33] FLAG: --etcd-count-metric-poll-period="1m0s"
I0826 11:13:42.412969 1 flags.go:33] FLAG: --etcd-keyfile="/etc/ssl/etcd/ssl/node-master2-key.pem"
I0826 11:13:42.413011 1 flags.go:33] FLAG: --etcd-prefix="/registry"
I0826 11:13:42.413033 1 flags.go:33] FLAG: --etcd-servers="[https://172.16.3.1:2379,https://172.16.3.2:2379,https://172.16.3.5:2379]"
I0826 11:13:42.413102 1 flags.go:33] FLAG: --etcd-servers-overrides="[]"
I0826 11:13:42.413140 1 flags.go:33] FLAG: --event-storage-age-limit="default=0"
I0826 11:13:42.413163 1 flags.go:33] FLAG: --event-storage-event-limit="default=0"
I0826 11:13:42.413185 1 flags.go:33] FLAG: --event-ttl="1h0m0s"
I0826 11:13:42.413221 1 flags.go:33] FLAG: --experimental-encryption-provider-config=""
I0826 11:13:42.413242 1 flags.go:33] FLAG: --external-hostname=""
I0826 11:13:42.413261 1 flags.go:33] FLAG: --feature-gates="SupportPodPidsLimit=true,TTLAfterFinished=true,VolumeSubpathEnvExpansion=true"
I0826 11:13:42.413372 1 flags.go:33] FLAG: --global-housekeeping-interval="1m0s"
I0826 11:13:42.413399 1 flags.go:33] FLAG: --help="false"
I0826 11:13:42.413448 1 flags.go:33] FLAG: --housekeeping-interval="10s"
I0826 11:13:42.413469 1 flags.go:33] FLAG: --http2-max-streams-per-connection="0"
I0826 11:13:42.413490 1 flags.go:33] FLAG: --insecure-bind-address="127.0.0.1"
I0826 11:13:42.413527 1 flags.go:33] FLAG: --insecure-port="8080"
I0826 11:13:42.413558 1 flags.go:33] FLAG: --kubelet-certificate-authority=""
I0826 11:13:42.417630 1 flags.go:33] FLAG: --kubelet-client-certificate="/etc/kubernetes/ssl/node-master2.pem"
I0826 11:13:42.417687 1 flags.go:33] FLAG: --kubelet-client-key="/etc/kubernetes/ssl/node-master2-key.pem"
I0826 11:13:42.417716 1 flags.go:33] FLAG: --kubelet-https="true"
I0826 11:13:42.417753 1 flags.go:33] FLAG: --kubelet-port="10250"
I0826 11:13:42.417789 1 flags.go:33] FLAG: --kubelet-preferred-address-types="[InternalDNS,InternalIP,Hostname,ExternalDNS,ExternalIP]"
I0826 11:13:42.417864 1 flags.go:33] FLAG: --kubelet-read-only-port="10255"
I0826 11:13:42.417902 1 flags.go:33] FLAG: --kubelet-timeout="5s"
I0826 11:13:42.417927 1 flags.go:33] FLAG: --kubernetes-service-node-port="0"
I0826 11:13:42.417949 1 flags.go:33] FLAG: --log-backtrace-at=":0"
I0826 11:13:42.417993 1 flags.go:33] FLAG: --log-cadvisor-usage="false"
I0826 11:13:42.418014 1 flags.go:33] FLAG: --log-dir=""
I0826 11:13:42.418048 1 flags.go:33] FLAG: --log-file=""
I0826 11:13:42.418068 1 flags.go:33] FLAG: --log-flush-frequency="5s"
I0826 11:13:42.418090 1 flags.go:33] FLAG: --logtostderr="true"
I0826 11:13:42.418125 1 flags.go:33] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
I0826 11:13:42.418153 1 flags.go:33] FLAG: --master-service-namespace="default"
I0826 11:13:42.418176 1 flags.go:33] FLAG: --max-connection-bytes-per-sec="0"
I0826 11:13:42.418212 1 flags.go:33] FLAG: --max-mutating-requests-inflight="200"
I0826 11:13:42.418234 1 flags.go:33] FLAG: --max-requests-inflight="400"
I0826 11:13:42.418270 1 flags.go:33] FLAG: --mesos-agent="127.0.0.1:5051"
I0826 11:13:42.418294 1 flags.go:33] FLAG: --mesos-agent-timeout="10s"
I0826 11:13:42.418315 1 flags.go:33] FLAG: --min-request-timeout="1800"
I0826 11:13:42.418350 1 flags.go:33] FLAG: --oidc-ca-file="/etc/kubernetes/tls.crt"
I0826 11:13:42.418376 1 flags.go:33] FLAG: --oidc-client-id="client-admin"
I0826 11:13:42.418398 1 flags.go:33] FLAG: --oidc-groups-claim=""
I0826 11:13:42.418431 1 flags.go:33] FLAG: --oidc-groups-prefix=""
I0826 11:13:42.418451 1 flags.go:33] FLAG: --oidc-issuer-url="https://auth-lx.inspurspring.com/auth/realms/picp"
I0826 11:13:42.418500 1 flags.go:33] FLAG: --oidc-required-claim=""
I0826 11:13:42.418534 1 flags.go:33] FLAG: --oidc-signing-algs="[RS256]"
I0826 11:13:42.418574 1 flags.go:33] FLAG: --oidc-username-claim="preferred_username"
I0826 11:13:42.418600 1 flags.go:33] FLAG: --oidc-username-prefix="trident-"
I0826 11:13:42.418622 1 flags.go:33] FLAG: --port="8080"
I0826 11:13:42.418658 1 flags.go:33] FLAG: --profiling="false"
I0826 11:13:42.418680 1 flags.go:33] FLAG: --proxy-client-cert-file="/etc/kubernetes/ssl/front-proxy-client.pem"
I0826 11:13:42.418708 1 flags.go:33] FLAG: --proxy-client-key-file="/etc/kubernetes/ssl/front-proxy-client-key.pem"
I0826 11:13:42.418766 1 flags.go:33] FLAG: --request-timeout="1m0s"
I0826 11:13:42.418788 1 flags.go:33] FLAG: --requestheader-allowed-names="[front-proxy-client]"
I0826 11:13:42.418840 1 flags.go:33] FLAG: --requestheader-client-ca-file="/etc/kubernetes/ssl/front-proxy-ca.pem"
I0826 11:13:42.418869 1 flags.go:33] FLAG: --requestheader-extra-headers-prefix="[X-Remote-Extra-]"
I0826 11:13:42.418913 1 flags.go:33] FLAG: --requestheader-group-headers="[X-Remote-Group]"
I0826 11:13:42.418963 1 flags.go:33] FLAG: --requestheader-username-headers="[X-Remote-User]"
I0826 11:13:42.418992 1 flags.go:33] FLAG: --runtime-config="admissionregistration.k8s.io/v1beta1="
I0826 11:13:42.419071 1 flags.go:33] FLAG: --secure-port="6443"
I0826 11:13:42.419099 1 flags.go:33] FLAG: --service-account-api-audiences="[]"
I0826 11:13:42.419145 1 flags.go:33] FLAG: --service-account-issuer=""
I0826 11:13:42.419172 1 flags.go:33] FLAG: --service-account-key-file="[/etc/kubernetes/ssl/service-account-key.pem]"
I0826 11:13:42.419230 1 flags.go:33] FLAG: --service-account-lookup="true"
I0826 11:13:42.419267 1 flags.go:33] FLAG: --service-account-max-token-expiration="0s"
I0826 11:13:42.419289 1 flags.go:33] FLAG: --service-account-signing-key-file=""
I0826 11:13:42.419309 1 flags.go:33] FLAG: --service-cluster-ip-range="10.150.0.0/16"
I0826 11:13:42.419359 1 flags.go:33] FLAG: --service-node-port-range="30000-32767"
I0826 11:13:42.419421 1 flags.go:33] FLAG: --skip-headers="false"
I0826 11:13:42.419444 1 flags.go:33] FLAG: --ssh-keyfile=""
I0826 11:13:42.419463 1 flags.go:33] FLAG: --ssh-user=""
I0826 11:13:42.419495 1 flags.go:33] FLAG: --stderrthreshold="2"
I0826 11:13:42.419517 1 flags.go:33] FLAG: --storage-backend="etcd3"
I0826 11:13:42.419538 1 flags.go:33] FLAG: --storage-driver-buffer-duration="1m0s"
I0826 11:13:42.419573 1 flags.go:33] FLAG: --storage-driver-db="cadvisor"
I0826 11:13:42.419595 1 flags.go:33] FLAG: --storage-driver-host="localhost:8086"
I0826 11:13:42.419618 1 flags.go:33] FLAG: --storage-driver-password="root"
I0826 11:13:42.419653 1 flags.go:33] FLAG: --storage-driver-secure="false"
I0826 11:13:42.419674 1 flags.go:33] FLAG: --storage-driver-table="stats"
I0826 11:13:42.419695 1 flags.go:33] FLAG: --storage-driver-user="root"
I0826 11:13:42.419729 1 flags.go:33] FLAG: --storage-media-type="application/vnd.kubernetes.protobuf"
I0826 11:13:42.419757 1 flags.go:33] FLAG: --target-ram-mb="0"
I0826 11:13:42.419777 1 flags.go:33] FLAG: --tls-cert-file="/etc/kubernetes/ssl/apiserver.pem"
I0826 11:13:42.419818 1 flags.go:33] FLAG: --tls-cipher-suites="[]"
I0826 11:13:42.419850 1 flags.go:33] FLAG: --tls-min-version=""
I0826 11:13:42.419890 1 flags.go:33] FLAG: --tls-private-key-file="/etc/kubernetes/ssl/apiserver-key.pem"
I0826 11:13:42.419918 1 flags.go:33] FLAG: --tls-sni-cert-key="[]"
I0826 11:13:42.419960 1 flags.go:33] FLAG: --token-auth-file=""
I0826 11:13:42.419980 1 flags.go:33] FLAG: --update-machine-info-interval="5m0s"
I0826 11:13:42.420002 1 flags.go:33] FLAG: --v="2"
I0826 11:13:42.420036 1 flags.go:33] FLAG: --version="false"
I0826 11:13:42.420070 1 flags.go:33] FLAG: --vmodule=""
I0826 11:13:42.420112 1 flags.go:33] FLAG: --watch-cache="true"
I0826 11:13:42.420134 1 flags.go:33] FLAG: --watch-cache-sizes="[]"
I0826 11:13:42.420225 1 server.go:559] external host was not specified, using 172.16.3.2
I0826 11:13:42.420278 1 server.go:602] Initializing cache sizes based on 0MB limit
I0826 11:13:42.420727 1 server.go:146] Version: v1.14.3
I0826 11:13:55.746424 1 plugins.go:158] Loaded 7 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,TaintNodesByCondition,Priority,DefaultTolerationSeconds,MutatingAdmissionWebhook.
I0826 11:13:55.746591 1 plugins.go:161] Loaded 6 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,ValidatingAdmissionWebhook,ResourceQuota.
E0826 11:13:55.753785 1 prometheus.go:138] failed to register depth metric admission_quota_controller: duplicate metrics collector registration attempted
E0826 11:13:55.753988 1 prometheus.go:150] failed to register adds metric admission_quota_controller: duplicate metrics collector registration attempted
E0826 11:13:55.754152 1 prometheus.go:162] failed to register latency metric admission_quota_controller: duplicate metrics collector registration attempted
E0826 11:13:55.754285 1 prometheus.go:174] failed to register work_duration metric admission_quota_controller: duplicate metrics collector registration attempted
E0826 11:13:55.754578 1 prometheus.go:189] failed to register unfinished_work_seconds metric admission_quota_controller: duplicate metrics collector registration attempted
E0826 11:13:55.754699 1 prometheus.go:202] failed to register longest_running_processor_microseconds metric admission_quota_controller: duplicate metrics collector registration attempted
I0826 11:13:55.754817 1 plugins.go:158] Loaded 7 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,TaintNodesByCondition,Priority,DefaultTolerationSeconds,MutatingAdmissionWebhook.
I0826 11:13:55.754855 1 plugins.go:161] Loaded 6 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,ValidatingAdmissionWebhook,ResourceQuota.
I0826 11:13:55.764515 1 client.go:352] parsed scheme: ""
I0826 11:13:55.764631 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:13:55.765413 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:13:55.765940 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:13:56.184528 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:13:56.184901 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:13:56.185156 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:13:56.189009 1 client.go:352] parsed scheme: ""
I0826 11:13:56.189086 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:13:56.189242 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:13:56.189521 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:13:56.681473 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:13:56.682313 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
W0826 11:13:56.682496 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:13:56.684029 1 store.go:1319] Monitoring customresourcedefinitions.apiextensions.k8s.io count at <storage-prefix>//apiextensions.k8s.io/customresourcedefinitions
I0826 11:13:56.738181 1 client.go:352] parsed scheme: ""
I0826 11:13:56.738251 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:13:56.738426 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:13:56.738833 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:13:57.177287 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:13:57.177829 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:13:57.178172 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:13:57.208106 1 master.go:233] Using reconciler: lease
I0826 11:13:57.226036 1 client.go:352] parsed scheme: ""
I0826 11:13:57.226101 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:13:57.226309 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:13:57.226593 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:13:57.633706 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:13:57.634043 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:13:57.634756 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:13:57.669737 1 client.go:352] parsed scheme: ""
I0826 11:13:57.669871 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:13:57.670129 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:13:57.670483 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:13:58.054731 1 store.go:1319] Monitoring podtemplates count at <storage-prefix>//podtemplates
I0826 11:13:58.055594 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.5:2379 <nil>}]
W0826 11:13:58.056630 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:13:58.057039 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:13:58.058644 1 client.go:352] parsed scheme: ""
I0826 11:13:58.058749 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:13:58.058986 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:13:58.059269 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:13:58.373169 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
I0826 11:13:58.373200 1 store.go:1319] Monitoring events count at <storage-prefix>//events
W0826 11:13:58.373501 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:13:58.373844 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:13:58.376697 1 client.go:352] parsed scheme: ""
I0826 11:13:58.376800 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:13:58.377038 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:13:58.377282 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:13:58.708088 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
I0826 11:13:58.708264 1 store.go:1319] Monitoring limitranges count at <storage-prefix>//limitranges
W0826 11:13:58.708637 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:13:58.708777 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:13:58.711357 1 client.go:352] parsed scheme: ""
I0826 11:13:58.711407 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:13:58.711561 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:13:58.711812 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:13:59.020801 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.5:2379 <nil>}]
I0826 11:13:59.021201 1 store.go:1319] Monitoring resourcequotas count at <storage-prefix>//resourcequotas
W0826 11:13:59.021598 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:13:59.021692 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:13:59.025234 1 client.go:352] parsed scheme: ""
I0826 11:13:59.025850 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:13:59.026174 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:13:59.026995 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:13:59.402051 1 store.go:1319] Monitoring secrets count at <storage-prefix>//secrets
I0826 11:13:59.403311 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.5:2379 <nil>}]
W0826 11:13:59.404136 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
W0826 11:13:59.404158 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:13:59.406956 1 client.go:352] parsed scheme: ""
I0826 11:13:59.407026 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:13:59.407234 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:13:59.407499 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:13:59.811729 1 store.go:1319] Monitoring persistentvolumes count at <storage-prefix>//persistentvolumes
I0826 11:13:59.814119 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:13:59.814545 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:13:59.814785 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:13:59.816415 1 client.go:352] parsed scheme: ""
I0826 11:13:59.816467 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:13:59.816642 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:13:59.817163 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:00.115333 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:00.115650 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
W0826 11:14:00.115762 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:00.116273 1 store.go:1319] Monitoring persistentvolumeclaims count at <storage-prefix>//persistentvolumeclaims
I0826 11:14:00.120217 1 client.go:352] parsed scheme: ""
I0826 11:14:00.120456 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:00.120859 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:00.121137 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:00.505324 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:00.507395 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:00.505470 1 store.go:1319] Monitoring configmaps count at <storage-prefix>//configmaps
W0826 11:14:00.508172 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:00.558826 1 client.go:352] parsed scheme: ""
I0826 11:14:00.559143 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:00.559519 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:00.559888 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:00.844968 1 store.go:1319] Monitoring namespaces count at <storage-prefix>//namespaces
I0826 11:14:00.846315 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:00.846824 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
W0826 11:14:00.847067 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:00.850296 1 client.go:352] parsed scheme: ""
I0826 11:14:00.850402 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:00.850699 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:00.851105 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:01.423391 1 store.go:1319] Monitoring endpoints count at <storage-prefix>//services/endpoints
I0826 11:14:01.425714 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:01.426292 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:01.427570 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:01.435768 1 client.go:352] parsed scheme: ""
I0826 11:14:01.435876 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:01.436126 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:01.436377 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:01.821876 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
I0826 11:14:01.822022 1 store.go:1319] Monitoring nodes count at <storage-prefix>//minions
W0826 11:14:01.822520 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:01.822741 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:01.827528 1 client.go:352] parsed scheme: ""
I0826 11:14:01.831855 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:01.832051 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:01.832279 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:02.270121 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:02.270981 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:14:02.271298 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:02.277215 1 store.go:1319] Monitoring pods count at <storage-prefix>//pods
I0826 11:14:02.280165 1 client.go:352] parsed scheme: ""
I0826 11:14:02.280219 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:02.280461 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:02.285785 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:02.677150 1 store.go:1319] Monitoring serviceaccounts count at <storage-prefix>//serviceaccounts
I0826 11:14:02.679493 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:02.680430 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:02.680672 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:02.700977 1 client.go:352] parsed scheme: ""
I0826 11:14:02.701084 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:02.701333 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:02.701527 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:03.014937 1 store.go:1319] Monitoring services count at <storage-prefix>//services/specs
I0826 11:14:03.015555 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.5:2379 <nil>}]
W0826 11:14:03.016265 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:14:03.016462 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:03.027096 1 client.go:352] parsed scheme: ""
I0826 11:14:03.027205 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:03.032812 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:03.034189 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:03.312851 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:03.313557 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:03.314104 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:03.319446 1 client.go:352] parsed scheme: ""
I0826 11:14:03.319500 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:03.319926 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:03.320201 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:03.650720 1 client.go:352] parsed scheme: ""
I0826 11:14:03.650779 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:03.652022 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:03.652742 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
I0826 11:14:03.652940 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
W0826 11:14:03.653538 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:03.654285 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:03.918897 1 store.go:1319] Monitoring replicationcontrollers count at <storage-prefix>//controllers
I0826 11:14:03.921790 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:03.922707 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
W0826 11:14:03.924187 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:04.492804 1 master.go:417] Skipping disabled API group "auditregistration.k8s.io".
I0826 11:14:04.493039 1 master.go:425] Enabling API group "authentication.k8s.io".
I0826 11:14:04.493120 1 master.go:425] Enabling API group "authorization.k8s.io".
I0826 11:14:04.497888 1 client.go:352] parsed scheme: ""
I0826 11:14:04.497999 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:04.498258 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:04.498517 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:04.798736 1 store.go:1319] Monitoring horizontalpodautoscalers.autoscaling count at <storage-prefix>//horizontalpodautoscalers
I0826 11:14:04.801035 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:04.802185 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:14:04.802780 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:04.807359 1 client.go:352] parsed scheme: ""
I0826 11:14:04.807467 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:04.807893 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:04.808374 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:05.122329 1 store.go:1319] Monitoring horizontalpodautoscalers.autoscaling count at <storage-prefix>//horizontalpodautoscalers
I0826 11:14:05.136563 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:05.137386 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:05.140699 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:05.145222 1 client.go:352] parsed scheme: ""
I0826 11:14:05.145275 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:05.145433 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:05.146053 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:05.431502 1 store.go:1319] Monitoring horizontalpodautoscalers.autoscaling count at <storage-prefix>//horizontalpodautoscalers
I0826 11:14:05.431837 1 master.go:425] Enabling API group "autoscaling".
I0826 11:14:05.433611 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:05.434241 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:14:05.434465 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:05.440936 1 client.go:352] parsed scheme: ""
I0826 11:14:05.441042 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:05.441285 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:05.441753 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:05.790300 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:05.791111 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:05.790508 1 store.go:1319] Monitoring jobs.batch count at <storage-prefix>//jobs
W0826 11:14:05.791869 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:05.795752 1 client.go:352] parsed scheme: ""
I0826 11:14:05.796943 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:05.797374 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:05.797936 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:06.118495 1 store.go:1319] Monitoring cronjobs.batch count at <storage-prefix>//cronjobs
I0826 11:14:06.118602 1 master.go:425] Enabling API group "batch".
I0826 11:14:06.120972 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:06.121365 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:06.121835 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:06.142747 1 client.go:352] parsed scheme: ""
I0826 11:14:06.142806 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:06.143234 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:06.143634 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:06.497247 1 store.go:1319] Monitoring certificatesigningrequests.certificates.k8s.io count at <storage-prefix>//certificatesigningrequests
I0826 11:14:06.497351 1 master.go:425] Enabling API group "certificates.k8s.io".
I0826 11:14:06.501678 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:06.502294 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:06.503457 1 client.go:352] parsed scheme: ""
I0826 11:14:06.503518 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:06.503701 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:06.504592 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
W0826 11:14:06.504839 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:06.823840 1 store.go:1319] Monitoring leases.coordination.k8s.io count at <storage-prefix>//leases
I0826 11:14:06.824397 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:06.825176 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:06.825630 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:06.826741 1 client.go:352] parsed scheme: ""
I0826 11:14:06.826799 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:06.826968 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:06.827118 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:07.209490 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
I0826 11:14:07.210687 1 store.go:1319] Monitoring leases.coordination.k8s.io count at <storage-prefix>//leases
I0826 11:14:07.210804 1 master.go:425] Enabling API group "coordination.k8s.io".
W0826 11:14:07.212048 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:07.210675 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:07.213941 1 client.go:352] parsed scheme: ""
I0826 11:14:07.213994 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:07.214165 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:07.214382 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:07.530069 1 store.go:1319] Monitoring replicationcontrollers count at <storage-prefix>//controllers
I0826 11:14:07.530327 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:07.530972 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:07.531183 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:07.534469 1 client.go:352] parsed scheme: ""
I0826 11:14:07.534575 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:07.535509 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:07.535975 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:07.852595 1 store.go:1319] Monitoring daemonsets.apps count at <storage-prefix>//daemonsets
I0826 11:14:07.855645 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:07.856340 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:07.856728 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:07.859584 1 client.go:352] parsed scheme: ""
I0826 11:14:07.860224 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:07.860585 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:07.860928 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:08.149987 1 store.go:1319] Monitoring deployments.apps count at <storage-prefix>//deployments
I0826 11:14:08.152177 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.5:2379 <nil>}]
W0826 11:14:08.152790 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
W0826 11:14:08.152915 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:08.157187 1 client.go:352] parsed scheme: ""
I0826 11:14:08.158650 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:08.159247 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:08.159898 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:08.521504 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.5:2379 <nil>}]
W0826 11:14:08.522657 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:08.523001 1 store.go:1319] Monitoring ingresses.extensions count at <storage-prefix>//ingress
W0826 11:14:08.523877 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:08.525892 1 client.go:352] parsed scheme: ""
I0826 11:14:08.525950 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:08.526101 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:08.527460 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:08.878844 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
I0826 11:14:08.880104 1 store.go:1319] Monitoring podsecuritypolicies.policy count at <storage-prefix>//podsecuritypolicy
W0826 11:14:08.881535 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:08.882020 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:08.884607 1 client.go:352] parsed scheme: ""
I0826 11:14:08.885160 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:08.885596 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:08.886002 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:09.202787 1 store.go:1319] Monitoring replicasets.apps count at <storage-prefix>//replicasets
I0826 11:14:09.203503 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:09.205485 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:09.205593 1 client.go:352] parsed scheme: ""
I0826 11:14:09.206573 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:09.206824 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
W0826 11:14:09.206091 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:09.207037 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:09.508984 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:09.512171 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:09.509214 1 store.go:1319] Monitoring networkpolicies.networking.k8s.io count at <storage-prefix>//networkpolicies
I0826 11:14:09.512403 1 master.go:425] Enabling API group "extensions".
W0826 11:14:09.512724 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:09.517541 1 client.go:352] parsed scheme: ""
I0826 11:14:09.517649 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:09.517890 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:09.518150 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:09.814438 1 store.go:1319] Monitoring networkpolicies.networking.k8s.io count at <storage-prefix>//networkpolicies
I0826 11:14:09.815209 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
I0826 11:14:09.817405 1 client.go:352] parsed scheme: ""
I0826 11:14:09.817461 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:09.817811 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:09.817969 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
W0826 11:14:09.819121 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:14:09.817420 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:10.079501 1 store.go:1319] Monitoring ingresses.extensions count at <storage-prefix>//ingress
I0826 11:14:10.079616 1 master.go:425] Enabling API group "networking.k8s.io".
I0826 11:14:10.085818 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:10.087660 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:10.087964 1 client.go:352] parsed scheme: ""
W0826 11:14:10.088367 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:10.088445 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:10.089133 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:10.089657 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:10.411171 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
I0826 11:14:10.411285 1 store.go:1319] Monitoring runtimeclasses.node.k8s.io count at <storage-prefix>//runtimeclasses
I0826 11:14:10.412240 1 master.go:425] Enabling API group "node.k8s.io".
W0826 11:14:10.412291 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:14:10.412946 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:10.419905 1 client.go:352] parsed scheme: ""
I0826 11:14:10.419959 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:10.420420 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:10.420751 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:10.713517 1 store.go:1319] Monitoring poddisruptionbudgets.policy count at <storage-prefix>//poddisruptionbudgets
I0826 11:14:10.717159 1 client.go:352] parsed scheme: ""
I0826 11:14:10.717264 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:10.717523 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:10.718422 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:10.718918 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:10.720437 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
W0826 11:14:10.721752 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:11.008259 1 store.go:1319] Monitoring podsecuritypolicies.policy count at <storage-prefix>//podsecuritypolicy
I0826 11:14:11.008834 1 master.go:425] Enabling API group "policy".
I0826 11:14:11.008485 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:11.009686 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
W0826 11:14:11.011268 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:11.013556 1 client.go:352] parsed scheme: ""
I0826 11:14:11.013973 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:11.014455 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:11.015070 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:11.340516 1 store.go:1319] Monitoring roles.rbac.authorization.k8s.io count at <storage-prefix>//roles
I0826 11:14:11.343739 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:11.344098 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:14:11.344257 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:11.346313 1 client.go:352] parsed scheme: ""
I0826 11:14:11.346414 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:11.346680 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:11.347068 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:11.627332 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
I0826 11:14:11.627325 1 store.go:1319] Monitoring rolebindings.rbac.authorization.k8s.io count at <storage-prefix>//rolebindings
W0826 11:14:11.627661 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:11.627857 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:11.633685 1 client.go:352] parsed scheme: ""
I0826 11:14:11.633831 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:11.634192 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:11.634482 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:11.951847 1 store.go:1319] Monitoring clusterroles.rbac.authorization.k8s.io count at <storage-prefix>//clusterroles
I0826 11:14:11.953641 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:11.953982 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
W0826 11:14:11.954119 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:11.958434 1 client.go:352] parsed scheme: ""
I0826 11:14:11.958830 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:11.959189 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:11.966830 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:12.269608 1 store.go:1319] Monitoring clusterrolebindings.rbac.authorization.k8s.io count at <storage-prefix>//clusterrolebindings
I0826 11:14:12.273177 1 client.go:352] parsed scheme: ""
I0826 11:14:12.273233 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:12.273419 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:12.273677 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.5:2379 <nil>}]
W0826 11:14:12.274497 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:12.275565 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
W0826 11:14:12.276314 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:12.568346 1 store.go:1319] Monitoring roles.rbac.authorization.k8s.io count at <storage-prefix>//roles
I0826 11:14:12.572326 1 client.go:352] parsed scheme: ""
I0826 11:14:12.572424 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:12.572656 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:12.573016 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:12.574171 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:12.574392 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
W0826 11:14:12.575855 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:13.066363 1 store.go:1319] Monitoring rolebindings.rbac.authorization.k8s.io count at <storage-prefix>//rolebindings
I0826 11:14:13.066823 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:13.067192 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:14:13.067636 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:13.070954 1 client.go:352] parsed scheme: ""
I0826 11:14:13.071052 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:13.071287 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:13.071467 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:13.392220 1 store.go:1319] Monitoring clusterroles.rbac.authorization.k8s.io count at <storage-prefix>//clusterroles
I0826 11:14:13.392735 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:13.393702 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:13.393808 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:13.404254 1 client.go:352] parsed scheme: ""
I0826 11:14:13.404622 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:13.404985 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:13.405301 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:13.872139 1 store.go:1319] Monitoring clusterrolebindings.rbac.authorization.k8s.io count at <storage-prefix>//clusterrolebindings
I0826 11:14:13.872308 1 master.go:425] Enabling API group "rbac.authorization.k8s.io".
I0826 11:14:13.872682 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:13.873573 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:14:13.874427 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:13.886257 1 client.go:352] parsed scheme: ""
I0826 11:14:13.886330 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:13.886529 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:13.886785 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:14.196352 1 store.go:1319] Monitoring priorityclasses.scheduling.k8s.io count at <storage-prefix>//priorityclasses
I0826 11:14:14.206624 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:14.207875 1 clientconn.go:1251] grpc: addrConn.createTransport failed to connect to {172.16.3.1:2379 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: context canceled". Reconnecting...
W0826 11:14:14.208173 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: context canceled; please retry.
W0826 11:14:14.209066 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:14.230496 1 client.go:352] parsed scheme: ""
I0826 11:14:14.230567 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:14.230752 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:14.230997 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:14.564086 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:14.564883 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:14.565434 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:14.566232 1 store.go:1319] Monitoring priorityclasses.scheduling.k8s.io count at <storage-prefix>//priorityclasses
I0826 11:14:14.566349 1 master.go:425] Enabling API group "scheduling.k8s.io".
I0826 11:14:14.567382 1 master.go:417] Skipping disabled API group "settings.k8s.io".
I0826 11:14:14.571920 1 client.go:352] parsed scheme: ""
I0826 11:14:14.572019 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:14.572811 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:14.573278 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:15.027231 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.5:2379 <nil>}]
I0826 11:14:15.027510 1 store.go:1319] Monitoring storageclasses.storage.k8s.io count at <storage-prefix>//storageclasses
W0826 11:14:15.028204 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:14:15.028461 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:15.031986 1 client.go:352] parsed scheme: ""
I0826 11:14:15.032121 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:15.032421 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:15.032760 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:15.349613 1 store.go:1319] Monitoring volumeattachments.storage.k8s.io count at <storage-prefix>//volumeattachments
I0826 11:14:15.351521 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:15.351884 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:15.352468 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:15.358060 1 client.go:352] parsed scheme: ""
I0826 11:14:15.358197 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:15.358500 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:15.359165 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:15.778230 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
I0826 11:14:15.778467 1 store.go:1319] Monitoring csinodes.storage.k8s.io count at <storage-prefix>//csinodes
W0826 11:14:15.778944 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:15.779768 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:15.790976 1 client.go:352] parsed scheme: ""
I0826 11:14:15.791036 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:15.791196 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:15.791392 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:16.217775 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:16.218149 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:16.221940 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:16.215884 1 store.go:1319] Monitoring csidrivers.storage.k8s.io count at <storage-prefix>//csidrivers
I0826 11:14:16.233480 1 client.go:352] parsed scheme: ""
I0826 11:14:16.233587 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:16.234119 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:16.234531 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:16.637273 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.5:2379 <nil>}]
W0826 11:14:16.638156 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:14:16.638187 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:16.646987 1 store.go:1319] Monitoring storageclasses.storage.k8s.io count at <storage-prefix>//storageclasses
I0826 11:14:16.650024 1 client.go:352] parsed scheme: ""
I0826 11:14:16.650093 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:16.650268 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:16.650546 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:16.940615 1 store.go:1319] Monitoring volumeattachments.storage.k8s.io count at <storage-prefix>//volumeattachments
I0826 11:14:16.940716 1 master.go:425] Enabling API group "storage.k8s.io".
I0826 11:14:16.941427 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:16.942106 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:16.942395 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:16.947197 1 client.go:352] parsed scheme: ""
I0826 11:14:16.947253 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:16.947410 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:16.947625 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:17.321689 1 store.go:1319] Monitoring deployments.apps count at <storage-prefix>//deployments
I0826 11:14:17.325411 1 client.go:352] parsed scheme: ""
I0826 11:14:17.325472 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:17.325630 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:17.326154 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.5:2379 <nil>}]
W0826 11:14:17.326527 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:17.327312 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
W0826 11:14:17.328147 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:17.631663 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:17.632283 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:17.636694 1 store.go:1319] Monitoring statefulsets.apps count at <storage-prefix>//statefulsets
I0826 11:14:17.640086 1 client.go:352] parsed scheme: ""
I0826 11:14:17.640284 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:17.640525 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:17.640811 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
W0826 11:14:17.641986 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:17.939189 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:17.940107 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:17.940583 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:17.944617 1 store.go:1319] Monitoring controllerrevisions.apps count at <storage-prefix>//controllerrevisions
I0826 11:14:17.948729 1 client.go:352] parsed scheme: ""
I0826 11:14:17.949260 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:17.949527 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:17.949851 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:18.242146 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:18.242635 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:18.243258 1 store.go:1319] Monitoring deployments.apps count at <storage-prefix>//deployments
W0826 11:14:18.244028 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:18.246865 1 client.go:352] parsed scheme: ""
I0826 11:14:18.247061 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:18.247436 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:18.247909 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:18.560736 1 store.go:1319] Monitoring statefulsets.apps count at <storage-prefix>//statefulsets
I0826 11:14:18.566170 1 client.go:352] parsed scheme: ""
I0826 11:14:18.566279 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:18.566561 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:18.567236 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:18.567672 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
W0826 11:14:18.571758 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:18.571910 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:18.926674 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
W0826 11:14:18.927384 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:18.927486 1 store.go:1319] Monitoring daemonsets.apps count at <storage-prefix>//daemonsets
W0826 11:14:18.930967 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:18.937741 1 client.go:352] parsed scheme: ""
I0826 11:14:18.937850 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:18.938096 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:18.938528 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:19.241212 1 store.go:1319] Monitoring replicasets.apps count at <storage-prefix>//replicasets
I0826 11:14:19.241389 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:19.242197 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:19.242289 1 controlbuf.go:382] transport: loopyWriter.run returning. connection error: desc = "transport is closing"
I0826 11:14:19.245964 1 client.go:352] parsed scheme: ""
I0826 11:14:19.246094 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:19.246351 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:19.246546 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:19.732413 1 store.go:1319] Monitoring controllerrevisions.apps count at <storage-prefix>//controllerrevisions
I0826 11:14:19.733303 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:19.735398 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:14:19.738563 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:19.740393 1 client.go:352] parsed scheme: ""
I0826 11:14:19.740502 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:19.740744 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:19.741118 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:20.012681 1 store.go:1319] Monitoring deployments.apps count at <storage-prefix>//deployments
I0826 11:14:20.019353 1 client.go:352] parsed scheme: ""
I0826 11:14:20.019469 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:20.019706 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:20.013239 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.5:2379 <nil>}]
W0826 11:14:20.020240 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:20.020864 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
W0826 11:14:20.022632 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:20.290347 1 store.go:1319] Monitoring statefulsets.apps count at <storage-prefix>//statefulsets
I0826 11:14:20.292262 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:20.292607 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:20.293261 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:20.296937 1 client.go:352] parsed scheme: ""
I0826 11:14:20.297020 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:20.297270 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:20.297484 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:20.617296 1 store.go:1319] Monitoring daemonsets.apps count at <storage-prefix>//daemonsets
I0826 11:14:20.618059 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:20.618708 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:20.619548 1 controlbuf.go:382] transport: loopyWriter.run returning. connection error: desc = "transport is closing"
I0826 11:14:20.620377 1 client.go:352] parsed scheme: ""
I0826 11:14:20.620433 1 client.go:352] scheme "" not registered, fallback to default scheme
W0826 11:14:20.620628 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:20.620650 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:20.623025 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:20.918949 1 store.go:1319] Monitoring replicasets.apps count at <storage-prefix>//replicasets
I0826 11:14:20.919260 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:20.919721 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:20.920003 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:20.922202 1 client.go:352] parsed scheme: ""
I0826 11:14:20.922260 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:20.922417 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:20.922702 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:21.324214 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:21.325070 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:21.326706 1 store.go:1319] Monitoring controllerrevisions.apps count at <storage-prefix>//controllerrevisions
I0826 11:14:21.328962 1 master.go:425] Enabling API group "apps".
W0826 11:14:21.328645 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:21.334752 1 client.go:352] parsed scheme: ""
I0826 11:14:21.335273 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:21.335939 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:21.336246 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:21.626512 1 store.go:1319] Monitoring validatingwebhookconfigurations.admissionregistration.k8s.io count at <storage-prefix>//validatingwebhookconfigurations
I0826 11:14:21.628652 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:21.628991 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:21.629334 1 client.go:352] parsed scheme: ""
W0826 11:14:21.629475 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:21.629862 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:21.630122 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:21.630597 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:21.883600 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:21.884224 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:21.884408 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
I0826 11:14:21.885123 1 store.go:1319] Monitoring mutatingwebhookconfigurations.admissionregistration.k8s.io count at <storage-prefix>//mutatingwebhookconfigurations
I0826 11:14:21.885699 1 master.go:425] Enabling API group "admissionregistration.k8s.io".
I0826 11:14:21.889577 1 client.go:352] parsed scheme: ""
I0826 11:14:21.891114 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:21.891685 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:21.892484 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:22.211520 1 store.go:1319] Monitoring events count at <storage-prefix>//events
I0826 11:14:22.211591 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.2:2379 <nil>}]
I0826 11:14:22.211616 1 master.go:425] Enabling API group "events.k8s.io".
W0826 11:14:22.212197 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
W0826 11:14:22.213651 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
W0826 11:14:22.926219 1 genericapiserver.go:344] Skipping API batch/v2alpha1 because it has no resources.
W0826 11:14:22.972755 1 genericapiserver.go:344] Skipping API node.k8s.io/v1alpha1 because it has no resources.
W0826 11:14:23.099468 1 genericapiserver.go:344] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
W0826 11:14:23.181756 1 genericapiserver.go:344] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources.
W0826 11:14:23.274603 1 genericapiserver.go:344] Skipping API storage.k8s.io/v1alpha1 because it has no resources.
E0826 11:14:29.005398 1 prometheus.go:138] failed to register depth metric admission_quota_controller: duplicate metrics collector registration attempted
E0826 11:14:29.005603 1 prometheus.go:150] failed to register adds metric admission_quota_controller: duplicate metrics collector registration attempted
E0826 11:14:29.006058 1 prometheus.go:162] failed to register latency metric admission_quota_controller: duplicate metrics collector registration attempted
E0826 11:14:29.006673 1 prometheus.go:174] failed to register work_duration metric admission_quota_controller: duplicate metrics collector registration attempted
E0826 11:14:29.007112 1 prometheus.go:189] failed to register unfinished_work_seconds metric admission_quota_controller: duplicate metrics collector registration attempted
E0826 11:14:29.007272 1 prometheus.go:202] failed to register longest_running_processor_microseconds metric admission_quota_controller: duplicate metrics collector registration attempted
I0826 11:14:29.007373 1 plugins.go:158] Loaded 7 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,TaintNodesByCondition,Priority,DefaultTolerationSeconds,MutatingAdmissionWebhook.
I0826 11:14:29.007410 1 plugins.go:161] Loaded 6 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,ValidatingAdmissionWebhook,ResourceQuota.
I0826 11:14:29.018070 1 client.go:352] parsed scheme: ""
I0826 11:14:29.018124 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:29.018702 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:29.019170 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:29.305631 1 store.go:1319] Monitoring apiservices.apiregistration.k8s.io count at <storage-prefix>//apiregistration.k8s.io/apiservices
I0826 11:14:29.309153 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>}]
W0826 11:14:29.310083 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:14:29.310634 1 asm_mips64x.s:646] Failed to dial 172.16.3.5:2379: grpc: the connection is closing; please retry.
I0826 11:14:29.311787 1 client.go:352] parsed scheme: ""
I0826 11:14:29.311845 1 client.go:352] scheme "" not registered, fallback to default scheme
I0826 11:14:29.312066 1 asm_mips64x.s:646] ccResolverWrapper: sending new addresses to cc: [{172.16.3.1:2379 0 <nil>}]
I0826 11:14:29.312364 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.1:2379 <nil>} {172.16.3.2:2379 <nil>} {172.16.3.5:2379 <nil>}]
I0826 11:14:29.594301 1 store.go:1319] Monitoring apiservices.apiregistration.k8s.io count at <storage-prefix>//apiregistration.k8s.io/apiservices
I0826 11:14:29.603806 1 asm_mips64x.s:646] balancerWrapper: got update addr from Notify: [{172.16.3.5:2379 <nil>}]
W0826 11:14:29.609189 1 asm_mips64x.s:646] Failed to dial 172.16.3.2:2379: grpc: the connection is closing; please retry.
W0826 11:14:29.613153 1 asm_mips64x.s:646] Failed to dial 172.16.3.1:2379: grpc: the connection is closing; please retry.
I0826 11:14:44.540188 1 deprecated_insecure_serving.go:51] Serving insecurely on 127.0.0.1:8080
I0826 11:14:44.557908 1 secure_serving.go:116] Serving securely on [::]:6443
I0826 11:14:44.563952 1 controller.go:81] Starting OpenAPI AggregationController
I0826 11:14:44.564169 1 crd_finalizer.go:242] Starting CRDFinalizer
I0826 11:14:44.566588 1 apiservice_controller.go:94] Starting APIServiceRegistrationController
I0826 11:14:44.566689 1 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller
I0826 11:14:44.567014 1 autoregister_controller.go:139] Starting autoregister controller
I0826 11:14:44.567043 1 cache.go:32] Waiting for caches to sync for autoregister controller
I0826 11:14:44.567092 1 available_controller.go:320] Starting AvailableConditionController
I0826 11:14:44.567134 1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
I0826 11:14:44.567204 1 customresource_discovery_controller.go:208] Starting DiscoveryController
I0826 11:14:44.567305 1 naming_controller.go:284] Starting NamingConditionController
I0826 11:14:44.567355 1 establishing_controller.go:73] Starting EstablishingController
I0826 11:14:44.568096 1 crdregistration_controller.go:112] Starting crd-autoregister controller
I0826 11:14:44.568153 1 controller_utils.go:1027] Waiting for caches to sync for crd-autoregister controller
E0826 11:14:44.584207 1 controller.go:148] Unable to remove old endpoints from kubernetes service: StorageError: key not found, Code: 1, Key: /registry/masterleases/172.16.3.2, ResourceVersion: 0, AdditionalErrorMsg:
I0826 11:14:45.687254 1 controller.go:107] OpenAPI AggregationController: Processing item
I0826 11:14:45.687425 1 controller.go:130] OpenAPI AggregationController: action for item : Nothing (removed from the queue).
I0826 11:14:45.687511 1 controller.go:130] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).
I0826 11:14:46.021460 1 log.go:172] http: TLS handshake error from 172.16.3.2:47840: EOF
I0826 11:14:46.853842 1 log.go:172] http: TLS handshake error from 127.0.0.1:43958: EOF
I0826 11:14:46.979530 1 log.go:172] http: TLS handshake error from 127.0.0.1:43956: EOF
I0826 11:14:47.471204 1 log.go:172] http: TLS handshake error from 127.0.0.1:43934: EOF
I0826 11:14:47.493265 1 log.go:172] http: TLS handshake error from 127.0.0.1:43914: EOF
I0826 11:14:47.691309 1 log.go:172] http: TLS handshake error from 127.0.0.1:43936: EOF
I0826 11:14:47.943575 1 log.go:172] http: TLS handshake error from 127.0.0.1:43938: EOF
I0826 11:14:47.953197 1 log.go:172] http: TLS handshake error from 127.0.0.1:43972: EOF
I0826 11:14:47.977733 1 log.go:172] http: TLS handshake error from 127.0.0.1:43964: EOF
I0826 11:14:48.000356 1 log.go:172] http: TLS handshake error from 172.16.3.2:47820: EOF
I0826 11:14:48.073043 1 log.go:172] http: TLS handshake error from 172.16.3.2:47992: EOF
I0826 11:14:48.185422 1 log.go:172] http: TLS handshake error from 172.16.3.3:50632: EOF
I0826 11:14:48.380016 1 log.go:172] http: TLS handshake error from 172.16.3.2:47832: EOF
I0826 11:14:48.406883 1 log.go:172] http: TLS handshake error from 127.0.0.1:44546: EOF
I0826 11:14:48.511056 1 log.go:172] http: TLS handshake error from 127.0.0.1:43952: EOF
I0826 11:14:48.551112 1 log.go:172] http: TLS handshake error from 127.0.0.1:44096: EOF
I0826 11:14:49.027412 1 log.go:172] http: TLS handshake error from 172.16.3.2:47838: EOF
I0826 11:14:49.027585 1 log.go:172] http: TLS handshake error from 127.0.0.1:43950: EOF
I0826 11:14:49.027688 1 log.go:172] http: TLS handshake error from 127.0.0.1:44072: EOF
I0826 11:14:49.027774 1 log.go:172] http: TLS handshake error from 127.0.0.1:44388: EOF
I0826 11:14:49.084820 1 log.go:172] http: TLS handshake error from 127.0.0.1:43940: EOF
I0826 11:14:49.340310 1 log.go:172] http: TLS handshake error from 127.0.0.1:44116: EOF
I0826 11:14:49.469725 1 log.go:172] http: TLS handshake error from 172.16.3.7:41022: EOF
I0826 11:14:50.124867 1 log.go:172] http: TLS handshake error from 127.0.0.1:43948: EOF
I0826 11:14:50.204850 1 log.go:172] http: TLS handshake error from 127.0.0.1:43982: EOF
I0826 11:14:50.345061 1 log.go:172] http: TLS handshake error from 127.0.0.1:44390: EOF
I0826 11:14:50.499936 1 log.go:172] http: TLS handshake error from 127.0.0.1:44080: EOF
I0826 11:14:50.532098 1 log.go:172] http: TLS handshake error from 127.0.0.1:44094: EOF
I0826 11:14:50.544917 1 log.go:172] http: TLS handshake error from 172.16.3.2:47874: EOF
I0826 11:14:50.559251 1 log.go:172] http: TLS handshake error from 127.0.0.1:43920: EOF
I0826 11:14:50.701075 1 log.go:172] http: TLS handshake error from 127.0.0.1:44086: EOF
I0826 11:14:50.803934 1 log.go:172] http: TLS handshake error from 127.0.0.1:44028: EOF
I0826 11:14:50.810338 1 log.go:172] http: TLS handshake error from 127.0.0.1:44124: EOF
I0826 11:14:51.298877 1 log.go:172] http: TLS handshake error from 127.0.0.1:44120: EOF
I0826 11:14:51.355132 1 log.go:172] http: TLS handshake error from 10.110.57.2:44944: EOF
I0826 11:14:51.776071 1 log.go:172] http: TLS handshake error from 127.0.0.1:44112: EOF
I0826 11:14:51.814959 1 log.go:172] http: TLS handshake error from 127.0.0.1:43944: EOF
I0826 11:14:51.877141 1 log.go:172] http: TLS handshake error from 172.16.3.2:48262: EOF
I0826 11:14:51.922989 1 cacher.go:648] cacher (*coordination.Lease): 1 objects queued in incoming channel.
I0826 11:14:52.037320 1 log.go:172] http: TLS handshake error from 127.0.0.1:43942: EOF
I0826 11:14:52.117411 1 log.go:172] http: TLS handshake error from 127.0.0.1:44090: EOF
I0826 11:14:52.143699 1 log.go:172] http: TLS handshake error from 127.0.0.1:44076: EOF
I0826 11:14:52.225909 1 log.go:172] http: TLS handshake error from 127.0.0.1:44640: EOF
I0826 11:14:52.433341 1 log.go:172] http: TLS handshake error from 172.16.3.2:48280: EOF
I0826 11:14:52.473215 1 cacher.go:648] cacher (*core.Pod): 1 objects queued in incoming channel.
I0826 11:14:52.473356 1 cacher.go:648] cacher (*core.Pod): 2 objects queued in incoming channel.
I0826 11:14:52.473579 1 cacher.go:648] cacher (*core.Pod): 3 objects queued in incoming channel.
I0826 11:14:52.717397 1 log.go:172] http: TLS handshake error from 127.0.0.1:44644: EOF
I0826 11:14:52.872935 1 log.go:172] http: TLS handshake error from 172.16.3.2:48018: EOF
I0826 11:14:53.073194 1 log.go:172] http: TLS handshake error from 127.0.0.1:44626: EOF
I0826 11:14:53.098337 1 log.go:172] http: TLS handshake error from 127.0.0.1:43922: EOF
I0826 11:14:53.355998 1 log.go:172] http: TLS handshake error from 127.0.0.1:43946: EOF
I0826 11:14:53.408759 1 log.go:172] http: TLS handshake error from 127.0.0.1:43974: EOF
I0826 11:14:53.409787 1 log.go:172] http: TLS handshake error from 172.16.3.2:47834: EOF
I0826 11:14:53.477271 1 log.go:172] http: TLS handshake error from 172.16.3.2:48544: EOF
I0826 11:14:53.478996 1 log.go:172] http: TLS handshake error from 127.0.0.1:44092: EOF
I0826 11:14:53.755546 1 cacher.go:648] cacher (*core.Endpoints): 1 objects queued in incoming channel.
I0826 11:14:53.755741 1 cacher.go:648] cacher (*core.Endpoints): 2 objects queued in incoming channel.
I0826 11:14:53.800695 1 log.go:172] http: TLS handshake error from 127.0.0.1:44070: EOF
I0826 11:14:54.011007 1 log.go:172] http: TLS handshake error from 127.0.0.1:44068: EOF
I0826 11:14:54.017834 1 log.go:172] http: TLS handshake error from 127.0.0.1:44620: EOF
I0826 11:14:54.167177 1 log.go:172] http: TLS handshake error from 127.0.0.1:44646: EOF
I0826 11:14:54.432211 1 log.go:172] http: TLS handshake error from 127.0.0.1:44132: EOF
I0826 11:14:54.809289 1 log.go:172] http: TLS handshake error from 127.0.0.1:43960: EOF
I0826 11:14:54.809839 1 log.go:172] http: TLS handshake error from 127.0.0.1:44408: EOF
I0826 11:14:54.910388 1 log.go:172] http: TLS handshake error from 172.16.3.2:47986: EOF
I0826 11:14:55.226772 1 log.go:172] http: TLS handshake error from 127.0.0.1:44918: EOF
I0826 11:14:55.227974 1 trace.go:81] Trace[1408556175]: "Reflector k8s.io/client-go/informers/factory.go:133 ListAndWatch" (started: 2019-08-26 11:14:44.576878176 +0000 UTC m=+62.908028094) (total time: 10.65082621s):
Trace[1408556175]: [10.65082621s] [10.65082621s] END
E0826 11:14:55.228015 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.StorageClass: Get https://[::1]:6443/apis/storage.k8s.io/v1/storageclasses?limit=500&resourceVersion=0: net/http: TLS handshake timeout
I0826 11:14:55.228744 1 trace.go:81] Trace[493615857]: "Reflector k8s.io/client-go/informers/factory.go:133 ListAndWatch" (started: 2019-08-26 11:14:44.578990597 +0000 UTC m=+62.910140341) (total time: 10.649615321s):
Trace[493615857]: [10.649615321s] [10.649615321s] END
E0826 11:14:55.228780 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Node: Get https://[::1]:6443/api/v1/nodes?limit=500&resourceVersion=0: net/http: TLS handshake timeout
I0826 11:14:55.229436 1 trace.go:81] Trace[94780368]: "Reflector k8s.io/client-go/informers/factory.go:133 ListAndWatch" (started: 2019-08-26 11:14:44.570739382 +0000 UTC m=+62.901890362) (total time: 10.65854726s):
Trace[94780368]: [10.65854726s] [10.65854726s] END
E0826 11:14:55.229471 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.ClusterRole: Get https://[::1]:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?limit=500&resourceVersion=0: net/http: TLS handshake timeout
I0826 11:14:55.230058 1 trace.go:81] Trace[1612690419]: "Reflector k8s.io/client-go/informers/factory.go:133 ListAndWatch" (started: 2019-08-26 11:14:44.596022799 +0000 UTC m=+62.927173172) (total time: 10.633900933s):
Trace[1612690419]: [10.633900933s] [10.633900933s] END
E0826 11:14:55.230087 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Namespace: Get https://[::1]:6443/api/v1/namespaces?limit=500&resourceVersion=0: net/http: TLS handshake timeout
I0826 11:14:55.380944 1 trace.go:81] Trace[1410441496]: "Reflector k8s.io/client-go/informers/factory.go:133 ListAndWatch" (started: 2019-08-26 11:14:44.564351052 +0000 UTC m=+62.895501208) (total time: 10.816402177s):
Trace[1410441496]: [10.816402177s] [10.816402177s] END
E0826 11:14:55.381064 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Pod: Get https://[::1]:6443/api/v1/pods?limit=500&resourceVersion=0: net/http: TLS handshake timeout
I0826 11:14:55.382823 1 trace.go:81] Trace[616004176]: "Reflector k8s.io/client-go/informers/factory.go:133 ListAndWatch" (started: 2019-08-26 11:14:44.565303219 +0000 UTC m=+62.896453483) (total time: 10.817225773s):
Trace[616004176]: [10.817225773s] [10.817225773s] END
E0826 11:14:55.382916 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.VolumeAttachment: Get https://[::1]:6443/apis/storage.k8s.io/v1/volumeattachments?limit=500&resourceVersion=0: net/http: TLS handshake timeout
F0826 11:14:55.383445 1 controller.go:157] Unable to perform initial IP allocation check: unable to refresh the service IP block: Get https://[::1]:6443/api/v1/services: net/http: TLS handshake timeout
What you expected to happen:
The apiserver can restart success every time.
How to reproduce it (as minimally and precisely as possible):
exec kubectl delete pod <one of apiserver's name> -nkube-system
It will restart serveral times until success. It got Unable to perform initial IP allocation check: unable to refresh the service IP block when failed.
Anything else we need to know?:
My etcd is always running normal.
Environment:
- Kubernetes version (use
kubectl version):
Client Version: version.Info{Major:“1”, Minor:“14”, GitVersion:“v1.14.3”, GitCommit:“5e53fd6bc17c0dec8434817e69b04a25d8ae0ff0”, GitTreeState:“archive”, BuildDate:“1970-01-01T00:00:00Z”, GoVersion:“go1.12.5”, Compiler:“gc”, Platform:“linux/mips64le”} Server Version: version.Info{Major:“1”, Minor:“14”, GitVersion:“v1.14.3”, GitCommit:“$Format:%H$”, GitTreeState:“archive”, BuildDate:“1970-01-01T00:00:00Z”, GoVersion:“go1.12.5”, Compiler:“gc”, Platform:“linux/mips64le”}
- OS (e.g:
cat /etc/os-release): NAME=“NeoKylin Linux Server” VERSION=“7.0 (loongson)” ID=“neokylin” ID_LIKE=“fedora” VERSION_ID=“7.0” PRETTY_NAME=“NeoKylin Linux Server 7.0 (loongson)” ANSI_COLOR=“0;31” CPE_NAME=“cpe:/o:neokylin:enterprise_linux:7.0:GA:server” HOME_URL=“https://www.cs2c.com.cn/” BUG_REPORT_URL=“https://bugzilla.cs2c.com.cn/”
NEOKYLIN_BUGZILLA_PRODUCT=“NeoKylin Linux Server 7.0” NEOKYLIN_BUGZILLA_PRODUCT_VERSION=7.0 NEOKYLIN_SUPPORT_PRODUCT=“NeoKylin Linux Server” NEOKYLIN_SUPPORT_PRODUCT_VERSION=“7.0”
- Kernel (e.g.
uname -a): Linux localhost 3.10.0-862.9.1.ns7_4.37.mips64el #1 SMP PREEMPT Tue Feb 26 19:23:39 CST 2019 mips64el mips64el mips64el GNU/Linux
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 34 (25 by maintainers)
It works for me in kubernetes:
v1.15.5The issue in my case was k8s api-server was using ipv6 address ([::1]) which was not working. I disabled ipv6 and it fixed the issue for me
Disable ipv6
cat >> /etc/sysctl.conf <<EOF net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 EOF sysctl -p
Why this has been closed? I can’t see any solution here 😦
This! I can’t thank you enough!