linkerd2: Linkerd 2.11.x Control Plane Components Failing

What is the issue?

After upgrading linkerd helm charts from stable 2.10.2 to 2.11.2, all control-plane components for linkerd are failing. We’re using AKS with kubenet, with k8s version 1.21.7

image

How can it be reproduced?

Upgrade from helm chart 2.10.2 to 2.11.x

Logs, error output, etc

Logs from linkerd-proxy-injectors and linkerd-destination’s linkerd-proxy container:

[0.005486s]  INFO ThreadId(01) linkerd2_proxy: Identity verified via linkerd-identity-headless.linkerd.svc.cluster.local:8080 (linkerd-identity.linkerd.serviceaccount.identity.linkerd.cluster.local)
[0.005494s]  INFO ThreadId(01) linkerd2_proxy: Destinations resolved via linkerd-dst-headless.linkerd.svc.cluster.local:8086 (linkerd-destination.linkerd.serviceaccount.identity.linkerd.cluster.local)
[0.008909s]  WARN ThreadId(01) daemon:identity: linkerd_app_core::control: Failed to resolve control-plane component error=no record found for name: linkerd-identity-headless.linkerd.svc.cluster.local. type: SRV class: IN
[0.012103s]  WARN ThreadId(01) daemon:identity: linkerd_app_core::control: Failed to resolve control-plane component error=no record found for name: linkerd-identity-headless.linkerd.svc.cluster.local. type: SRV class: IN
[0.028235s]  INFO ThreadId(01) linkerd_proxy::signal: received SIGTERM, starting shutdown

output of linkerd check -o short

kubernetes-api
--------------
√ can initialize the client
√ can query the Kubernetes API

kubernetes-version
------------------
√ is running the minimum Kubernetes API version
√ is running the minimum kubectl version

linkerd-existence
-----------------
√ 'linkerd-config' config map exists
√ heartbeat ServiceAccount exist
√ control plane replica sets are ready
√ no unschedulable pods
| linkerd-destination-74cbd87444-lfckk status is CrashLoopBackOff

Environment

  • Kubernetes version - 1.21.7
  • AKS Cluster
  • Host OS - linux
  • Linkerd version - helm chart 2.11.2

Possible solution

No response

Additional context

No response

Would you like to work on fixing this bug?

No response

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 19 (7 by maintainers)

Most upvoted comments

@olix0r @adleong any updates here? 2.11.0 also started failing for us now 😐