linkerd2: linkerd2 (2.11) control plane pod failure on k8s 1.21

What is the issue?

When installing linkerd2 (version 2.11) on k8s 1.21 (EKS running on AWS) the control plane services fail to come up.

How can it be reproduced?

I’m installing linkerd2 via helm here, passing in manually generated the cert/keys as flags to helm.

The same setup has worked for us when running linkerd2 version 2.9 on k8s 1.18 and 1.19.

Logs, error output, etc

; k logs pods/linkerd-destination-6b4bfb9f87-hpvg4 -n linkerd linkerd-proxy
time="2022-01-28T18:13:19Z" level=info msg="Found pre-existing key: /var/run/linkerd/identity/end-entity/key.p8"
time="2022-01-28T18:13:19Z" level=info msg="Found pre-existing CSR: /var/run/linkerd/identity/end-entity/csr.der"
[     0.001141s] ERROR ThreadId(01) linkerd_app::env: Could not read LINKERD2_PROXY_IDENTITY_TOKEN_FILE: Permission denied (os error 13)
[     0.001176s] ERROR ThreadId(01) linkerd_app::env: LINKERD2_PROXY_IDENTITY_TOKEN_FILE="/var/run/secrets/kubernetes.io/serviceaccount/token" is not valid: InvalidTokenSource
Invalid configuration: invalid environment variable

output of linkerd check -o short

Linkerd core checks
===================

linkerd-existence
-----------------
\ pod/linkerd-destination-6b4bfb9f87-hpvg4 container sp-validator is not ready

Environment

Kubernetes: 1.21 Host Env: EKS/AWS Linkerd version: 2.11 HostOs: Amazon Linux2

Possible solution

No response

Additional context

No response

Would you like to work on fixing this bug?

No response

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 16 (6 by maintainers)

Most upvoted comments

@olix0r looks like the upgrade fixes the issue. thanks again!

@olix0r: Sorry, I wasn’t able to test the edge release. Will leave it up to you to decide what do with this issue.

Indeed, applying the same fix as https://github.com/metallb/metallb/commit/d36e8dd4caa4f0c768c898fcf6eefd353ba55547 to linkerd2 pod configs resolves the issue for me.