dapr: sidecar-injector failed to get authentication uid from service account on Typhoon K8s distro

In what area(s)?

/area runtime

What version of Dapr?

0.11.0

Expected Behavior

dapr init -k should start all required components

Actual Behavior

erverything starts except dapr-sidecar-injector

Steps to Reproduce the Problem

dapr init -k

wait a bit

dapr status -k

returns

NAME                   NAMESPACE    HEALTHY  STATUS                      REPLICAS  VERSION  AGE
dapr-sidecar-injector  dapr-system  False    Waiting (CrashLoopBackOff)  1         0.11.3   9m 
dapr-operator          dapr-system  True     Running                     1         0.11.3   9m 
dapr-sentry            dapr-system  True     Running                     1         0.11.3   9m 
dapr-dashboard         dapr-system  True     Running                     1         0.3.0    9m 
dapr-placement         dapr-system  True     Running                     1         0.11.3   9m

get the logs of the pod

kubectl logs -p dapr-sidecar-injector-55c5fbbb9-v2dmp -n dapr-system

brings failed to get authentication uid from service account: serviceaccounts "replicaset-controller" not found

time="2020-10-29T12:43:55.22267316Z" level=info msg="log level set to: info" instance=dapr-sidecar-injector-55c5fbbb9-v2dmp scope=dapr.injector type=log ver=0.11.3
time="2020-10-29T12:43:55.223719992Z" level=info msg="metrics server started on :9090/" instance=dapr-sidecar-injector-55c5fbbb9-v2dmp scope=dapr.metrics type=log ver=0.11.3
time="2020-10-29T12:43:55.224864169Z" level=info msg="starting Dapr Sidecar Injector -- version 0.11.3 -- commit a1a8e11" instance=dapr-sidecar-injector-55c5fbbb9-v2dmp scope=dapr.injector type=log ver=0.11.3
time="2020-10-29T12:43:55.230166822Z" level=info msg="Healthz server is listening on :8080" instance=dapr-sidecar-injector-55c5fbbb9-v2dmp scope=dapr.injector type=log ver=0.11.3
time="2020-10-29T12:43:55.255863265Z" level=fatal msg="failed to get authentication uid from service account: serviceaccounts \"replicaset-controller\" not found" instance=dapr-sidecar-injector-55c5fbbb9-v2dmp scope=dapr.injector type=log ver=0.11.3

I am using typhoon minimal and free Kubernetes distribution.

dapr mtls -k

reports

Mutual TLS is enabled in your Kubernetes cluster

Release Note

RELEASE NOTE:

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 16 (2 by maintainers)

Most upvoted comments

Update: Apparently the below makes the sidecar injector start, but I have the issue now that when I start a deployment it returns level=error msg="<nil>" instance=dapr-sidecar-injector-55c5fbbb9-8sqmq

@Liriel figured it out and this solved it for me!

So the error that pops up is something like this:

time="2020-11-07T11:47:35.255144983Z" level=fatal msg="failed to get authentication uid from service account: serviceaccounts \"replicaset-controller\" not found" instance=dapr-sidecar-injector-55c5fbbb9-p2gn4 scope=dapr.injector type=log ver=0.11.3

Which is obviously due to the missing service account replicaset-controller which we can check through kubectl get serviceaccounts -A which doesn’t list this service account.

Now when going to https://kubernetes.io/docs/reference/access-authn-authz/rbac/#controller-roles it shows that the kubernetes controller manager runs the controllers but only creates the service account system:controller:replicaset-controller when it’s started with the flag --use-service-account-credentials. Without this flag it will run the control loops using its own credential.

The resolution I used is to run this:

# Create the service account manually
kubectl create serviceaccount replicaset-controller -n kube-system

# Assign the clusterrole binding to it 
# clusterrole = system:controller:replicaset-controller
# check with kubectl get clusterroles
kubectl create clusterrolebinding dapr-replicaset-controller -n default --clusterrole=system:controller:replicaset-controller --serviceaccount=kube-system:replicaset-controller

Now the big question that pops up is why this is happening… it seems that in Azure you have to enable RBAC for this but it’s also happening for me on DigitalOcean where apparently RBAC is enabled by default… but it might be that they are missing this flag.

The question for @msfussell and team is if Dapr should “hijack” the uid here or create their own service account seeing the nature of Dapr which could benefit for this in terms of security?

Looking for a more “permanent” resolution here and looking forward to other comments 😊

Note: I am not a Kubernetes specialist, so I might be completely wrong or missing a command to actually enable this service account in an easier way, so I am definitely open for suggestions!

XavierGeerinck on Nov 7, 2020