aws-privateca-issuer: MissingAuthenticationToken: Request is missing Authentication Token

I wanted to upgrade to the latest v0.3.0 but when I tried I noticed I started seeing exceptions in the logs.

We are using AWS IAM roles as annotations on the serviceAccount to handle the authorization. This works in v0.2.1 and I’ve looked through the code but I see no differences between 0.2.1 and 0.3.0 that would cause this.

{"level":"error","ts":1626803861.6337998,"logger":"controllers.GenericIssuer","msg":"failed to sts.GetCallerIdentity","genericissuer":"/aws-issuer","error":"operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: dd011a20-5b5f-4191-9996-cd709af162f9, api error MissingAuthenticationToken: Request is missing Authentication Token","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/zapr@v0.2.0/zapr.go:132\ngithub.com/cert-manager/aws-privateca-issuer/pkg/controllers.(*GenericIssuerReconciler).Reconcile\n\t/workspace/pkg/controllers/genericissuer_controller.go:122\ngithub.com/cert-manager/aws-privateca-issuer/pkg/controllers.(*AWSPCAClusterIssuerReconciler).Reconcile\n\t/workspace/pkg/controllers/awspcaclusterissuer_controller.go:57\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:99"}
{"level":"error","ts":1626803861.6339357,"logger":"controller-runtime.manager.controller.awspcaclusterissuer","msg":"Reconciler error","reconciler group":"awspca.cert-manager.io","reconciler kind":"AWSPCAClusterIssuer","name":"aws-issuer","namespace":"","error":"operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: dd011a20-5b5f-4191-9996-cd709af162f9, api error MissingAuthenticationToken: Request is missing Authentication Token","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/zapr@v0.2.0/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:302\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:99"}

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 15 (3 by maintainers)

Most upvoted comments

I believe the issue that I’m having is https://github.com/cert-manager/aws-privateca-issuer/issues/50 . I will follow up there. Thanks!

@divyansh-gupta I have been working on a fix for it. To get it to work I had to redo how the aws config is loaded. I put together a branch and draft PR here but I am very new to Go so I am not sure if I have something wrong. Just wanted to get something for you to see and/or use to get a fix in place. I tested this and this code works for me when using the steps above. I was even able to get istio-csr and istiod working with it.

https://github.com/cert-manager/aws-privateca-issuer/pull/41