trivy-operator: Unauthorized error, scanjob pods can't connect to ACR

What steps did you take: Installed the helm with Terraform (helm release resource) and checked if it’s working. When the trivy-operator pods starts up I can see that the scan-vulnerabilityreport pods are trying to connect to the ACR to scan the images.

And what happened: The scan pods fail.

What did you expect to happen: The scan pods would scan the images.

Anything else you would like to add: The service account is configured according to the documentation. The scan pods do have an azure-identity-token and the azure.workload.identity/use=true label. Here is the error:

{ "level": "error", "ts": "2023-10-10T06:53:43Z", "logger": "reconciler.scan job", "msg": "Scan job container", "job": "monitoring/scan-vulnerabilityreport-69d869d787", "container": "<IMAGE_NAME>", "status.reason": "Error", "status.message": "2023-10-10T06:53:38.907Z\t\u001b[31mFATAL\u001b[0m\timage scan error: scan error: unable to initialize a scanner: unable to initialize a docker scanner: 4 errors occurred:\n\t* unable to inspect the image (<IMAGE_REGISTRY>.azurecr.io/<IMAGE_NAME>:<IMAGE_TAG>): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?\n\t* containerd socket not found: /run/containerd/containerd.sock\n\t* unable to initialize Podman client: no podman socket found: stat podman/podman.sock: no such file or directory\n\t* GET https://<IMAGE_REGISTRY>.azurecr.io/oauth2/token?scope=repository%3A<IMAGE_NAME>%3Apull&service=<IMAGE_REGISTRY>.azurecr.io: UNAUTHORIZED: authentication required, visit https://aka.ms/acr/authorization for more information.\n\n\n", "stacktrace": "github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).processFailedScanJob\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:299\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:81\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.2/pkg/reconcile/reconcile.go:111\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.2/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.2/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.2/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.2/pkg/internal/controller/controller.go:227" }

The <IMAGE_REGISTRY>, <IMAGE_NAME> AND <IMAGE_TAG> had the correct values.

Trivy-Operator version (use 0.16.1):

About this issue

Original URL
State: open
Created 9 months ago
Comments: 18

Most upvoted comments

I create the secret like this: kubectl create secret docker-registry my-docker-creds --docker-server=<REGISTRY_NAME>.azurecr.io --docker-username=<REGISTRY_USERNAME> --docker-password=<REGISTRY_PASSWORD> --docker-email=<ANY_EMAIL> My environment is Azure. I am running the last Trivy Operator Version

I assume that for your example above. the create secret should be named trivy-auth and deployed into namespace monitoring can you confirm ?

Yes, i did, but the problem persists 😦

thanks , I’ll try to reproduce it and update you.

chen-keinan on Dec 17, 2023