trivy-operator: timage scan error: scan error: unable to initialize a scanner: the length of usernames and passwords must match

Enviornment Details

chartVersion : v0.12.1 trivytag : v0.38.3 GKE Version: 1.25.6-gke.1000 mode: standalone

after updating to latest version we are getting errors on trivy scanjob

{"level":"error","ts":"2023-03-24T09:54:10Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-57d5bdbb9d","container":"webapp-promtheus-metrics","status.reason":"Error","status.message":"2023-03-24T09:54:07.646Z\t\u001b[31mFATAL\u001b[0m\timage scan error: scan error: unable to initialize a scanner: the length of usernames and passwords must match\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).processFailedScanJob\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:254\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:79\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/reconcile/reconcile.go:102\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:235"}


configs

data:
  configAuditReports.scanner: Trivy
  node.collector.imageRef: 'ghcr.io/aquasecurity/node-collector:0.0.5'
  report.recordFailedChecksOnly: 'true'
  scanJob.compressLogs: 'true'
  scanJob.podTemplateContainerSecurityContext: >-
    {"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"privileged":false,"readOnlyRootFilesystem":true}
  vulnerabilityReports.scanner: Trivy```

```apiVersion: v1
data:
  trivy.additionalVulnerabilityReportFields: ''
  trivy.command: image
  trivy.dbRepository: ghcr.io/aquasecurity/trivy-db
  trivy.dbRepositoryInsecure: 'false'
  trivy.httpProxy: 'http://proxy.internal.svc.cluster.local:80'
  trivy.httpsProxy: 'http://proxy.internal.svc.cluster.local:80'
  trivy.imagePullSecret: artifact-registry
  trivy.mode: Standalone
  trivy.noProxy: >-
    172.20.0.0/16, *.alertmanager-operated, *.monitoring, *.cert-manager.svc,
    127.0.0.1, localhost, 169.254.169.254, metadata, metadata.google.internal,
    *.googleapis.com, *.alpha.applis.renault.fr
  trivy.repository: >-
    europe-docker.pkg.dev/irn-71889-adm-ope-ope-d2/docker-shared-gke-irn70740/trivy
  trivy.resources.limits.cpu: 500m
  trivy.resources.limits.memory: 2Gi
  trivy.resources.requests.cpu: 100m
  trivy.resources.requests.memory: 900Mi
  trivy.severity: 'HIGH,CRITICAL'
  trivy.slow: 'true'
  trivy.supportedConfigAuditKinds: >-
    Workload,Service,Role,ClusterRole,NetworkPolicy,Ingress,LimitRange,ResourceQuota
  trivy.tag: 0.38.3
  trivy.timeout: 5m0s
  trivy.useBuiltinRegoPolicies: 'true'```

Envvars
--------

`env:
        - name: OPERATOR_NAMESPACE
          value: trivy-system
        - name: OPERATOR_TARGET_NAMESPACES
        - name: OPERATOR_EXCLUDE_NAMESPACES
          value: 'kube-system, trivy-system'
        - name: OPERATOR_TARGET_WORKLOADS
          value: >-
            pod,replicaset,replicationcontroller,statefulset,daemonset,cronjob,job
        - name: OPERATOR_SERVICE_ACCOUNT
          value: trivy-custom
        - name: OPERATOR_LOG_DEV_MODE
          value: 'false'
        - name: OPERATOR_SCAN_JOB_TTL
          value: 5m
        - name: OPERATOR_SCAN_JOB_TIMEOUT
          value: 5m
        - name: OPERATOR_CONCURRENT_SCAN_JOBS_LIMIT
          value: '10'
        - name: OPERATOR_CONCURRENT_NODE_COLLECTOR_LIMIT
          value: '1'
        - name: OPERATOR_SCAN_JOB_RETRY_AFTER
          value: 30s
        - name: OPERATOR_BATCH_DELETE_LIMIT
          value: '10'
        - name: OPERATOR_BATCH_DELETE_DELAY
          value: 10s
        - name: OPERATOR_METRICS_BIND_ADDRESS
          value: ':8080'
        - name: OPERATOR_METRICS_FINDINGS_ENABLED
          value: 'true'
        - name: OPERATOR_METRICS_VULN_ID_ENABLED
          value: 'false'
        - name: OPERATOR_HEALTH_PROBE_BIND_ADDRESS
          value: ':9090'
        - name: OPERATOR_VULNERABILITY_SCANNER_ENABLED
          value: 'true'
        - name: OPERATOR_VULNERABILITY_SCANNER_SCAN_ONLY_CURRENT_REVISIONS
          value: 'true'
        - name: OPERATOR_SCANNER_REPORT_TTL
          value: 24h
        - name: OPERATOR_CONFIG_AUDIT_SCANNER_ENABLED
          value: 'true'
        - name: OPERATOR_RBAC_ASSESSMENT_SCANNER_ENABLED
          value: 'false'
        - name: OPERATOR_INFRA_ASSESSMENT_SCANNER_ENABLED
          value: 'false'
        - name: OPERATOR_CONFIG_AUDIT_SCANNER_SCAN_ONLY_CURRENT_REVISIONS
          value: 'true'
        - name: OPERATOR_EXPOSED_SECRET_SCANNER_ENABLED
          value: 'true'
        - name: OPERATOR_METRICS_EXPOSED_SECRET_INFO_ENABLED
          value: 'false'
        - name: OPERATOR_WEBHOOK_BROADCAST_URL
        - name: OPERATOR_WEBHOOK_BROADCAST_TIMEOUT
          value: 30s
        - name: OPERATOR_PRIVATE_REGISTRY_SCAN_SECRETS_NAMES
          value: '{}'
        - name: OPERATOR_ACCESS_GLOBAL_SECRETS_SERVICE_ACCOUNTS
          value: 'true'
        - name: OPERATOR_BUILT_IN_TRIVY_SERVER
          value: 'false'
        - name: TRIVY_SERVER_HEALTH_CHECK_CACHE_EXPIRATION
          value: 10h
        - name: OPERATOR_MERGE_RBAC_FINDING_WITH_CONFIG_AUDIT
          value: 'false'`

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Reactions: 1
  • Comments: 35 (14 by maintainers)

Most upvoted comments

@quixoten – Found a fix. Instead of specifying CS_REGISTRY_PASSWORD and CS_REGISTRY_USER separately, Trivy supports setting GOOGLE_APPLICATION_CREDENTIALS pointed to your service account key file.

This seems to be working well for me.

This issue (i think) started effecting our gitlab security scans:

.scan-image-template:
  extends: container_scanning
  variables:
    CS_REGISTRY_PASSWORD: $GCLOUD_AUTH_JSON
    CS_REGISTRY_USER: _json_key

As best as I can tell, the above started failing for us around 2023-10-24T14:05:59.968Z. I can’t find any record of anything changing on our side, but we are using gitlab’s shared template, so something might have changed there. i’m also wondering if this upstream change could potentially be the culprit.

the value of the GCLOUD_AUTH_JSON variable looks like this:

{
  "type": "service_account",
  "project_id": "[redacted]",
  "private_key_id": "[redacted]",
  "private_key": "[redacted]",
  "client_email": "[redacted]",
  "client_id": "[redacted]",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "[redacted]",
  "universe_domain": "googleapis.com"
}

i tried a version with all of the extra white space removed as well, but the failure persists, i.e.,

FATAL flag error: registry flag error: the length of usernames and passwords must match

@outbreaker there is a PR #1401 to support service account json type

@btwseeu78 can you make a simple test , create a secret as I describe above, and deploy a pod with container from private repo which associated with that secret , I want to make sure that you can get containers fro. private repo scanned at all , once we figure out that maybe we can check what enhancement can be done to support your use case

scan-vulnerabilityreport-fb5464d9f-p6rlq 0/1 Error 0 15s scan-vulnerabilityreport-fb5464d9f-p6rlq 0/1 Error 0 15s scan-vulnerabilityreport-fb5464d9f-p6rlq 0/1 Terminating 0 16s scan-vulnerabilityreport-fb5464d9f-p6rlq 0/1 Terminating 0 16s

I know why the error is happening ,I need to check why its hapening in your use case