kyverno: [Bug] kyverno_policy_results_total is missing from metrics in 1.9.0

Kyverno Version

1.9.0

Kubernetes Version

1.21.x

Kubernetes Platform

EKS

Kyverno Rule Type

Validate

Description

I have read the docs at https://kyverno.io/docs/monitoring/policy-results-info/ and have created a simple clusterpolicy

☁ ➤  k get clusterpolicy require-labels -o yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  creationTimestamp: "2023-02-13T22:59:08Z"
  generation: 3
  name: require-labels
  resourceVersion: "229158175"
  uid: 01963d25-fafd-4515-b657-66a4c5b26406
spec:
  background: true
  rules:
  - match:
      any:
      - resources:
          kinds:
          - Pod
    name: check-for-labels
    validate:
      message: label 'app.kubernetes.io/name' is required
      pattern:
        metadata:
          labels:
            app.kubernetes.io/name: ?*
  validationFailureAction: Audit
status:
  autogen:
    rules:
    - exclude:
        resources: {}
      generate:
        clone: {}
        cloneList: {}
      match:
        any:
        - resources:
            kinds:
            - DaemonSet
            - Deployment
            - Job
            - StatefulSet
            - ReplicaSet
            - ReplicationController
        resources: {}
      mutate: {}
      name: autogen-check-for-labels
      validate:
        message: label 'app.kubernetes.io/name' is required
        pattern:
          spec:
            template:
              metadata:
                labels:
                  app.kubernetes.io/name: ?*
    - exclude:
        resources: {}
      generate:
        clone: {}
        cloneList: {}
      match:
        any:
        - resources:
            kinds:
            - CronJob
        resources: {}
      mutate: {}
      name: autogen-cronjob-check-for-labels
      validate:
        message: label 'app.kubernetes.io/name' is required
        pattern:
          spec:
            jobTemplate:
              spec:
                template:
                  metadata:
                    labels:
                      app.kubernetes.io/name: ?*
  conditions:
  - lastTransitionTime: "2023-02-13T22:59:14Z"
    message: ""
    reason: Succeeded
    status: "True"
    type: Ready
  ready: true
  rulecount:
    generate: 0
    mutate: 0
    validate: 1
    verifyimages: 0

The kubectl interface work great

kubectl get polr -A
###########                cpol-require-labels        43     71     0      0       0      7h20m
###########                cpol-require-labels        0      24     0      0       0      7h19m
###########                cpol-require-labels        42     0      0      0       0      7h19m
###########                cpol-require-labels        0      6      0      0       0      7h19m
###########                cpol-require-labels        5      0      0      0       0      7h19m
###########                cpol-require-labels        0      4      0      0       0      7h19m
###########                cpol-require-labels        5      0      0      0       0      7h19m
###########                 cpol-require-labels        9      0      0      0       0      7h19m
###########                 cpol-require-labels        39     0      0      0       0      7h19m
###########                 cpol-require-labels        0      13     0      0       0      7h19m

However I then went to investigate the metrics and I do not see kyverno_policy_results_total

kubectl port-forward -n kaas-policy svc/kyverno-svc-metrics  8000:8000

I keep reading over documentation and helm chart values. I also tried just a policy but no metrics showed up. I feel like I missed something but I cannot find any issues like this.