kyverno: [Bug] err: invalid kind" "kind"="RangeAllocation" in OpenShift

Kyverno Version

1.8.1

Kubernetes Version

1.23.x

Kubernetes Platform

OpenShift

Kyverno Rule Type

Mutate

Description

I have a strange behaviour after updating to 1.8.1 (same with 1.8.0)…

OpenShift adds some annotations to namespaces regarding SCC:

kind: Namespace
metadata:
  annotations:
    openshift.io/sa.scc.mcs: 's0:c33,c27'
    openshift.io/sa.scc.supplemental-groups: 1001110000/10000
    openshift.io/sa.scc.uid-range: 1001110000/10000

I use a mutating ClusterPolicy like the following… and as you can see this policy is NOT operating on namespaces.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: team-mutate-resource-labels
spec:
  background: false
  failurePolicy: Ignore
  validationFailureAction: enforce
  rules:
  - name: required-labels
    match:
      any:
      - resources:
          kinds:
          - "*"
          namespaces:
          - "team-susi-*"
    exclude:
      any:
      - resources:
          kinds:
          - Namespace
          - TokenReview
          - EndpointSlice
    mutate:
      patchStrategicMerge:
        metadata:
          labels:
            owner.ns.example.com/team: ""
            team.example.com/name: "susi"

With this poilcy in place I inspect the logs and see tons of messages like:

E1024 14:47:09.120489 1 handlers.go:180] webhooks/resource/mutate "msg"="mutation failed" "error"="mutation policy team-mutate-resource-labels error: failed to validate resource mutated by policy team-mutate-resource-labels: pre-validation: couldn't find model , err: invalid kind" "kind"="RangeAllocation" "name"="scc-uid" "namespace"="" "operation"="UPDATE" "uid"="0d17f66d-196b-4015-9804-379aeaa4b90f"

This Object really exists as an OpenShift internal one:

apiVersion: security.internal.openshift.io/v1
data: AgAzzBACgAzz///////////w==
kind: RangeAllocation
metadata:
  name: scc-uid
range: 1000000000-1999999999/10000

But additionally to the tons of log entries OpenShift stops adding the scc annotations to new namespaces:

kind: Namespace
metadata:
  annotations:
     !all annotations from above are missing!!

Strange…

Now I reconfigure my kyverno instance:

apiVersion: v1
kind: ConfigMaps
metadata:
  name: kyverno
data:
   resourceFilters: '[RangeAllocation,*,*].............

and… the error logs disappear and the annotations are back…

With 1.7.2 everything is working as ecpected…

Any idea?

Steps to reproduce

See Description…

Expected behavior

No error logs and scc annotations on namespaces.

Screenshots

No response

Kyverno logs

E1024 14:47:09.120489       1 handlers.go:180] webhooks/resource/mutate "msg"="mutation failed" "error"="mutation policy team-mutate-resource-labels error: failed to validate resource mutated by policy team-mutate-resource-labels: pre-validation: couldn't find model , err: invalid kind" "kind"="RangeAllocation" "name"="scc-uid" "namespace"="" "operation"="UPDATE" "uid"="0d17f66d-196b-4015-9804-379aeaa4b90f"

Slack discussion

No response

Troubleshooting

  • I have read and followed the documentation AND the troubleshooting guide.
  • I have searched other issues in this repository and mine is not recorded.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 56 (30 by maintainers)

Most upvoted comments

  • ⚠️BREAKING: Unresolved JMESPath expressions are fixed to properly evaluate as null instead of an empty string (''). Some policy updates to use existence checks may be necessary or otherwise may now deny validating or mutating requests.

I have added this statement to the 1.8.0 release notes:

  • ⚠️BREAKING: Unresolved JMESPath expressions are fixed to properly evaluate as null instead of an empty string (''). Some policy updates to use existence checks may be necessary.

Do you agree this is sufficient?

Thank you! This makes it very clear the issue here is as suspected: the JMESPath filter {{namespaceLabels.sourceLabel}} is evaluating to null for the this-one-is-denied Deployment thereby resulting in an invalid mutate directive.