kyverno: [Bug] [CLI] Variables present in the message section of the policy throws an error

Kyverno CLI Version

1.5.x

Description

While applying polices present in the local workspace in a cluster, throws an error. Why so ?? After inspection, what I found that, if the message section of the policy contains variables. There won’t be any problem until and unless the result or status of the policy is pass. But if the status or result of the policy fails, in that case message section of the policy plays an important role in providing info for the cause of policy failure. Therefore, whole message section is printed on the command line as it is. Above as I mention that the message section contains variables. The values corresponding to variables in message section couldn’t get replaced. Why, may be the policy is itself failing because of variables present in the policy could not get substituted as no variable.yaml file is provided.

Steps to reproduce

Store this policy in separate folder. Let say other/failing/exclude_namespaces_dynamically.yaml
kyverno apply other/failing/ --cluster or go run cmd/cli/kubectl-kyverno/main.go apply other/failing/ --cluster

Applying 1 policy to 17 resources... 
(Total number of result count may vary as the policy is mutated by Kyverno. To check the mutated policy please try with log level 5)
panic: key is not a string: "failed to resolve namespacefilters.data.exclude at path : JMESPath query failed: Unknown key \"namespacefilters\" in path"

goroutine 1 [running]:
k8s.io/klog/v2/klogr.flatten({0xc00350e640, 0x2, 0xc0048c3180?})
	/home/linuzz/go/pkg/mod/k8s.io/klog/v2@v2.60.1-0.20220317184644-43cc75f9ae89/klogr/klogr.go:82 +0x4c5
k8s.io/klog/v2/klogr.klogger.Info({0x0, 0x1, {0x3792a50, 0xe}, {0xc0003cafa0, 0xa, 0xa}, {0x378843a, 0x9}}, 0x0, ...)
	/home/linuzz/go/pkg/mod/k8s.io/klog/v2@v2.60.1-0.20220317184644-43cc75f9ae89/klogr/klogr.go:124 +0x4a6
github.com/go-logr/logr.Logger.Info({{0x428ebe0?, 0xc000400f00?}, 0xc0052a42a0?}, {0x37f64cd, 0x2d}, {0xc0037b6650, 0x1, 0x1})
	/home/linuzz/go/pkg/mod/github.com/go-logr/logr@v1.2.2/logr.go:252 +0xd0
github.com/kyverno/kyverno/pkg/engine.(*validator).buildErrorMessage(0xc000d90800, {0x42678a0, 0xc000803560}, {0xc000e97ea8, 0x11})
	/home/linuzz/go/src/kyverno/kyverno/pkg/engine/validation.go:568 +0x305
github.com/kyverno/kyverno/pkg/engine.(*validator).validatePatterns(0xc000d90800, {0x0?})
	/home/linuzz/go/src/kyverno/kyverno/pkg/engine/validation.go:483 +0x512
github.com/kyverno/kyverno/pkg/engine.(*validator).validateResourceWithRule(0xc000d90800)
	/home/linuzz/go/src/kyverno/kyverno/pkg/engine/validation.go:403 +0xa6
github.com/kyverno/kyverno/pkg/engine.(*validator).validate(0xc000d90800)
	/home/linuzz/go/src/kyverno/kyverno/pkg/engine/validation.go:223 +0x153
github.com/kyverno/kyverno/pkg/engine.processValidationRule({{0x428ebe0?, 0xc000400f00?}, 0x2?}, 0x37a74c7?, 0xc0003b1400)
	/home/linuzz/go/src/kyverno/kyverno/pkg/engine/validation.go:139 +0x4b
github.com/kyverno/kyverno/pkg/engine.validateResource({{0x428ebe0?, 0xc000400eb0?}, 0x203000?}, 0xc0001be700)
	/home/linuzz/go/src/kyverno/kyverno/pkg/engine/validation.go:108 +0x3de
github.com/kyverno/kyverno/pkg/engine.Validate(0xc0001be700)
	/home/linuzz/go/src/kyverno/kyverno/pkg/engine/validation.go:39 +0x216
github.com/kyverno/kyverno/cmd/cli/kubectl-kyverno/utils/common.ApplyPolicyOnResource({0x429f280?, 0xc004426e00}, 0xc0006b40f0, {0x0, 0x0}, 0x94?, _, {{0x0, 0x0, 0x0}, ...}, ...)
	/home/linuzz/go/src/kyverno/kyverno/cmd/cli/kubectl-kyverno/utils/common/common.go:572 +0x1192
github.com/kyverno/kyverno/cmd/cli/kubectl-kyverno/apply.applyCommandHelper({0x57f7840?, 0x0, 0x0}, {0x0, 0x0}, 0x1, 0x0?, {0x0, 0x0}, {0x0, ...}, ...)
	/home/linuzz/go/src/kyverno/kyverno/cmd/cli/kubectl-kyverno/apply/apply_command.go:315 +0x15a5
github.com/kyverno/kyverno/cmd/cli/kubectl-kyverno/apply.Command.func1(0xc000afe000?, {0xc00070ca60?, 0x2?, 0x2?})
	/home/linuzz/go/src/kyverno/kyverno/cmd/cli/kubectl-kyverno/apply/apply_command.go:125 +0x225
github.com/spf13/cobra.(*Command).execute(0xc000afe000, {0xc00070ca40, 0x2, 0x2})
	/home/linuzz/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:856 +0x67c
github.com/spf13/cobra.(*Command).ExecuteC(0xc0007d1180)
	/home/linuzz/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:974 +0x3b4
github.com/spf13/cobra.(*Command).Execute(...)
	/home/linuzz/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:902
main.main()
	/home/linuzz/go/src/kyverno/kyverno/cmd/cli/kubectl-kyverno/main.go:37 +0xe5

Now remove the variables( this and this present in the <message> section of the policy.

Now , repeat 2nd step. You won’t be getting any errors this time.

Expected behavior

According, to my opinion if there isn’t any values of the variable provided. Then in that case message section should print empty or nil in place of variables present in the policy.

Screenshots

Screenshot from 2022-05-21 11-21-00

Kyverno logs

No response

Slack discussion

No response

Troubleshooting

I have read and followed the troubleshooting guide.
I have searched other issues in this repository and mine is not recorded.

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 46 (20 by maintainers)

Most upvoted comments

Hey @viveksahu26, I mainly discussed the implementation with Chip while working on the new operators. I used to check this comment for clarity (here). There is some documentation about these operators here. Let me know if there are any doubts here or on slack.

anushkamittal20 on Jun 14, 2022

To investigate further, I need to know how different operators works. If I’m not wrong hey @anushkamittal20 , you worked upon extending new operators. Do you have any design doc of how these operators works. It would be really helpful.

viveksahu26 on Jun 9, 2022

So, if we don’t provide any values either manually or through values file, then pre-condition won’t allow resource to proceed further checking and it will stop there. Therefore, the final status or Result will be skip. So, at the end independent of the operator(in this policy, AnyNotIn used) used in the pre-condition , the final Result or Status should be skip if values through file or manually are not provided. Am I right till here.

I just checked this in the webhook, and if you do not create the ConfigMap and apply a Pod, the Pod will be allowed since the policy is in audit mode. If you change the policy to enforce it will be blocked because it fails looking up the context:

$ k apply -f pod.yaml 
Error from server: error when applying patch:
{"metadata":{"annotations":{"annotation.corp.com/restrict1":"foo1","kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"kind\":\"Pod\",\"metadata\":{\"annotations\":{\"annotation.corp.com/restrict1\":\"foo1\"},\"labels\":{\"app\":\"busybox\"},\"name\":\"mypod\",\"namespace\":\"default\"},\"spec\":{\"automountServiceAccountToken\":false,\"containers\":[{\"args\":[\"sleep\",\"9999\"],\"image\":\"harbor2.zoller.com/library/busybox:1.28\",\"name\":\"busybox\",\"resources\":{\"limits\":{\"memory\":\"1Gi\"},\"requests\":{\"cpu\":\"100m\",\"memory\":\"50Mi\"}}}]}}\n"},"labels":{"allns":null}},"spec":{"$setElementOrder/containers":[{"name":"busybox"}],"containers":[{"image":"harbor2.zoller.com/library/busybox:1.28","name":"busybox"}]}}
to:
Resource: "/v1, Resource=pods", GroupVersionKind: "/v1, Kind=Pod"
Name: "mypod", Namespace: "default"
for: "pod.yaml": admission webhook "validate.kyverno.svc-fail" denied the request: 

resource Pod/default/mypod was blocked due to the following policies

exclude-namespaces-example:
  exclude-namespaces-dynamically: 'failed to load context: failed to retrieve config
    map for context entry namespacefilters: failed to get configmap default/namespace-filters
    : configmaps "namespace-filters" not found'

We need to make the behavior consistent here regardless of the operator.

chipzoller on Jun 8, 2022