kyma: PSP is not correctly applied to istiod pod in Istio

Description

In Kyma there are two main predefined PodSecurityPolicy’s: unprivileged (001-kyma-unprivileged) and privileged (002-kyma-privileged). We expect that each deployed pod will be granted with one of these. By default pods should not have privileged access to disallow making any trouble.

Expected result

An expected situation is that pods that belong to Istio (istio-ingressgateway and istiod) are assigned to 001-kyma-unprivileged PSP. istio-egressgateway is not considered as we do not validate output traffic.

Actual result

Unfortunately, while deploying Kyma ootb istiod is not always assigned to the correct PSP.

Steps to reproduce

1. Provision GKE cluster from Gardener
2. Deploy Kyma from master on freshly-created cluster
3. Check annotations of istiod pod kubectl get pod -n istio-system <istiod-pod-name> -o json | jq '.metadata.annotations'
4. Check istiod securityContext from container specs kubectl get pod -n <istiod-pod-name> -o json | jq '.spec.containers[0].securityContext'

It looks like on Gardener istiod pod is assigned to Gardener’s PSP (which is provided ootb and we can not manage it anyhow). After comparing securityContext it looks like it’s kind of similar to the one that we use, but not the same.

Troubleshooting

After deeper investigation it looks like the issue managing configuration of the cluster itself that enforces unprivileged access can solve the issue. In order to test it go to Gardener’s YAML configuration and set: allowPrivilegedContainers: false.