istio: Kubernetes DNS cert signing using Kubernetes API in unsupported way
See https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/20190607-certificates-api.md for context.
The API is explicitly NOT intended to be used the way we currently use it. There are multiple issues.
The first, which we already knew, is
Distribution of trust happens out of band for these signers. Any trust outside of those described above are strictly coincidental. For instance, some distributions may honor kubernetes.io/legacy-unknown as client-certificates for the kube-apiserver, but this is not a standard. None of these usages are related to ServiceAccount token secrets .data[ca.crt] in any way. That ca-bundle is only guaranteed to verify a connection the kube-apiserver using the default service.
This means this may break at any time and will only work on some Kubernetes providers that just so happen to use the same CA for both. We already know this is not the case for many.
The other issue is expecting the Kubernetes CA to accept our CSRs in the first place. Look at https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/20190607-certificates-api.md#signers, these are intended for internal usage, not what we are doing. See the signer names like kubernetes.io/kube-apiserver-client-kubelet. We get away with this by using kubernetes.io/legacy-unknown, which happens to sort of work but is likely to be removed in the v1 API.
The intent of this API is to allow other signers, like cert-manager, to sign certs. It is not meant to be used the way we use it.
About this issue
- Original URL
- State: open
- Created 4 years ago
- Comments: 40 (29 by maintainers)
As anecdotal evidence, OpenShift uses distinct roots of trust for the CSR API and
serviceaccount/ca.crt[1].[1] https://github.com/openshift/cluster-kube-controller-manager-operator/pull/150
On the
istiod.istio-system.svcserving cert bit:Recently I have been thinking about having Kube itself provide a built-in short lived, auto rotated CA for the service network (similar to https://github.com/openshift/service-ca-operator but more deeply integrated without the use of annotations). I will not try to go into specifics here, but basically the goal would be to provide trust distribution for this short lived CA and allow a service to request a serving cert for itself which a pod would then mount. Basically, it should be trivial for every service on Kube cluster to always use TLS without needing to install cert-manager. You should be able to set up a webhook that uses
clientConfig.serviceor an API service that usesspec.servicewithout having to fuss around with serving certs (other than tell your code “use this file as the serving cert and make sure to reload it”).No, this is broken in Kubernetes1.22 which is an officially supported version for 1.11 I believe. So this does belong in the 1.11 milestone.
Most of the work is just documenting that this does not work as it did before though. I think @shankgan is taking that on (?)