kubernetes: K8s IngressClass Blocks Ingress Controllers Deployed on Namespace Scope
Hi k8s community,
After upgrading of k8s cluster to v1.18+ we get the following error during startup of several ingress-controllers (traefik v2.3+, nginx v1.9.0+)
Traefik v2.3+:
E0107 06:45:40.895430 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.2/tools/cache/reflector.go:125: Failed to list *v1beta1.IngressClass: ingressclasses.networking.k8s.io is forbidden: User "system:serviceaccount:traefik:traefik" cannot list resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope
NGINX v1.9.0+:
F0305 07:02:12.462868 1 main.go:300] Error when getting IngressClass nginx: ingressclasses.networking.k8s.io "nginx" is forbidden: User "system:serviceaccount:nginx:nginx" cannot get resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope
The problem: Within k8s v1.18+ the “IngressClass” was introduced to avoid bad annotations on “Ingress” resource objects themselves (see here) . However, the idea of that concept is pretty cool but lacks in implementation because it requires access on cluster scope level. Hence, developers or teams which can only work on namespace scope cannot use ingress controllers on k8s > 1.18.x.
Possible solutions could be: Provide access to “ingressclasses” on k8s v1.18+ on namespace scope, too. In my opinion it must be possible to define (i.e. read/write) ingressclasses also on namespace scope.
Temporary Solution: Currently we downgraded our ingress-controllers to lower versions where listing of “ingressclasses” is not implemented and uses the old annotation scheme within ingress specs. We also raised issues for the ingress-controller suppliers. Nothing happened so far. Shameful.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 21 (12 by maintainers)
Thanks to @santode and @maplain for linking this issue to the corresponding Gateway API discussions. I’m quite hesitant to add a new namespace-scoped {Ingress/Gateway}Class resource to these APIs but I do think we need to provide a better path forward for namespace-scoped Ingress/Gateway implementations.
One option that’s been mentioned for Gateway API would be to make GatewayClass an optional resource and allow the
gatewayClassNamefield on Gateway to represent a name that was already meaningful to the controller. I think a similar approach could potentially work for Ingress as well.We’re hard at work trying to make this and many other concepts better with the Gateway API. We’d love to have more participants in our efforts to improve these APIs. Of course we’re also pulling back whatever we can from the Gateway API to Ingress, such as the recent addition of namespace-scoped parameters references for Ingress.
I think it will likely be easier to continue this discussion on https://github.com/kubernetes-sigs/gateway-api/issues/567 since this is so closely related. Once we reach a conclusion there I’m hoping it will be equally helpful for both APIs. So if you’re interested in improving these APIs, please join the discussions we’re having there.
I’m assigning this issue to myself, but that doesn’t mean I can solve it on my own. Continued discussion and likely eventually a KEP will be necessary to solve this.
/assign /kind feature