strimzi-kafka-operator: [Bug]: strimzi-cluster-operator RBAC prevents creation of Kafka cluster

Bug Description

Kafka resource reports failure due to missing RBAC permissions for the strimzi-cluster-operator

Status:
  Auto Rebalance:
    Last Transition Time:  2025-05-11T11:01:53.345351445Z
    State:                 Idle
  Conditions:
    Last Transition Time:  2025-05-11T11:01:53.469575515Z
    Message:               Failure executing: POST at: [https://172.30.0.1:443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings](https://172.30.0.1/apis/rbac.authorization.k8s.io/v1/clusterrolebindings). Message: clusterrolebindings.rbac.authorization.k8s.io "strimzi-kafka-preview-kafka-preview-kafka-init" is forbidden: user "system:serviceaccount:strimzi-operator:strimzi-cluster-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:strimzi-operator" "system:authenticated"]) is attempting to grant RBAC permissions not currently held:
{APIGroups:[""], Resources:["nodes"], Verbs:["get"]}. Received status: Status(apiVersion=v1, code=403, details=StatusDetails(causes=[], group=rbac.authorization.k8s.io, kind=clusterrolebindings, name=strimzi-kafka-preview-kafka-preview-kafka-init, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=clusterrolebindings.rbac.authorization.k8s.io "strimzi-kafka-preview-kafka-preview-kafka-init" is forbidden: user "system:serviceaccount:strimzi-operator:strimzi-cluster-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:strimzi-operator" "system:authenticated"]) is attempting to grant RBAC permissions not currently held:
{APIGroups:[""], Resources:["nodes"], Verbs:["get"]}, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Forbidden, status=Failure, additionalProperties={}).

Steps to reproduce

  1. Strimzi operator installs without issues. Kafka resource gets reconciled but reports failure

Kafka kakfa-preview resource recognized and attempts to reconcile

2025-05-11 06:19:45 INFO  AbstractOperator:520 - Reconciliation #1(watch) Kafka(kafka-preview/kafka-preview): Kafka kafka-preview in namespace kafka-preview was ADDED
2025-05-11 06:19:45 INFO  AbstractOperator:266 - Reconciliation #1(watch) Kafka(kafka-preview/kafka-preview): Kafka kafka-preview will be checked for creation or modification
2025-05-11 06:19:45 INFO  CrdOperator:123 - Reconciliation #1(watch) Kafka(kafka-preview/kafka-preview): Status of Kafka kafka-preview in namespace kafka-preview has been updated
2025-05-11 06:19:45 INFO  AbstractOperator:520 - Reconciliation #2(watch) Kafka(kafka-preview/kafka-preview): Kafka kafka-preview in namespace kafka-preview was MODIFIED
2025-05-11 06:19:45 INFO  Ca:987 - Reconciliation #1(watch) Kafka(kafka-preview/kafka-preview): Generating CA with subject=Subject(organizationName='io.strimzi', commonName='cluster-ca v0', dnsNames=[], ipAddresses
=[])
2025-05-11 06:19:45 INFO  KafkaAssemblyOperator:792 - Reconciliation #3(watch) Kafka(kafka-preview/kafka-preview): KafkaNodePool controller-broker-zone-c in namespace kafka-preview was ADDED
2025-05-11 06:19:45 INFO  KafkaAssemblyOperator:792 - Reconciliation #4(watch) Kafka(kafka-preview/kafka-preview): KafkaNodePool controller-broker-zone-b in namespace kafka-preview was ADDED
2025-05-11 06:19:45 INFO  KafkaAssemblyOperator:792 - Reconciliation #5(watch) Kafka(kafka-preview/kafka-preview): KafkaNodePool controller-broker-zone-a in namespace kafka-preview was ADDED
2025-05-11 06:19:46 INFO  Ca:987 - Reconciliation #1(watch) Kafka(kafka-preview/kafka-preview): Generating CA with subject=Subject(organizationName='io.strimzi', commonName='clients-ca v0', dnsNames=[], ipAddresses
=[])
2025-05-11 06:19:46 INFO  Ca:442 - Reconciliation #1(watch) Kafka(kafka-preview/kafka-preview): Generating certificate Subject(organizationName='io.strimzi', commonName='cluster-operator', dnsNames=[], ipAddresses=
[]), signed by CA cluster-ca
2025-05-11 06:19:46 WARN  VersionUsageUtils:60 - The client is using resource type 'kafkanodepools' with unstable version 'v1beta2'
2025-05-11 06:19:46 ERROR AbstractOperator:285 - Reconciliation #1(watch) Kafka(kafka-preview/kafka-preview): createOrUpdate failed
io.vertx.core.impl.NoStackTraceThrowable: Failure executing: POST at: https://172.30.0.1:443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings. Message: clusterrolebindings.rbac.authorization.k8s.io "strimzi-ka
fka-preview-kafka-preview-kafka-init" is forbidden: user "system:serviceaccount:strimzi-operator:strimzi-cluster-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:strimzi-operator" "system:authent
icated"]) is attempting to grant RBAC permissions not currently held:
{APIGroups:[""], Resources:["nodes"], Verbs:["get"]}. Received status: Status(apiVersion=v1, code=403, details=StatusDetails(causes=[], group=rbac.authorization.k8s.io, kind=clusterrolebindings, name=strimzi-kafka-
preview-kafka-preview-kafka-init, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=clusterrolebindings.rbac.authorization.k8s.io "strimzi-kafka-preview-kafka-preview-kafka-init" is f
orbidden: user "system:serviceaccount:strimzi-operator:strimzi-cluster-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:strimzi-operator" "system:authenticated"]) is attempting to grant RBAC perm
issions not currently held:
{APIGroups:[""], Resources:["nodes"], Verbs:["get"]}, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Forbidden, status=Failure, addi
tionalProperties={}).
  1. strimzi-cluster-operator RBAC requires adding get permissions for nodes
> kubectl get clusterrole strimzi-cluster-operator.-<generated> -oyaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    olm.managed: "true"
    olm.owner: strimzi-cluster-operator.v0.46.0
  name: strimzi-cluster-operator.-<generated>
rules:
- apiGroups:
  - rbac.authorization.k8s.io
  resources:
  - clusterrolebindings
  verbs:
  - get
  - list
  - watch
  - create
  - delete
  - patch
  - update
- apiGroups:
  - storage.k8s.io
  resources:
  - storageclasses
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - list 
  - get  # Required addition added manually
  1. Kafka brokers get created and Kafka cluster goes into a stable state
> kubectl get pods -n kafka-preview
NAME                                             READY   STATUS    RESTARTS        AGE
cruise-control-ui-5999c48476-t8r4k               2/2     Running   405 (26m ago)   34h
kafka-preview-controller-broker-zone-a-0         1/1     Running   0               27m
kafka-preview-controller-broker-zone-b-1         1/1     Running   0               27m
kafka-preview-controller-broker-zone-c-2         1/1     Running   0               27m
kafka-preview-cruise-control-5cc47dbd6f-9dsvq    1/1     Running   0               25m
kafka-preview-entity-operator-67c5b69f7b-nc5xv   2/2     Running   0               26m
kafka-preview-kafka-exporter-6698658776-qrhxk    1/1     Running   0               25m

Expected behavior

strimzi-cluster-operator clusterrole RBAC is a managed resource by the strimzi-kafka-operator https://github.com/strimzi/strimzi-kafka-operator/blob/main/install/cluster-operator/021-ClusterRole-strimzi-cluster-operator-role.yaml

There should be no manual intervention to apply or update these resources after operator is installed.

Strimzi version

0.46.0

Kubernetes version

OpenShift 4.16.38

Installation method

YAML subscription

Infrastructure

OpenShift on AWS

Configuration files and logs

strimzi operator generated RBAC

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    olm.managed: "true"
    olm.owner: strimzi-cluster-operator.v0.46.0
  name: strimzi-cluster-operator.-<generated>
rules:
- apiGroups:
  - rbac.authorization.k8s.io
  resources:
  - clusterrolebindings
  verbs:
  - get
  - list
  - watch
  - create
  - delete
  - patch
  - update
- apiGroups:
  - storage.k8s.io
  resources:
  - storageclasses
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - list

Additional context

No response

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 24 (24 by maintainers)

Most upvoted comments

I am not strongly against a proper 0.46.1 even if I can’t see any real confusion in users. Said that I think @im-konge volunteered to run this release (with me at the back). There will be really nothing in terms of difference in the Strimzi codebase but we’ll need a CHANGELOG 0.46.1 section with something explaining we are fixing this bug.

@ppatierno Sounds good. Will you roll out the new version for both OperatorHub.io as well as OCP?

Yes. That’s the plan. I wanted first doing some local tests by building a catalog and check upgrades. But quay.io is unavailable to push containers since this morning. I will work on this as soon as quay comes back working.