strimzi-kafka-operator: [Bug]: Migrating from dual to separate role node pools doesn't unregister brokers

Bug Description

If you create a Kafka cluster with a dual role node pool and then follow the migration guide to migrate to separate broker and controller node pools. The controller nodes never get unregistered as brokers so are left as fenced brokers in the metadata. I believe the issue is here: https://github.com/strimzi/strimzi-kafka-operator/blob/8907528796c418f092f03de27ff0a3f1290446a8/cluster-operator/src/main/java/io/strimzi/operator/cluster/operator/assembly/KafkaReconciler.java#L1008-L1014

List<Integer> currentNodeIds = kafka.nodes().stream().map(NodeRef::nodeId).sorted().toList(); doesn’t check node role so it treats controllers as brokers.

I am interested in contributing to this issue if it is welcome and you agree that this is a bug.

Steps to reproduce

  1. Create a Kafka cluster with a dual role node pool with 3 replicas (ids 0,1,2).
  2. Create a second broker role node pool with 3 replicas (ids 3,4,5).
  3. KafkaRebalance with remove-brokers 0,1,2 and the remove broker role from the first node pool.
  4. Check cluster metadata by running kafka-metadata-shell.sh in a controller pod and you will see registered broker ids 0,1,2,3,4,5 with 0,1,2 being fenced.

Expected behavior

Strimzi to unregister nodes that used to be brokers but are now only controllers.

Strimzi version

0.44.0

Kubernetes version

1.32

Installation method

No response

Infrastructure

No response

Configuration files and logs

No response

Additional context

No response

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 19 (13 by maintainers)

Most upvoted comments

You can take a look at the default replica placement logic in Apache Kafka. The way it operates is well described in https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/metadata/placement/StripedReplicaPlacer.java#L33-L116