rook: Rook 1.5.9 does not watch for added nodes
Is this a bug report or feature request?
- Bug Report
Deviation from expected behavior: When adding new nodes to the k8s cluster, rook operator doesn’t notice. OSDs are not launched on the newly added node. Restarting the operator causes the expected orchestration to happen.
Ref: https://rook-io.slack.com/archives/CK9CF5H2R/p1616494779156200
Expected behavior: Rook should notice new nodes, and should deploy OSDs to the new nodes
How to reproduce it (minimal and precise): Create cluster that watches for all nodes and devices. There are no traces of any node watch handlers being initiate in the operator log.
- Cluster CR (custom resource), typically called
cluster.yaml, if necessary
spec:
storage:
useAllNodes: true
useAllDevices: true
Environment: OS (e.g. from /etc/os-release): NAME=“Ubuntu” VERSION=“20.04.1 LTS (Focal Fossa)” Kernel (e.g. uname -a): Linux ip-10-4-129-216.eu-north-1.compute.internal 5.4.0-1035-aws Cloud provider or hardware configuration: AWS Rook version (use rook version inside of a Rook Pod): 1.5.9 Storage backend version (e.g. for ceph do ceph -v): ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable) Kubernetes version (use kubectl version): 1.17.17 Kubernetes cluster type (e.g. Tectonic, GKE, OpenShift): kops (1.19.1) Storage backend status (e.g. for Ceph use ceph health in the Rook Ceph toolbox): HEALTH_OK
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 15 (8 by maintainers)
Commits related to this issue
- ceph: update OSDs reconcile OSDs were not created on the newely added nodes beacause operator was not triggering the reconcile, for adding OSD Closes: https://github.com/rook/rook/issues/7462 Signed... — committed to parth-gr/rook by parth-gr 3 years ago
- ceph: update OSDs reconcile OSDs not automatically started when adding nodes to existing cluster. Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush m... — committed to parth-gr/rook by parth-gr 3 years ago
- ceph: update OSDs reconcile OSDs not automatically started when adding nodes to existing cluster. Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush m... — committed to parth-gr/rook by parth-gr 3 years ago
- ceph: update OSDs reconcile OSDs not automatically started when adding nodes to existing cluster. Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush m... — committed to parth-gr/rook by parth-gr 3 years ago
- ceph: update OSDs reconcile OSDs not automatically started when adding nodes to existing cluster. Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush m... — committed to parth-gr/rook by parth-gr 3 years ago
- ceph: update OSDs reconcile OSDs not automatically started when adding nodes to existing cluster. Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush m... — committed to parth-gr/rook by parth-gr 3 years ago
- ceph: update OSDs reconcile OSDs not automatically started when adding nodes to existing cluster. Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush m... — committed to parth-gr/rook by parth-gr 3 years ago
- ceph: update OSDs reconcile OSDs not automatically started when adding nodes to existing cluster. Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush m... — committed to parth-gr/rook by parth-gr 3 years ago
- ceph: update OSDs reconcile OSDs not automatically started when adding nodes to existing cluster. Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush m... — committed to parth-gr/rook by parth-gr 3 years ago
- ceph: update OSDs reconcile OSDs not automatically started when adding nodes to existing cluster. Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush m... — committed to parth-gr/rook by parth-gr 3 years ago
- ceph: update OSDs reconcile OSDs not automatically started when adding nodes to existing cluster. Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush m... — committed to parth-gr/rook by parth-gr 3 years ago
- ceph: update OSDs reconcile OSDs not automatically started when adding nodes to existing cluster. Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush m... — committed to rook/rook by parth-gr 3 years ago
- ceph: update OSDs reconcile OSDs not automatically started when adding nodes to existing cluster. Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush m... — committed to rook/rook by parth-gr 3 years ago
- ceph: update OSDs reconcile OSDs not automatically started when adding nodes to existing cluster. Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush m... — committed to rook/rook by parth-gr 3 years ago
- ceph: update OSDs reconcile OSDs not automatically started when adding nodes to existing cluster. Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush m... — committed to subhamkrai/rook by parth-gr 3 years ago
@paalkr, please wait until #7568 is merged and I’ll suggest using the official rook image. Thanks
Please let me know if you have a docker image with the fixes applied, that you would like me to test.
@travisn , I appreciate that you spend time to follow up on the issues I have reported. Thx!