rook: Nightly CI issue - OBC bucket lifecycle delete bug - ceph:main [Apr 16 2025]
Is this a bug report or feature request?
- Bug Report
Deviation from expected behavior:
Ceph nightly canaries are failing.
Expected behavior:
Ceph nightly canaries should pass.
How to reproduce it (minimal and precise):
Check this run for failure details. https://github.com/rook/rook/actions/runs/14500172645/job/40677697925
I was investigating other ongoing failures related to object/OBC lifecycle issues and ran a fresh nightly run after having fixed mon space issues.
I am getting a new failure, which seems like it may be related to a recent Ceph PR. I have informed the Ceph devs and asked for more details here: https://github.com/ceph/ceph/pull/61786#issuecomment-2810519520
There may be layered failures with Ceph main, but this will need fixed before we can investigate others.
File(s) to submit:
- Cluster CR (custom resource), typically called
cluster.yaml, if necessary
Logs to submit:
-
Operator’s logs, if necessary
-
Crashing pod(s) logs, if necessary
To get logs, use
kubectl -n <namespace> logs <pod name>When pasting logs, always surround them with backticks or use theinsert codebutton from the Github UI. Read GitHub documentation if you need help.
Cluster Status to submit:
-
Output of kubectl commands, if necessary
To get the health of the cluster, use
kubectl rook-ceph healthTo get the status of the cluster, usekubectl rook-ceph ceph statusFor more details, see the Rook kubectl Plugin
Environment:
- OS (e.g. from /etc/os-release):
- Kernel (e.g.
uname -a): - Cloud provider or hardware configuration:
- Rook version (use
rook versioninside of a Rook Pod): - Storage backend version (e.g. for ceph do
ceph -v): - Kubernetes version (use
kubectl version): - Kubernetes cluster type (e.g. Tectonic, GKE, OpenShift):
- Storage backend status (e.g. for Ceph use
ceph healthin the Rook Ceph toolbox):
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 16 (8 by maintainers)
testing a fix in https://github.com/ceph/ceph/pull/62980
thanks @BlaineEXE. i checked our s3-tests suite to make sure we had test coverage of GetBucketLifecycle after Delete, and found no coverage of Delete at all. i wrote a test case in https://github.com/ceph/s3-tests/pull/645 and see the same failure you are. tracking the regression in https://tracker.ceph.com/issues/71083