rook: Nightly CI issue - OBC bucket lifecycle delete bug - ceph:main [Apr 16 2025]

Is this a bug report or feature request?

  • Bug Report

Deviation from expected behavior:

Ceph nightly canaries are failing.

Expected behavior:

Ceph nightly canaries should pass.

How to reproduce it (minimal and precise):

Check this run for failure details. https://github.com/rook/rook/actions/runs/14500172645/job/40677697925

I was investigating other ongoing failures related to object/OBC lifecycle issues and ran a fresh nightly run after having fixed mon space issues.

I am getting a new failure, which seems like it may be related to a recent Ceph PR. I have informed the Ceph devs and asked for more details here: https://github.com/ceph/ceph/pull/61786#issuecomment-2810519520

There may be layered failures with Ceph main, but this will need fixed before we can investigate others.

File(s) to submit:

  • Cluster CR (custom resource), typically called cluster.yaml, if necessary

Logs to submit:

  • Operator’s logs, if necessary

  • Crashing pod(s) logs, if necessary

    To get logs, use kubectl -n <namespace> logs <pod name> When pasting logs, always surround them with backticks or use the insert code button from the Github UI. Read GitHub documentation if you need help.

Cluster Status to submit:

  • Output of kubectl commands, if necessary

    To get the health of the cluster, use kubectl rook-ceph health To get the status of the cluster, use kubectl rook-ceph ceph status For more details, see the Rook kubectl Plugin

Environment:

  • OS (e.g. from /etc/os-release):
  • Kernel (e.g. uname -a):
  • Cloud provider or hardware configuration:
  • Rook version (use rook version inside of a Rook Pod):
  • Storage backend version (e.g. for ceph do ceph -v):
  • Kubernetes version (use kubectl version):
  • Kubernetes cluster type (e.g. Tectonic, GKE, OpenShift):
  • Storage backend status (e.g. for Ceph use ceph health in the Rook Ceph toolbox):

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 16 (8 by maintainers)

Most upvoted comments

thanks @BlaineEXE. i checked our s3-tests suite to make sure we had test coverage of GetBucketLifecycle after Delete, and found no coverage of Delete at all. i wrote a test case in https://github.com/ceph/s3-tests/pull/645 and see the same failure you are. tracking the regression in https://tracker.ceph.com/issues/71083