velero: Expired Backup objects from BackupStorageLocations with accessMode=ReadOnly don't get cleaned up
When a BackupStorageLocation is defined with accessMode: ReadOnly Velero imports Backup objects into the cluster as usual.
apiVersion: velero.io/v1
kind: BackupStorageLocation
metadata:
name: my-backups
namespace: velero
spec:
accessMode: ReadOnly
Once the Backups objects expire, Velero usually would remove the Backup objects in the BackupStorageLocation and in the cluster if the accessMode is set to ReadWrite. However, for ReadOnly BackupStorageLocations Velero neither removes the entities in the storage location (which is expected), nor does it remove it in the K8S cluster is it installed. Over the time this behavior could lead to a vast number of Backup objects, which could slow down Velero’s operations.
It is expected that Velero cleans up the Backup (and other objects) in the cluster it is installed even of the accessMode of the BackupStorageLocation is ReadOnly.
About this issue
- Original URL
- State: open
- Created 8 months ago
- Comments: 17 (7 by maintainers)
@phoenix-bjoern This issue will be discussed in the next release(v1.14)'s planning.
@blackpiglet @phoenix-bjoern Hmm. So the use case is the BSL is in 2 clusters, read-write in cluster1, read-only in cluster 2. Cluster 2 won’t delete the backup on expiration since it’s read-only, but cluster1 will delete it and clean out the BSL.
However, the backup sync controller has code to delete orphaned backups. Once the backup is removed from cluster1 and the BSL, it becomes an orphaned backup in cluster2, and the controller shoould delete this backup CR from clsuter2 now, not because it’s expired but because it’s orphaned. It could be that there’s a bug in the backup sync controller that’s preventing this from happening – that’s where I’d look to see what’s going on.