velero: Velero fails to expose correct backup metrics after a pod restart

What steps did you take and what happened: The metric velero_backup_last_status exposes the status of the latest backups. Once a backup has been taken, the metric gets updated. However, a pod restart in between any two scheduled backups resets the metric exposed by velero_backup_last_status. The metric only gets updated for backups after they are created.

What did you expect to happen: Ideally, the metric should read the list of backups and set the velero_backup_last_status metric. So if a backup happens at 12:00 and the velero pod is restarted or killed at 12:30, the metric should not be set to 0 (which indicates no backup has been taken).

The following information will help us better understand what’s going on:

Environment:

  • Velero version (use velero version): v1.11.0
  • Velero features (use velero client config get features): <NOT SET>
  • Kubernetes version (use kubectl version): v1.26.8
  • Kubernetes installer & version: kops v1.26.5
  • Cloud provider or hardware configuration: AWS
  • OS (e.g. from /etc/os-release): Ubuntu 20.04.5 LTS

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project’s top voted issues listed here.
Use the “reaction smiley face” up to the right of this comment to vote.

  • 👍 for “I would like to see this bug fixed as soon as possible”
  • 👎 for “There are more important bugs to focus on right now”

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 8
  • Comments: 23 (9 by maintainers)

Most upvoted comments

@vinayan3 we are using velero_backup_last_successful_timestamp, since it works as expected.

velero_backup_last_status is just un-useable for now.

I feel this is still relevant

Yes, I’m expecting the same behavior from velero_backup_last_successful_timestamp where the timestamp is the real timestamp of the latest backup and not an default value.