velero: don't error when taking a restic backup if volume is empty

restic returns an error if you try to take a snapshot of an empty directory. I think for Velero’s purposes this should not be reported to the user as an error. We should either check the contents of the volume beforehand, or check for this specific error and handle it gracefully.

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 18 (17 by maintainers)

Most upvoted comments

If you follow all those steps and can’t repro, let me know and we can pair.

skriss on May 7, 2019

I just repro’ed this in my test cluster. I installed velero using --with-restic, deployed the nginx example, added the backup.velero.io/backup-volumes=nginx-logs annotation to the nginx pod, and exec’ed into the pod and deleted all contents of /var/log/nginx.

Here’s what I get in the backup log:

steve@steve-heptio:velero $ velero backup logs nginx-restic-empty | grep error
time="2019-05-07T15:55:01-06:00" level=info msg="1 errors encountered backup up item" backup=velero/nginx-restic-empty group=v1 logSource="pkg/backup/resource_backupper.go:265" name=nginx-deployment-84bbbd6dc7-2x4v9 namespace=nginx resource=pods
time="2019-05-07T15:55:01-06:00" level=error msg="Error backing up item" backup=velero/nginx-restic-empty error="pod volume backup failed: error running restic backup, stderr=Fatal: unable to save snapshot: snapshot is empty\n: exit status 1" error.file="/Users/steve/go/src/github.com/heptio/velero/pkg/restic/backupper.go:173" error.function="github.com/heptio/velero/pkg/restic.(*backupper).BackupPodVolumes" group=v1 logSource="pkg/backup/resource_backupper.go:269" name=nginx-deployment-84bbbd6dc7-2x4v9 namespace=nginx resource=pods

And here’s an abridged version of velero backup describe for the backup:

Name:         nginx-restic-empty
Namespace:    velero
Labels:       velero.io/storage-location=default
Annotations:  <none>

Phase:  PartiallyFailed (run `velero backup logs nginx-restic-empty` for more information)

Errors:    1
Warnings:  0

...

Restic Backups:
  Failed:
    nginx/nginx-deployment-84bbbd6dc7-2x4v9: nginx-logs

skriss on May 7, 2019

Nope – that looks like the log file didn’t get uploaded to object storage – is your backup Completed?

The error message you’re looking for says something like “empty snapshot” – don’t remember the exact text. It’s pretty clear.

skriss on May 7, 2019