velero: Restic backup fails with Fatal: invalid id \"8bcb4ae8\": no matching ID found\n

What steps did you take and what happened: We have a daily schedule that does restic backup of around 120 volumes to s3. Only a part of the volumes are successfully backed up. Let’s say out of 120, 67 succeed and the rest failed with the following error:

Fatal: invalid id "8bc4dae8": no matching ID found\n

The same happens when I trigger one of the failed backup manually. As far as I can see there is no difference from the PVC that succeeds and one that fails.

What did you expect to happen: All volumes are backed up successfully.

The output of the following commands will help us better understand what’s going on: (Pasting long output into a GitHub gist or other pastebin is fine.)

• velero backup describe <backupname> or kubectl get backup/<backupname> -n velero -o yaml https://pastebin.com/MrVKQe3f
• velero backup logs <backupname> https://pastebin.com/2PrfCLQz

Anything else you would like to add: I can provide more logs if needed

Environment:

• Velero version (use velero version): 1.2.0
• Velero features (use velero client config get features): -
• Kubernetes version (use kubectl version): Client Version: version.Info{Major:“1”, Minor:“14”, GitVersion:“v1.14.0”, GitCommit:“641856db18352033a0d96dbc99153fa3b27298e5”, GitTreeState:“clean”, BuildDate:“2019-03-25T15:53:57Z”, GoVersion:“go1.12.1”, Compiler:“gc”, Platform:“darwin/amd64”} Server Version: version.Info{Major:“1”, Minor:“15”, GitVersion:“v1.15.5”, GitCommit:“20c265fef0741dd71a66480e35bd69f18351daea”, GitTreeState:“clean”, BuildDate:“2019-10-15T19:07:57Z”, GoVersion:“go1.12.10”, Compiler:“gc”, Platform:“linux/amd64”}