restic: Unclear how to recover from "pack ID does not match" errors from "restic check"

Output of `restic version`

restic 0.9.2 compiled with go1.10.3 on windows/amd64 (for initial backup and check) restic 0.9.4 compiled with go1.11.4 on windows/amd64 (for later diagnosis)

How did you run restic exactly?

AWS_ACCESS_KEY_ID=foo AWS_SECRET_ACCESS_KEY=foo RESTIC_REPOSITORY=s3:https://s3.amazonaws.com/myrepo RESTIC_PASSWORD=foo:bar

Ran restic check --check-unused --read-data against the repository after generating 3 snapshots.

Output included:

Pack ID does not match, want 7691f738, got e3b0c442

What backend/server/service did you use to store the repository?

Expected behavior

A simpe means of resolving the error, given that I still have the original data available locally.

Actual behavior

restic find --pack 7691f738 got me as far as

Found blob 1b0ccb7af24b6221798ee900d9f5943e56a186f3f30707e5a7366033415a3e50 … in file /f/2018-04/foo-Sun.7z (tree 3229cbefa8f2e1b09726ef36f701516d01ae0d8c80339125508ca62e23ab7479) … in snapshot 5c8529d9 (2019-02-06 15:58:21) Found blob 11e01454070d44043df7536c6b9a2cde366d7aa97a066daf10d1049d4c7d1c55 … in file /f/2018-04/foo-web-Sun.7z (tree 3229cbefa8f2e1b09726ef36f701516d01ae0d8c80339125508ca62e23ab7479) … in snapshot 5c8529d9 (2019-02-06 15:58:21)

but it’s unclear how I can resolve the issues. Will re-uploading the affected files and then running a forget on the repo simply replace the broken pack with the new version? (Admittedly in a new snapshot, but that’s OK.)

Steps to reproduce the behavior

Upload ~1.5TB of data from a USB-attached SSD from a Windows server to an S3-hosted repo.

If it makes any difference, the upload was rate-limited and interrupted / restarted at a couple of points in order to adjust the rate limit. Unfortunately I’m not sure whether any of the affected files were being uploaded at the time.

Do you have any idea what may have caused this?

Possible I/O issue, as the drives are connected to a Windows server via a USB to SATA cable and at least once (although not while restic was running) the event log shows that Windows believes the drive was disconnected and reconnected (it wasn’t).

Do you have an idea how to solve the issue?

Better worked example of recovering from issues raised by restic check, particularly when --read-data was used.

Did restic help you or made you happy in any way?

Generally I’ve been very impressed with restic so far.

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 2
Comments: 20 (11 by maintainers)

Most upvoted comments

@jkirk The unplanned backup run shouldn’t cause any additional damage and the prune run failed during the initial integrity check so no harm done here either. It is possible that the damaged files had different filenames but still shared the same content, that would be a benign explanation for the too few reported file recoveries.

The name of a pack file depends on the exact file content which also includes random bytes for encryption. This effectively guarantees that restic never creates the same pack filename again. The recovered file chunks are now stored in some other pack files. That is it is expected that the damaged pack file is not recreated.

The --check-unused option is rather uninteresting, it merely reports that a few blobs could be pruned, which is expected once a snapshot was forgotten or a backup run was interrupted. Please just use restic check --read-data to check the repository integrity.

MichaelEischer on Dec 28, 2020