restic: Unable to backup/restore files/dirs with same name
Output of restic version
restic 0.1.0 compiled at 2016-07-20 12:42:43 with go1.6.3
Expected behavior
After restore all directories should be restored.
Actual behavior
Only one directory is restored.
Steps to reproduce the behavior
- Create files
/tmp/restic/FILESNEW01/Dir01/Test01.txt /tmp/restic/FILESNEW01/Dir01/Test02.txt /tmp/restic/FILESNEW01/Dir01/Test03.txt
/tmp/restic/FILESNEW02/Dir01/Test01.txt /tmp/restic/FILESNEW02/Dir01/Test02.txt /tmp/restic/FILESNEW02/Dir01/Test03.txt
content of files:
cat /tmp/restic/FILESNEW01/Dir01/Test0*
Content file. /tmp/restic/FILESNEW01/Dir01/Test01.txt
Content file. /tmp/restic/FILESNEW01/Dir01/Test02.txt
Content file. /tmp/restic/FILESNEW01/Dir01/Test03.txt
cat /tmp/restic/FILESNEW02/Dir01/Test0*
Content file. /tmp/restic/FILESNEW02/Dir01/Test01.txt
Content file. /tmp/restic/FILESNEW02/Dir01/Test01.txt
Content file. /tmp/restic/FILESNEW02/Dir01/Test03.txt
I want backup
- /tmp/restic/FILESNEW01/Dir01/
- /tmp/restic/FILESNEW02/Dir01/
Commands: Initiate repository in /tmp/restic/BACKUP directory
- restic -r /tmp/restic/BACKUP/ init
Make backup
- restic backup /tmp/restic/FILESNEW01/Dir01 /tmp/restic/FILESNEW02/Dir01 -r /tmp/restic/BACKUP/
scan [/tmp/restic/FILESNEW01/Dir01 /tmp/restic/FILESNEW02/Dir01]
scanned 2 directories, 6 files in 0:00
[0:00] 16.67% 0B/s 51B / 306B 0 / 8 items 0 errors ETA 0:00 duration: 0:00, 0.01MiB/s
snapshot 4d197b90 saved
Checking if backup exists in repository
- restic -r /tmp/restic/BACKUP/ snapshots
ID Date Host Directory
4d197b90 2016-07-26 14:14:43 nebss /tmp/restic/FILESNEW01/Dir01 /tmp/restic/FILESNEW02/Dir01
Restore backup
- restic -r /tmp/restic/BACKUP/ restore 4d197b90 -t /tmp/restic/RESTORE/
restoring <Snapshot 4d197b90 of [/tmp/restic/FILESNEW01/Dir01 /tmp/restic/FILESNEW02/Dir01] at 2016-07-26 14:14:43.208840145 +0300 EEST> to /tmp/restic/RESTORE/
Checking if directories/files exists
- ls /tmp/restic/RESTORE/
Dir01 - cat /tmp/restic/RESTORE/Dir01/Test0*
Content file. /tmp/restic/FILES01/Dir01/Test01.txtContent file. /tmp/restic/FILES01/Dir01/Test02.txtContent file. /tmp/restic/FILES01/Dir01/Test03.txt
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Reactions: 7
- Comments: 28 (16 by maintainers)
Commits related to this issue
- Resolve name collisions At the moment when two items to be saved have the same directory name, restic only saves the first one to the repo. Let's say we have a structure like this: dir1 └── ... — committed to restic/restic by fd0 7 years ago
- Resolve name collisions At the moment when two items to be saved have the same directory name, restic only saves the first one to the repo. Let's say we have a structure like this: dir1 └── ... — committed to restic/restic by fd0 7 years ago
Maybe not for 0.7.1, but for 0.8.0 or so. I’ve already started working on it though. Maybe a bit of background: This is caused by the archiver code, which is the oldest code present in restic. Unfortunately (as I was just beginning to learn Go back in 2013/2014) the archiver code is very complex and I made a lot of beginner mistakes (too much concurrency, too many channels). I also worried about things that turned out to be not a problem at all, and overlooked things that became a problem (e.g. the index handling).
So, I’ve already started on reimplementing the archiver code completely, using concurrency only when it makes sense (i.e. processing individual chunks) and not reading 20 files from the disk in parallel. This code also includes proper directory walking and will insert the complete paths into the repo.
Fortunately, this is really just the archiver that needs to be touched, the rest of the code will (thanks to the design of restic and the repo) just continue to work fine.
@mholt I agree, I’m already working on this. As I said, this is caused by a bad design decision early on and needs to be corrected.
I’ve thought about this and I think we need to change the backup behavior so that always the full path (as given on the command-line) is saved. That’s what
tardoes, and it works very well. This is unfortunately a relict of a bad design decision early in the development for restic.So, #1209 has been merged and it improves the situation by detecting name conflicts and resolving them (by renaming), but this issue is still not fully resolved. I’m working on it 😃
Given that you’re not 1.0 yet, I vote that, if a breaking change has to be made in order for the ideal fix, go ahead and do it sooner rather than later.
-1 for
--store-full-path. I would much rather see the full path always going in the backup and then having a--strip-components <N>to take parts away if you don’t need them at restore time. This means the full data is always available in the backup and if the user strips too many components from the path at restore time and therefore combines subdirs, it becomes a recoverable user error.As to prefixing the hostname to the backup location, this seems it can be easily done from the cmdline, as most people know from which host they are going to restore beforehand 😃