restic: restic should not read CACHEDIR.TAG in parent directories

Output of `restic version`

restic 0.9.6 compiled with go1.13.4 on linux/amd64 restic 0.9.6 (v0.9.6-200-g39f58a2b-dirty) compiled with go1.14.2 on linux/amd64

How did you run restic exactly?

/tmp/sample_dir% tree
.
├── CACHEDIR.TAG
└── src
    └── backup_me.txt

/tmp/sample_dir% restic backup --exclude-caches $PWD/src

% restic ls latest                       
repository 22fc71ff opened successfully, password is correct
snapshot 3c6a0afd of [/tmp/sample_dir/src] filtered by [] at 2020-05-07 15:54:30.679914516 +0300 EEST):
/tmp
/tmp/sample_dir

What backend/server/service did you use to store the repository?

local/rest-server

Expected behavior

I think that tar behavior should be preserver here

/tmp/sample_dir% tar czvf test.tar.gz --exclude-caches $PWD/src
tar: Removing leading `/' from member names
/tmp/sample_dir/src/
/tmp/sample_dir/src/backup_me.txt

So if user explicitly asked to backup /tmp/sample_dir/src then restic should not look for CACHEDIR.TAG files in parent directories.

Actual behavior

restic doesn’t backup files if CACHEDIR.TAG is present in parent directory

Steps to reproduce the behavior

See above

Do you have any idea what may have caused this?

Do you have an idea how to solve the issue?

Did restic help you today? Did it make you happy in any way?

Excellent tool

About this issue

Original URL
State: open
Created 4 years ago
Reactions: 2
Comments: 22 (17 by maintainers)

Most upvoted comments

I think the current behavior is a bug: If you move the src folder from the sample dir into a new folder src0 and then create a backup using restic backup --exclude-if-present CACHEDIR.TAG src0/src, then the content of the src folder is actually included in the backup.

MichaelEischer on May 9, 2020

So how about the ~/.cache directory? If this change would be implemented, I have to mark EVERY directory under ~/.cache with the CACHEDIR.TAG

Only if you run backup ~/.cache/* (and the CACHEDIR.TAG is in ~/cache). The point of this issue is to allow one to backup parts of a directory tree which contains a CACHEDIR.TAG in some parent folder, it restic is explicitly told to backup data within such a folder.

MichaelEischer on Mar 5, 2022

So how about the ~/.cache directory? If this change would be implemented, I have to mark EVERY directory under ~/.cache with the CACHEDIR.TAG

And Brian Ford, the author of the proposal of CACHEDIR.TAG, wrote (I marked the relevant part):

Application Semantics

The presence of a cache directory tag within a given directory indicates that the entire contents of that directory, including any and all subdirectories underneath it, consists of cached information that can be re-generated if necessary from appropriate source material located elsewhere.

See: https://bford.info/cachedir/

So I vote for closing this feature request (not bug)

bohni on Feb 27, 2022

Thanks for reopening this!.

There is no inconsistency in ‘excluding’ part (actually reason of this bug). All tools except restic do exactly same thing.

As about storing empty dir + CACHEDIR.TAG itself, it’s a bit different. And I think that restic is doing right thing here (same as tar). It basically stores information that ‘directory was present, but just excluded from backup’. Ideally restic should also store information about direct exclude flags somewhere in snapshot, but it’s out of scope of this ticket.

dionorgua on May 9, 2020

Again though; why don’t you just put your files that you want backed up in another directory than where you have the CACHEDIR.TAG file? It’s just one directory away.

The problem is that the current behavior is what it is. If we change that, many many other users may see unexpected results. I don’t envision many users asking to back up specific dirs/files under a directory that has a CACHEDIR.TAG file, but we don’t know, and in particular your use case for this is quite specific.

rawtaz on May 7, 2020