gsutil: gsutil rsync doesn't recognize some directories as suitable for transfer

While trying to move directories recursively to GCS, I’ve run into this error on almost every directory:

CommandException: arg (./guix-backup) does not name a directory, bucket, or bucket subdir.

I’m guessing the tool detects these as files instead of directories for some reason. Here’s output from two commands. One that works and one that doesn’t.

build-staging ~ # ls -lsa /var/log
total 244
  8 drwxr-xr-x.  4 root root              4096 Jun 13 08:49 .
  8 drwxr-xr-x. 11 root root              4096 Aug  2 16:48 ..
  4 -rw-------.  1 root utmp                 0 Jun 13 08:49 btmp
  4 -rw-r--r--.  1 root root                 0 Jun 13 08:49 faillog
  8 drwxr-sr-x.  4 root systemd-journal   4096 Jun 13 08:49 journal
  8 -rw-r--r--.  1 root root            146292 Aug 10 18:46 lastlog
  8 drwx------.  2 root root              4096 Jun  7 23:19 sssd
196 -rw-rw-r--.  1 root utmp            190080 Aug 10 18:46 wtmp
build-staging ~ #  gsutil rsync -r /var/log gs://guix/logs
Building synchronization state...
Starting synchronization
build-staging ~ # ls -lsa /gnu
total 2884
   8 drwxr-xr-x.    5 root root      4096 Aug  2 16:09 .
   4 drwxr-xr-x.   19 root root      4096 Jul 12 08:50 ..
   0 drwxrwxr-t. 1304 root hamann       0 Aug  9 21:44 store
 488 drwxrwxr-t. 1053 root hamann  491520 Aug  2 14:25 store-old
2384 drwxr-xr-x.   35 root root   2433024 Jul 31 18:28 store2
build-staging ~ #  gsutil rsync -r /gnu/store2 gs://guix/store-test
CommandException: arg (/gnu/store2) does not name a directory, bucket, or bucket subdir.

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 1
  • Comments: 15

Most upvoted comments

Sounds sane 😃

A major use case for using gsutil on CoreOS would be downloading some files (onto the host file system) needed to launch a container, IIUC. Given this, it doesn’t seem like gsutil should default to running within a container and not having access to the host file system being referenced in the original arguments to gsutil. I’ve passed along this feedback to our GCE team.

Thanks! This is easy enough to fix by creating another alias that mounts your home directory into the container. Maybe easier and more CoreOS-friendly than installing Python directly on the host.