moby: tar command fails with OverlayFS storage driver but not with aufs

Command to reproduce:

docker run -it --rm heroku/cedar:14 /bin/bash -c "curl -sS https://install.meteor.com | /bin/sh"

On Docker v1.8.1, aufs storage driver, the command completes successfully:

Downloading Meteor distribution
######################################################################## 100.0%

Meteor 1.2.1 has been installed in your home directory (~/.meteor).
Writing a launcher script to /usr/local/bin/meteor for your convenience.

To get started fast:

  $ meteor create ~/my_cool_app
  $ cd ~/my_cool_app
  $ meteor

Or see the docs at:

  docs.meteor.com

On Docker v1.8.3, overfayfs storage driver, the command fails:

Downloading Meteor distribution
######################################################################## 100.0%
tar: .meteor/packages/coffeescript/.1.0.11.148kw9n++os+web.browser+web.cordova/plugin.compileCoffeescript.os/npm/babel-compiler/node_modules/meteor-babel/node_modules/.bin: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova/npm/node_modules/meteor-babel/node_modules/with/node_modules/.bin: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova/npm/node_modules/meteor-babel/node_modules/with/node_modules: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova/npm/node_modules/meteor-babel/node_modules/with: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova/npm/node_modules/meteor-babel/node_modules/transformers/node_modules/.bin: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova/npm/node_modules/meteor-babel/node_modules/transformers/node_modules: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova/npm/node_modules/meteor-babel/node_modules/transformers: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova/npm/node_modules/meteor-babel/node_modules/defs/node_modules/.bin: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova/npm/node_modules/meteor-babel/node_modules/defs/node_modules: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova/npm/node_modules/meteor-babel/node_modules/defs: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova/npm/node_modules/meteor-babel/node_modules/.bin: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova/npm/node_modules/meteor-babel/node_modules: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova/npm/node_modules/meteor-babel: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova/npm/node_modules: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova/npm: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler/.5.8.24_1.127zxv6++os+web.browser+web.cordova: Directory renamed before its status could be extracted
tar: .meteor/packages/babel-compiler: Directory renamed before its status could be extracted
tar: Exiting with failure status due to previous errors
Installation failed.

More logs:

core@deis-03 ~ $ docker version
Client:
 Version:      1.8.3
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   cedd534-dirty
 Built:        Sat Dec  5 05:57:26 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.8.3
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   cedd534-dirty
 Built:        Sat Dec  5 05:57:26 UTC 2015
 OS/Arch:      linux/amd64
core@deis-03 ~ $ docker info
Containers: 11
Images: 69
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.2.2-coreos-r1
Operating System: CoreOS 835.9.0
CPUs: 1
Total Memory: 1.959 GiB
Name: deis-03
ID: SSCW:KJB5:3MI4:WCSV:RXKY:WG2W:HB7U:TRYL:36L4:ZDTC:5KUL:JIMD

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 7
Comments: 56 (26 by maintainers)

Commits related to this issue

Add overlay issue(docker/docker#12327): pip fails Not sure whether this is related to docker/docker#19647.. — committed to AkihiroSuda/issues-docker by AkihiroSuda 8 years ago
Default docker storage set to "aufs" due to https://github.com/docker/docker/issues/19647 — committed to evilmartians/chef-kubernetes by Bregor 7 years ago
Default docker storage set to "aufs" due to https://github.com/docker/docker/issues/19647 — committed to evilmartians/chef-kubernetes by Bregor 7 years ago
replaces bzip2 with bsdtar in order to overcome docker/docker#19647 — committed to mhubig/docker-partkeepr by deleted user 7 years ago

Most upvoted comments

Still present on 2.0.0.2 (30215) (macOS 10.13.6) Moreover, the “storage-driver” : “aufs” soln no longer seems to be accepted. Daemon does not start with that set.

tahsmith on Feb 4, 2019

The bsdtar fix suggested above did not work for me on Mac OS 10.13.4, using Docker 2.0.0.3.

Additionally, Docker removed support for aufs in Docker Community Edition 2.0.0.0-mac78 on 2018-11-19:

https://docs.docker.com/docker-for-mac/release-notes/

While it’s less than ideal, for anyone currently stuck with no alternative solutions and wanting to test builds on Mac OS, it’s possible to switch storage drivers – vfs worked for me. There are caveats, as noted on the page below:

The vfs storage driver is intended for testing purposes, and for situations where no copy-on-write filesystem can be used. Performance of this storage driver is poor, and is not generally recommended for production use.

It requires one line of JSON in the daemon config file:

"storage-driver" : "vfs"

The full list is here:

https://docs.docker.com/storage/storagedriver/select-storage-driver/

Adding this comment in case someone finds it helpful.

rpkyle on Apr 5, 2019

@xinity It plays tricks that tar does not like.

jasonkeene on Feb 7, 2017

Sure, I’ll reopen 😃

Hm, not sure if reporting upstream will help with overlay; we’re actually trying to get the upstream maintainer to apply some patches, but no response so far 😢

thaJeztah on Jan 26, 2016