moby: Can create a deeply-nested directory structure that can't be deleted
While trying to build coreutils inside a Docker container, I found that the coreutils ./configure step left behind a very deep directory structure that could not be deleted:
$ rm -fr confdir3/
rm: cannot remove 'confdir3/confdir3/confdir3/confdir3/.../confdir3/confdir3/confdir3': File name too long
$ while cd confdir3/; do true; done # Takes some time ...
-bash: cd: confdir3/: File name too long
$ pwd | wc -c
8099
Users could conceivably be affected in other cases in which an application (presumably accidentally) creates an excessively deep directory structure that can not be deleted. It does need to be quite deep: 8000 characters generally represent quite a few directories.
I can reproduce this simply with the following script:
$ while mkdir tempdir3; do cd tempdir3; done
mkdir: cannot create directory 'tempdir3': File name too long
$ pwd | wc -c
8092
$ cd ..
$ rmdir tempdir3/
rmdir: failed to remove 'tempdir3/': File name too long
Initially, I was tempted to blame aufs, but I can run the same experiment outside of a Docker container without hitting the same problem. I generated a directory structure over 10000 directories deep (aufs running on top of ext4), with a path length in excess of 90000 characters, but ran out of patience before finding any upper limit to aufs’ path length.
The coreutils configuration runs code from getcwd-path-max.m4
(from the gnulib project), which includes C code to probe the limits of PATH_MAX
for getcwd()
. It does this by generating a directory structure that runs out to PATH_MAX
, and testing that getcwd()
remains well-behaved. At the end, it politely attempts to remove this directory structure, but silently fails. This does not have any immediate impact on the configuration of coreutils, though a second attempt to ./configure
does have a different result (as mkdir confdir3/
fails). It leaves the user unable to delete the coreutils directory after building, triggering (for instance) this downstream OpenEmbedded/Yocto defect: https://bugzilla.yoctoproject.org/show_bug.cgi?id=7338
Work-arounds:
-
From the top, manually move the deep structure up by one level, and then delete it, eg:
$ mv confdir3/confdir3/ fred $ rm -fr fred/ confdir3/
-
For the coreutils build, patch
getcwd-path-max.m4
(and regenerate the./configure
script).--- coreutils-7.2.original/configure 2009-04-01 01:34:13.000000000 +1300 +++ coreutils-7.2/configure 2015-05-25 12:50:39.616794241 +1200 @@ -19177,6 +19177,7 @@ #include <sys/stat.h> #include <sys/types.h> #include <fcntl.h> +#include <stdio.h> #ifndef AT_FDCWD # define AT_FDCWD 0 @@ -19304,6 +19305,13 @@ /* Try rmdir first, in case the chdir failed. */ rmdir (DIR_NAME); + /* Getting rid of the very bottom dirs inside a Docker container is tricky */ + if (chdir ("../..") < 0) exit (errno); + rename (DIR_NAME"/"DIR_NAME, "c"); + rename (DIR_NAME, "d"); + rmdir ("c"); + rmdir ("d"); + /* Now for the rest */ for (i = 0; i <= n_chdirs; i++) { if (chdir ("..") < 0)
-
Build coreutils in a data volume.
-
Delete the directory structure from the underlying fs (not ideal - requires root privilege).
Docker 1.5.0 Host Ubuntu 15.04 Guest Ubuntu 14.04
About this issue
- Original URL
- State: closed
- Created 9 years ago
- Reactions: 1
- Comments: 22 (11 by maintainers)
Commits related to this issue
- coreutils: Fix deeply nested folder under docker See: - https://github.com/moby/moby/issues/13451 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=7338 — committed to termux/termux-packages by fornwall 7 years ago
- Somes fixes to Debian build, though we're now hitting [https://github.com/moby/moby/issues/13451] — committed to jgoldfar/octave-docker by jgoldfar 5 years ago
- Fix broken Docker builds https://github.com/moby/moby/issues/13451 https://github.com/LedgerHQ/dockcross/blob/master/moxiebox-moxie/packages/gdb/8.2/0001-DouglasRoyds-workaround-for-deeply-nested-con... — committed to HelenOS/binutils-gdb by vhotspur 7 months ago
I have the same problem while building octave 4.0.0 docker image
For me workaround was to change docker storage driver from aufs to overlay2
System information:
I solved this problem. Try this.
echo 9000 > /sys/module/apparmor/parameters/path_max
http://wiki.apparmor.net/index.php/FAQ#Failed_name_lookup_-_name_too_long
This problem with being unable to remove deeply nested confdir3 directories happens when building GNU Octave also. @DouglasRoyds’s patch for getcwd-path-max.m4 is a good workaround. I was also able to run the following line at the bash terminal.