moby: Docker build that fails with overlay storage backend only?

This Dockerfile doesn’t seem to build with the overlay storage backend:

from ubuntu:trusty
RUN apt-get update && apt-get -y upgrade
RUN apt-get install -y build-essential g++ libblas-dev liblapack-dev gfortran python-dev python-pip
RUN pip install --upgrade numpy
RUN pip install --upgrade statsmodels
RUN pip install --upgrade flask

It ends up spitting out the below errors:

Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1443, in install
    requirement.commit_uninstall()
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 610, in commit_uninstall
    self.uninstalled.commit()
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1860, in commit
    rmtree(self.save_dir)
  File "/usr/lib/python2.7/dist-packages/pip/util.py", line 43, in rmtree
    onerror=rmtree_errorhandler)
  File "/usr/lib/python2.7/shutil.py", line 247, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "/usr/lib/python2.7/shutil.py", line 247, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "/usr/lib/python2.7/shutil.py", line 247, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "/usr/lib/python2.7/shutil.py", line 247, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "/usr/lib/python2.7/shutil.py", line 256, in rmtree
    onerror(os.rmdir, path, sys.exc_info())
  File "/usr/lib/python2.7/shutil.py", line 254, in rmtree
    os.rmdir(path)
OSError: [Errno 39] Directory not empty: '/tmp/pip-dXgb99-uninstall/usr/lib/python2.7/dist-packages'

The same Dockerfile works with both AUFS and DeviceMapper storage backends (have not tried with BTRFS).

docker version:

Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.4.1
Git commit (client): a8a31ef
OS/Arch (client): linux/amd64
Server version: 1.5.0
Server API version: 1.17
Go version (server): go1.4.1
Git commit (server): a8a31ef

docker info:

Debug mode (server): false
Debug mode (client): true
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Kernel Version: 3.18.0-031800-generic
Fds: 10
Goroutines: 15
EventsListeners: 0
Init Path: /usr/bin/docker
Docker Root Dir: /var/lib/docker
WARNING: No swap limit support

uname -a:

Linux my_server 3.18.0-031800-generic #201412071935 SMP Mon Dec 8 00:36:34 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Any suggestions or additional info I can provide?

About this issue

  • Original URL
  • State: closed
  • Created 9 years ago
  • Reactions: 1
  • Comments: 48 (22 by maintainers)

Commits related to this issue

Most upvoted comments

For those who don’t know, until this gets merged, you can bypass this problem by running the pip upgrade command together with apt-get install to avoid file system layering. Something like this:

RUN apt-get install python-pip && pip install --upgrade pip
+7
pkout on Jan 14, 2016

What works for me as a workaround is to force installing the package that needs to be updated in the same layer as the update itself.

Example: using python as base image and wanting to upgrade pip. But pip has been installed in python base image, so it gives an error.

FROM python:3.4.4
RUN pip install pip==8.0.2

Solution:

FROM python:3.4.4
RUN pip install --ignore-installed pip && pip install pip==8.0.2

The same goes for dependent packages.

Example: installing ipython which upgrades setuptools. Although it looks like an install, not an update, internally ipython tries to update setuptools, causing an error.

FROM python:3.4.4
RUN pip install ipython

Solution:

FROM python:3.4.4
RUN pip install --ignore-installed setuptools && pip install ipython

It’s just a workaround, but it’s pretty simple and effective. Of course, as everyone else, I’d love to see the patch for overlayfs merged.

+1
jgoclawski on Feb 22, 2016
Read more comments on GitHub
← moby: Docker build is not running cause the permission of one folder not used in the build process
moby: Docker build/push performance is inexplicably slow →