WSL: Docker-compose and incorrect absolute paths for volumes

windows 10 build 1503.11 running Xenial 16.04.2 LTS with docker client 17.03.0 connecting to the hyper-V docker-for-windows daemon 17.03.1 and docker-compose 1.11.1

From within wsl things like docker run --rm -v c:/my/stuff:/var/goes/here bash ls /var/goes/here work fantastically. This command will simply list the contents of c:/my/stuff.

What does not work is If you have a docker-compose.yml that looks like:

version: '2'
  services:
    bash_it:
      image: bash
      volumes:
        - ./:/var/goes/here

and you run something like docker-compose -p test -d bash_it up && docker exec test_bash_it_1 ls /var/goes/here. Everything will explode and die - you may have to restart the docker-for-windows daemon / engine.

the error will look something like:

ERROR: for proxy  Cannot start service proxy: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:359: container init caused \"ro
otfs_linux.go:54: mounting \\\"/mnt/c/my/stuff\\\" to rootfs \\\"/var/lib/docker/overlay2/e4bda146bae2ce38c13ad8f3c4ff5f6e
6f3198041ce02d4c6f593f65948e7086/merged\\\" at \\\"/var/lib/docker/overlay2/e4bda146bae2ce38c13ad8f3c4ff5f6e6f3198041ce02d4c6f593f65948e7086/merged/var/goes/here\\\"
 caused \\\"not a directory\\\"\""
: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
ERROR: Encountered errors while bringing up the project.

It’s all rather complicated looking, but I’m pretty sure I can tell you what’s going on: the windows-for-docker daemon expects a valid windows path (eg c:/my/stuff) BUT docker-compose is calling os.path.abspath to resolve the absolute path of the resource and it’s getting a URI/path that looks like /mnt/c/my/stuff - which is absolute nonsense unless you’re in the windows linux subsytem… which the hyper-v daemon is clearly not.

so… that’s an interesting problem. Is there a switch like MSYS_NO_PATHCONV=1 or something I can do to prevent os.path.abspath from returning a wsl alias? OR - how should this work?

About this issue

  • Original URL
  • State: closed
  • Created 10 years ago
  • Reactions: 37
  • Comments: 40 (9 by maintainers)

Commits related to this issue

Most upvoted comments

@sakai135 – ah, interesting: I initially assumed Docker for Windows did Windows path translation server-side but it would make a lot of sense for it to do it client-side (in which case the Linux client would indeed need to pass in a VM-compatible path).

@therealkenc – ooh, that’s a good thought: WSL already supports bind-mounts. Therefore it does (in essence) support mounting DrvFS on arbitrary paths. So, proposed workaround (untested):

$ sudo mkdir /c
$ sudo mount --bind /mnt/c /c
$ cd /c/path/to/project
$ docker-compose ...

From my holistic point of view, the ideal solution would be for all us (and I really expect it happen some day), iff WSL will converge with Docker and Docker on Windows will move to use WSL compatibility layer instead of MobyLinux VM. In this case Docker Compose could be used the way it is used on Linux and transparently rely on WSL underneath.

How to access WSL filesystem from composer? Each folder I ‘mount’ using volumes command appear empty in container… The workarounds described here shows how to access windows drives, not WSL…

@villasv I can confirm that adding the following to the /etc/wsl.conf file on 17093 appears to work:

[automount]
root = /

I brought up my docker project and can access all of my volume mounts 👍

you_dont_need_a_license_to_drive_a_whale After fighting with Windows and Docker for a bit, I found out the secret to mapping the drives is as follows: Update Docker so that it has access to the drive in question (see screenshot), and begin the path using /[drive] (in this case, /c/whatever)

For what it’s worth, there are some applications (Web servers in particular) that will deliberately refuse to read files from any path that contains a symlink. Usually for security reasons, because malicious users could use symlinks to escape DocumentRoot. Bind mounts usually don’t suffer from this limitation – anyone can create a symlink anywhere, but you need elevated permissions (traditionally, root) to set up a bind-mount, so sysadmins can still use them but they’re harder to exploit.

Has anyone found a way to make sudo mount --bind /mnt/c /c permanent yet? I know I could just add it to something like ~/.zshrc .However, that requires me to enter a password every time I want to open the terminal >.>. It would be nice if we could add it to something like /etc/rc.local and have it automatically start up when I run the terminal

I think the bind trick is a hack, this should be made to work:

     volumes:
     - "./db:/var/db"

Having to bind the drive first, then navigating to that new fake drive, makes things really complicated. If you already have a development setup with scripts that run the docker-compose, this is annoying thing. I can’t go and edit the development scripts because they are correct.

Right now I should do a new script just for WSL, and that is clunky.

os.path.abspath returning /mnt/c/path within WSL makes total sense though. The issue is the mismatch between /mnt/c of WSL and /c in the VM of Docker for Windows.

Within the Hyper-V VM, the C: is mounted to /c instead of /mnt/c. I believe https://github.com/docker/for-win/issues/371 would be the ideal solution, which is to make Docker for Windows mount C: into /mnt/c inside the VM.

Please refer to wsl.conf to help with your configuration settings. You may also find this post on using Docker with WSL to be useful

Hello! Yesterday I’ve had same issue with mounting folders with Docker-compose and this tip has helped me: https://github.com/docker/compose/issues/4303#issuecomment-391394800

it was just as simple as “Reset credentials” in Docker “Shared drives”, update password to current and restart Docker (in reset menu).

Maybe it will help somebody too.

Ah, ok. For Docker Toolbox (using VirtualBox) you can actually map drives (host C: or D:) under (guest, visible to docker containers) /mnt/c /mnt/d directories, so the workarounds mentioned in this issue could be skipped and the problem solved through VM settings.

Just as a reference for anyone facing this with Docker Toolbox (using VirtualBox by default).

Either make the bind or the symbolic link sugested above inside the docker-machine environment: sudo ln -s /c/ /mnt/c

@aseering you made me realize that it was unusual that the WSL docker client was taking non-canonical paths as arguments, so I went back and tried

$ docker run --rm -v /mnt/c/Users/ej3/dev/play/:/var/play bash ls -la /var/play
total 4
drwxr-xr-x    5 root     root           100 Apr  5 21:05 .
drwxr-xr-x    1 root     root          4096 Apr  5 21:47 ..
drwxr-xr-x    2 root     root            40 Apr  5 20:58 Dockerfile
drwxr-xr-x    2 root     root            40 Apr  5 21:02 configure.file
drwxr-xr-x    2 root     root            40 Apr  5 21:05 test.dev.conf

which seemed to work! (but the listing wasn’t complete?!) so then I had no idea what the problem could have been, and I panicked a bit.

Then the conversation went on, and the bind for DrvFS came up:

$ docker run --rm -v /c/Users/ej3/dev/play/:/var/play bash ls -la /var/play
total 12
drwxr-xr-x    2 root     root          4096 Apr  5 21:44 .
drwxr-xr-x    1 root     root          4096 Apr  5 21:47 ..
drwxr-xr-x    2 root     root             0 Apr  5 21:44 .git
-rwxr-xr-x    1 root     root            81 Apr  5 21:10 Dockerfile
-rwxr-xr-x    1 root     root           675 Apr  5 21:44 README.md
-rwxr-xr-x    1 root     root           139 Apr  5 21:19 bash_it.sh
-rwxr-xr-x    1 root     root           116 Apr  5 21:04 docker-compose.yml
-rwxr-xr-x    1 root     root            39 Apr  5 21:09 test.conf
-rwxr-xr-x    1 root     root            40 Apr  5 21:08 test.dev.conf
$

Then I was able to reproduce my issue with a simple docker-compose example, and the bind completely resolves the issue.

Just a further note: When I first tried using the bind I was getting a new set of IO errors because I had created the containers with docker-compose on the /mnt/c volume, moved to the /c volume and simply did a docker-compose up without recreating the containers from the images. If anyone else has this problem: be sure to re-instantiate new containers from the original images after you switch mount points.

Thanks.

So, this is a little more conceptually complicated: Unlile in Cygwin, /mnt/c/... is not an alias in WSL. It is the canonical and only way to refer to that directory. C:\... is not a valid Linux path; therefore not a valid WSL path. (C:\ would immediately cause errors on Linux, so WSL does likewise.)

In this particular case, maybe you could try putting an absolute Windows path in your config file? You might also want to bring this up on the Docker forums – they seem to be pretty active, and the Docker team seems to care a lot about compatibility. (And this would be a really easy, if kind of hackish, thing for them to work around.)