WSL: Docker-compose and incorrect absolute paths for volumes
windows 10 build 1503.11 running Xenial 16.04.2 LTS with docker client 17.03.0 connecting to the hyper-V docker-for-windows daemon 17.03.1 and docker-compose 1.11.1
From within wsl things like docker run --rm -v c:/my/stuff:/var/goes/here bash ls /var/goes/here work fantastically. This command will simply list the contents of c:/my/stuff.
What does not work is If you have a docker-compose.yml that looks like:
version: '2'
services:
bash_it:
image: bash
volumes:
- ./:/var/goes/here
and you run something like docker-compose -p test -d bash_it up && docker exec test_bash_it_1 ls /var/goes/here. Everything will explode and die - you may have to restart the docker-for-windows daemon / engine.
the error will look something like:
ERROR: for proxy Cannot start service proxy: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:359: container init caused \"ro
otfs_linux.go:54: mounting \\\"/mnt/c/my/stuff\\\" to rootfs \\\"/var/lib/docker/overlay2/e4bda146bae2ce38c13ad8f3c4ff5f6e
6f3198041ce02d4c6f593f65948e7086/merged\\\" at \\\"/var/lib/docker/overlay2/e4bda146bae2ce38c13ad8f3c4ff5f6e6f3198041ce02d4c6f593f65948e7086/merged/var/goes/here\\\"
caused \\\"not a directory\\\"\""
: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
ERROR: Encountered errors while bringing up the project.
It’s all rather complicated looking, but I’m pretty sure I can tell you what’s going on: the windows-for-docker daemon expects a valid windows path (eg c:/my/stuff) BUT docker-compose is calling os.path.abspath to resolve the absolute path of the resource and it’s getting a URI/path that looks like /mnt/c/my/stuff - which is absolute nonsense unless you’re in the windows linux subsytem… which the hyper-v daemon is clearly not.
so… that’s an interesting problem. Is there a switch like MSYS_NO_PATHCONV=1 or something I can do to prevent os.path.abspath from returning a wsl alias? OR - how should this work?
About this issue
- Original URL
- State: closed
- Created 10 years ago
- Reactions: 37
- Comments: 40 (9 by maintainers)
@sakai135 – ah, interesting: I initially assumed Docker for Windows did Windows path translation server-side but it would make a lot of sense for it to do it client-side (in which case the Linux client would indeed need to pass in a VM-compatible path).
@therealkenc – ooh, that’s a good thought: WSL already supports bind-mounts. Therefore it does (in essence) support mounting DrvFS on arbitrary paths. So, proposed workaround (untested):
From my holistic point of view, the ideal solution would be for all us (and I really expect it happen some day), iff WSL will converge with Docker and Docker on Windows will move to use WSL compatibility layer instead of MobyLinux VM. In this case Docker Compose could be used the way it is used on Linux and transparently rely on WSL underneath.
How to access WSL filesystem from composer? Each folder I ‘mount’ using volumes command appear empty in container… The workarounds described here shows how to access windows drives, not WSL…
@villasv I can confirm that adding the following to the
/etc/wsl.conffile on 17093 appears to work:I brought up my docker project and can access all of my volume mounts 👍
For what it’s worth, there are some applications (Web servers in particular) that will deliberately refuse to read files from any path that contains a symlink. Usually for security reasons, because malicious users could use symlinks to escape DocumentRoot. Bind mounts usually don’t suffer from this limitation – anyone can create a symlink anywhere, but you need elevated permissions (traditionally, root) to set up a bind-mount, so sysadmins can still use them but they’re harder to exploit.
Has anyone found a way to make
sudo mount --bind /mnt/c /cpermanent yet? I know I could just add it to something like~/.zshrc.However, that requires me to enter a password every time I want to open the terminal >.>. It would be nice if we could add it to something like/etc/rc.localand have it automatically start up when I run the terminalI think the bind trick is a hack, this should be made to work:
Having to bind the drive first, then navigating to that new fake drive, makes things really complicated. If you already have a development setup with scripts that run the docker-compose, this is annoying thing. I can’t go and edit the development scripts because they are correct.
Right now I should do a new script just for WSL, and that is clunky.
os.path.abspathreturning/mnt/c/pathwithin WSL makes total sense though. The issue is the mismatch between/mnt/cof WSL and/cin the VM of Docker for Windows.Within the Hyper-V VM, the
C:is mounted to/cinstead of/mnt/c. I believe https://github.com/docker/for-win/issues/371 would be the ideal solution, which is to make Docker for Windows mountC:into/mnt/cinside the VM.Please refer to wsl.conf to help with your configuration settings. You may also find this post on using Docker with WSL to be useful
Hello! Yesterday I’ve had same issue with mounting folders with Docker-compose and this tip has helped me: https://github.com/docker/compose/issues/4303#issuecomment-391394800
it was just as simple as “Reset credentials” in Docker “Shared drives”, update password to current and restart Docker (in reset menu).
Maybe it will help somebody too.
Ah, ok. For Docker Toolbox (using VirtualBox) you can actually map drives (host C: or D:) under (guest, visible to docker containers) /mnt/c /mnt/d directories, so the workarounds mentioned in this issue could be skipped and the problem solved through VM settings.
Just as a reference for anyone facing this with Docker Toolbox (using VirtualBox by default).
Either make the bind or the symbolic link sugested above inside the
docker-machineenvironment:sudo ln -s /c/ /mnt/c@aseering you made me realize that it was unusual that the WSL docker client was taking non-canonical paths as arguments, so I went back and tried
which seemed to work! (but the listing wasn’t complete?!) so then I had no idea what the problem could have been, and I panicked a bit.
Then the conversation went on, and the
bindfor DrvFS came up:Then I was able to reproduce my issue with a simple
docker-composeexample, and thebindcompletely resolves the issue.Just a further note: When I first tried using the
bindI was getting a new set of IO errors because I had created the containers with docker-compose on the/mnt/cvolume, moved to the/cvolume and simply did adocker-compose upwithout recreating the containers from the images. If anyone else has this problem: be sure to re-instantiate new containers from the original images after you switch mount points.Thanks.
So, this is a little more conceptually complicated: Unlile in Cygwin,
/mnt/c/...is not an alias in WSL. It is the canonical and only way to refer to that directory.C:\...is not a valid Linux path; therefore not a valid WSL path. (C:\would immediately cause errors on Linux, so WSL does likewise.)In this particular case, maybe you could try putting an absolute Windows path in your config file? You might also want to bring this up on the Docker forums – they seem to be pretty active, and the Docker team seems to care a lot about compatibility. (And this would be a really easy, if kind of hackish, thing for them to work around.)