micromamba-docker: mambauser cannot write to bind mounts on Linux with Docker in rootless-mode

If a script wants to write to the current working directory on the host system, an obvious way is to use a bind mount to map a directory on the host to a directory inside the container.

Unfortunately, this does not work with Docker on Linux systems; the non-root mambauser cannot write to directories from bind mounts, no matter if we set the UID/GID to that of the user on the host or not:

# Minimal example (works on Docker Desktop on OSX)
$ docker run --rm -it -v "$(pwd):/tmp" \
   mambaorg/micromamba:1.5.6 /bin/bash
$ id
uid=57439(mambauser) gid=57439(mambauser) groups=57439(mambauser)
$ touch test.txt
touch: cannot touch 'test.txt': Permission denied
$ touch /tmp/test.txt
touch: cannot touch '/tmp/test.txt': Permission denied
# With using the host user's UID and GID
$ docker run --rm -it --user $UID:$GID -v "$(pwd):/home/mambauser"  \
    mambaorg/micromamba:1.5.6 /bin/bash
$ cd /home/mambauser/
$ echo test > test.md
bash: test.md: Permission denied
$ ls -la
total 8
drwxr-xr-x 2 root root 4096 Jan 11 03:15 .
drwxrwxrwx 3 root root 4096 Dec 30 15:30 ..
$ pwd
/home/mambauser

Writing to Docker bind volumes on Linux systems as non-root users is a well-known and complicated topic, but I wonder if there is an elegant way of adding the mambauser to the group that has write access to a bind mount point.

Or is there any other way of writing to a directory on the host from the mambauser?

Note: The issue does not appear on Docker Desktop for OSX, as the built-in VM maps between the host system and the Docker environment.

References:

Addendum: Tested with Micromamba:1.5.6, Docker version 24.0.7, build afdd53b on Debian 11.8 Docker version 24.0.7, build afdd53b

About this issue

  • Original URL
  • State: open
  • Created 6 months ago
  • Comments: 17 (3 by maintainers)

Most upvoted comments

Solution and Conclusion

1. The best solution is to install Podman, which is super-simple, and to use the --userns=keep-id option. As the additional uid/gid parameters described in the current version are not available in e.g. Podman 3.0.1, you best use this command:

podman run --rm -it -v "$(pwd):/tmp" --userns=keep-id --user $(id -u):$(id -g) docker.io/mambaorg/micromamba:1.5.8 /bin/bash

In there, touch testfile.txt works like a charm, and the external username and id is also visible from inside.

2. The second-best solution is to use --user root when starting the container if and only if Docker is running in rootless mode and if the host user has no sudo rights.**. This can be a quick fix with acceptable risks in most cases.

docker run --rm -it -v "$(pwd):/tmp" --user root mambaorg/micromamba:1.5.8 /bin/bash

touch testfile.txt works, too, and the container root user is mapped to the local user on the host system.

The other options do not work; tweaking the subuid and subgid mapping does not work without brittle and intransparent modifications on the host system.

Hope this is useful for many of you! Frankly, it was a nightmare to get to this solution and insight and I learned many things that are intellectually interesting but were not on my bucket-list to master 😉.

Most important to say: micromamba-docker is not all all to blame that it was such a painful journey; it is Docker’s insufficient documentation and the lack of --userns=host and/or --userns=keep-id support; deeply hidden in Docker issues and other sources. The hundreds of related posts and discussions indicates that many, many others are wasting their life-time with this Docker problem. So please spread the word.

TODO: Add summary and pointer to this to the documentation.

@mfhepp thanks for the detailed investigation and clear recommendations. I am certainly in support of increasing our documentation in this area. Do you have any interest in putting together a documentation PR?

Wow, thanks @mfhepp for your persistence in getting to the bottom of this issue! This looks like it was quite some effort.

I have not yet used rootless mode myself (although I probably should), so this is a bit beyond my current comfort level. Thus I don’t have a preference among your suggested approaches for a fix. Perhaps @wholtz has some thoughts?

Still working on the issue… and I think there is an underlying problem that surfaces when you are using micromamba-docker

  1. with the plain Docker daemon (not Docker Desktop),
  2. on a Linux machine
  3. in rootless mode.

As far as I understand, the non-root mambauser conflicts with the way Docker in rootless mode deals with bind mounts and namespace mapping in general.

Docker in rootless mode utilizes rootlesskit, which maps user namespaces between the host system and the system inside the container.

See how rootlesskit changes the username, UID, and GID of resources owned by a local user:

# Create a test folder and file
# Default permissions set by umask
$ mkdir foo
$ cd foo
$ mkdir bar
$ touch mytext.txt
# Show permissions from the host user's perspective
$ ls -la
total 12
drwxr-xr-x  3 myusername myusername 4096 Apr 30 19:23 .
drwxr-xr-x 15 myusername myusername 4096 Apr 30 19:23 ..
drwxr-xr-x  2 myusername myusername 4096 Apr 30 19:23 bar
-rw-r--r--  1 myusername myusername    0 Apr 30 19:23 mytext.txt
$ id
uid=1000(myusername) gid=1000(myusername) groups=1000(myusername),996(docker)
# Now turning on RootlessKit, as used by Docker
# RootlessKit creates user_namespaces and executes newuidmap/newgidmap along with subuid and subgid
# https://github.com/rootless-containers/rootlesskit
$ rootlesskit bash
root@110:~/foo# ls -la
total 12
drwxr-xr-x  3 root root 4096 Apr 30 19:23 .
drwxr-xr-x 15 root root 4096 Apr 30 19:23 ..
drwxr-xr-x  2 root root 4096 Apr 30 19:23 bar
-rw-r--r--  1 root root    0 Apr 30 19:23 mytext.txt
root@110:~/foo# id
uid=0(root) gid=0(root) groups=0(root),65534(nogroup)
root@110:~/foo# exit

You can see that rootlesskit changes the owning user and user group from myusername to root and maps the UID from 1000 to 0.

This means that the non-root mambauser cannot access such files and folders, even if the UID/GID of the local user is passed to the container when Docker is run in the rootless mode (for which there are good reasons).

The exact mapping is determined by /etc/subuid and /etc/subgid.

I am reopening the issue, because

  1. I have been unable to fix this despite considerable effort and
  2. the WWW is full of unsatisfying solutions for the underlying problem.

It may be possible to fix this by

  1. configuring /etc/subuid and /etc/subgid, or modifying
  2. the _entrypoint.sh shell script or
  3. the Dockerfile.

Any ideas will be very much appreciated!

PS: The test from earlier on fails now, too, since I changed my system to a rootless Docker configuration:

$ docker run --rm -it -v "$(pwd):/tmp" --user $(id -u):$(id -g) mambaorg/micromamba:1.5.8 /bin/bash
(base) I have no name!@a84cfa7e0ace:/tmp$ touch test
touch: cannot touch 'test': Permission denied
(base) I have no name!@a84cfa7e0ace:/tmp$ id
uid=1000 gid=1000 groups=1000
(base) I have no name!@a84cfa7e0ace:/tmp$ ls -la
total 12
drwxr-xr-x  3 root root 4096 Apr 30 19:23 .
drwxr-xr-x 17 root root 4096 Apr 30 19:54 ..
drwxr-xr-x  2 root root 4096 Apr 30 19:23 bar
-rw-r--r--  1 root root    0 Apr 30 19:23 mytext.txt

As shown above with rootlesskit alone, despite the indicated GID and UID being identical, the user namespace mechanism changes the owner to root from inside the container:

# Permissions from the host user's perspective
$ ls -la
total 12
drwxr-xr-x  3 myusername myusername 4096 Apr 30 19:23 .
drwxr-xr-x 15 myusername myusername 4096 Apr 30 19:23 ..
drwxr-xr-x  2 myusername myusername 4096 Apr 30 19:23 bar
-rw-r--r--  1 myusername myusername    0 Apr 30 19:23 mytext.txt
$ id
uid=1000(myusername) gid=1000(myusername) groups=1000(myusername),996(docker)