compose: Secrets fail to set the `uid`, `gid` and `mode` specified in `docker-compose.yml`

Description

Docker secrets specified using the long syntax for the docker-compose.yml file fail to set the specified uid, gid and mode.

Also, from the docs, the default value of the uid and gid fields should be the user that runs the container however the value remains whatever was set on the host machine.

Steps to reproduce the issue:

Create a Dockerfile

❯ cat Dockerfile
FROM ubuntu:20.04

RUN adduser tester -u 1005 --disabled-password
CMD ["/bin/bash", "-c", "ls -al /run/secrets/target_secret_file.txt"]

Create a docker-compose.yml

❯ cat docker-compose.yml
services:
  secrets-tester:
    build: .
    secrets:
      - source: some_secret_file
        target: target_secret_file.txt
        uid: "1005"
        gid: "1005"
        mode: 0440

secrets:
  some_secret_file:
    file: somefile.txt

Create a text file for secrets (somefile.txt)

❯ cat somefile.txt
Text from a secret file

Run the service docker compose run secrets-tester

Describe the results you received: Received Output:

❯ docker compose run secrets-tester
-rw-r--r-- 1 1000 1000 24 Jul 13 16:47 /run/secrets/target_secret_file.txt

Describe the results you expected: Expected Output:

❯ docker compose run secrets-tester
-r--r----- 1 1005 1005 24 Jul 13 16:47 /run/secrets/target_secret_file.txt

Additional information you deem important (e.g. issue happens only occasionally): Same behavior is observed in these cases:

Without creating the tester user
With same source and target names for the file in docker-compose.yml
Using the uid and gid for root
Using random values for uid and gid
Different values for mode

Output of docker compose version:

❯ docker --version
Docker version 20.10.17, build 100c701

❯ docker compose version
Docker Compose version v2.6.0

Output of docker info:

❯ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.8.2-docker)
  compose: Docker Compose (Docker Inc., v2.6.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 2
  Running: 0
  Paused: 0
  Stopped: 2
 Images: 1
 Server Version: 20.10.14
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc version: v1.1.2-0-ga916309
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.102.1-microsoft-standard-WSL2
 Operating System: Ubuntu 20.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 7.763GiB
 Name: DESKTOP-IGS6AOM
 ID: MBVL:76QX:UWDQ:AK7Z:BNGW:PYGL:EXFG:ZHGD:JM53:RBFZ:4SQV:5I7Z
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

Additional environment details:

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 17
Comments: 47

Commits related to this issue

Applied work around for https://github.com/docker/compose/issues/9648 — committed to serversideup/docker-ssh by jaydrogers a year ago

Most upvoted comments

I’m using a secret for a SSH Private Key that must have a 0400 mode. Setting the uid, gid and mode is a requirement.

    secrets:
      - source: ssh_host_ecdsa_key
        target: /etc/ssh/ssh_host_ecdsa_key
        mode: 0400

2022-12-14 20:41:23 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2022-12-14 20:41:23 @         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
2022-12-14 20:41:23 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2022-12-14 20:41:23 Permissions 0777 for '/etc/ssh/ssh_host_ed25519_key' are too open.
2022-12-14 20:41:23 It is required that your private key files are NOT accessible by others.
2022-12-14 20:41:23 This private key will be ignored.
2022-12-14 20:41:23 sshd: no hostkeys available -- exiting.

mario-gravel on Dec 15, 2022

sure, I’m not suggesting here to write your own hack-ish script to use docker cp, just saying we have an opportunity here to replace bind mounts for secrets/config with copy to better control permissions and ownership within container’s filesystem

ndeloof on May 22, 2023

The documentation says mode, uid, and gid all work for compose services for both configs and secrets: https://docs.docker.com/compose/compose-file/05-services/#long-syntax and https://docs.docker.com/compose/compose-file/05-services/#long-syntax-4

Why is this issue being closed as “not planned”. It is critical that files be loaded into the container with controlled / explicit permissions. Go try running filebeat with a custom filebeat.yml file loaded in without setting the permissions on the file correctly and see how far you get.

What possible excuse can there be for this commitment to be backed out on?

Loading files into the service without control of permissions is absurd.

quarky42 on Dec 12, 2023

@ianhinder yes indeed

ndeloof on May 20, 2023

latest docker compose release warn you about this feature not being supported

WARN[0000] secrets `uid`, `gid` and `mode` are not supported, they will be ignored

the reason is that secrets are implemented as bind mounts, which comes with this limlitation.

ndeloof on Nov 23, 2023

Could this also be extended to configs? Currently I have to build a new image with the config files baked in, as docker configs in compose don’t work with a remote docker host. If they were copied, similar to the secrets here, this would enable the use of local config files which appeared as standard configs on a remote host.

ianhinder on May 22, 2023

Marking this issue as “kind/enhancement” as secrets never have supported having uid/gid set by compose (v1 or v2), this feature was introduced and only supported by docker swarm

ndeloof on Jan 12, 2023

@quarky42 compose specification is a vendor-neutral definition of the compose file format, it doesn’t define implementation details and limitation by Docker Compose. This feature can’t be implemented by Docker Compose as the docker engine has no support for config or secret (only when ran in swarm mode), and it uses a bind mount hack as a workaround, which can’t offer such options.

I feel like the confusion around this is warranted. The documentation that one would use for building a docker-compose.yaml doesn’t indicate anywhere that there’s a split in how secrets work when you use docker compose vs swarm. It’s defined in the specification, and seemingly works partially in docker compose because of the aforementioned “bind mount hack”…though there’s no indication via the documentation that any such hack is why it works in docker compose.

isujtauke on Dec 12, 2023

@quarky42 compose specification is a vendor-neutral definition of the compose file format, it doesn’t define implementation details and limitation by Docker Compose. This feature can’t be implemented by Docker Compose as the docker engine has no support for config or secret (only when ran in swarm mode), and it uses a bind mount hack as a workaround, which can’t offer such options.

ndeloof on Dec 12, 2023

@peterge1998 the key thing about the docs is the explicit wording

uid and gid: The numeric UID or GID that owns the file within /run/secrets/ in the service’s task containers. Default value is USER running container.

a service task explictly refers to swarm containers

That was totally unclear to me. Can we make more explicit which features are only supported with docker swarm?

bodograumann on Jul 1, 2023

unfortunately this approach is currently blocked by https://github.com/moby/moby/issues/34142

ndeloof on May 23, 2023

we could copy secret again at any time, or have a dedicated command for this purpose, but the more obvious one would be to trigger a container restart

ndeloof on Dec 15, 2022

As bind mounts do not support different permissions (uid, gid and mode), I suspect that in order to support this, the content of such secrets must be copied into the container (as in #9553), or, use POSIX ACLs (more complex). Am I right @ndeloof? But why does the compose documentation mention such options if they are not supported in the first place?

markkrj on Oct 25, 2022