opentelemetry-collector-contrib: [processor/resourcedetection], [receiver/dockerstats] Collector cannot query Docker socket in official contrib images

Describe the bug

The docker detector from the resource detection processor and the dockerstats receiver do not work on official opentelemetry-collector-contrib images, or any other image that runs the Collector under a user other than root.

Steps to reproduce

Run the resource detection processor docker detector or the dockerstats receiver, while mounting the /var/run/docker.sock socket:

docker run -v /var/run/docker.sock:/var/run/docker.sock:ro -v <mount config here> otel/opentelemetry-collector-contrib

What did you expect to see?

The Docker detector should add the host.name of the host machine, and its operating system.

The Docker stats receiver should produce valid metrics.

What did you see instead?

Both components fail because of lack of permissions

What version did you use?

Can be reproduced on the latest version, happens since v0.40.0 (more specifically, since #6380).

What config did you use?

For both components the default configuration on the README can reproduce this; see e.g. the resource detection processor:

processors:
  resourcedetection/docker:
    detectors: [env, docker]
    timeout: 2s
    override: false

Environment

This happens on every Docker version and every Collector image since v0.40.0

Additional context

This happens since #6380, because of a permissions issue: the mounted socket is only readable by root. AFAICT, Docker does not currently allow mounting volumes with permissions for a specific user (see moby/moby#2259), and we can’t chown the socket at build time, so we have to choose between running as rootless or supporting this.

This is not a problem on downstream or custom distributions that run as root.

For getting the hostname on the Docker detector, a workaround is to override the OS hostname on the Docker image using something like --hostname $(hostname). I don’t know of a workaround for getting the hosts’ operating system or getting the metrics on the dockerstats receiver.

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 17 (14 by maintainers)

Most upvoted comments

@carlreid on this front specifically no. I am not aware of a better method unfortunately.

This is a general docker concern and the container user needs to be in the host’s docker group:

$ docker run -v /var/run/docker.sock:/var/run/docker.sock:ro --group-add $(stat -c '%g' /var/run/docker.sock) otel/opentelemetry-collector-contrib <...>
# or if specifying the user:group directly
$ docker run -v /var/run/docker.sock:/var/run/docker.sock:ro --user "some.user:$(stat -c '%g' /var/run/docker.sock)" otel/opentelemetry-collector-contrib <...>

The dockerstatsreceiver also queries the Docker socket and thus suffers from the same problem https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/dockerstatsreceiver#configuration

This is also why I’m here