moby: Cannot load hosts in configuration json file when starting docker daemon

This is a problem if you are using the default systemd unit file, which is hardcoded to start dockerd with -H fd://.

quick workaround (Debian 8)): create /etc/systemd/system/docker.service.d/docker.conf

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd

then systemctl daemon-reload

This is definitely a bug in the systemd configuration. I have DOCKER_OPTS="" but systemd on Ubuntu 16.04 starts the docker daemon with -H fd://. I want to configure docker with TLS over the network.

Please re-open this critical bug I consider this a blocking and breaking bug.

$ grep 'ExecStart' /lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS

Considering I’m trying to use daemon.json because /etc/default/docker states:

# Here in Debian, this file is sourced by:
#   - /etc/init.d/docker (sysvinit)
#   - /etc/init/docker (upstart)
#   - systemd's docker.service

# Use of this file for configuring your Docker daemon is discouraged.

# The recommended alternative is "/etc/docker/daemon.json", as described in:
#   https://docs.docker.com/v1.11/engine/reference/commandline/daemon/#daemon-configuration-file

I recommend either fixing this bug or removing the message Use of this file for configuring your Docker daemon is discouraged. from /etc/default/docker to avoid confusion.

# apt-cache show docker.io
Package: docker.io
Priority: optional
Section: universe/admin
Installed-Size: 54232
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Paul Tagliamonte <paultag@debian.org>
Architecture: amd64
Version: 1.12.1-0ubuntu13~16.04.1
Replaces: docker (<< 1.5~)
Depends: adduser, containerd (>= 0.2.3~), iptables, runc (>= 1.0.0~rc1~), init-system-helpers (>= 1.18~), lsb-base (>= 4.1+Debian11ubuntu7), libapparmor1 (>= 2.6~devel), libc6 (>= 2.14), libdevmapper1.02.1 (>= 2:1.02.97)
Recommends: ca-certificates, cgroupfs-mount | cgroup-lite, git, ubuntu-fan, xz-utils, apparmor
Suggests: aufs-tools, btrfs-tools, debootstrap, docker-doc, rinse, zfs-fuse | zfsutils
Breaks: docker (<< 1.5~)
Filename: pool/universe/d/docker.io/docker.io_1.12.1-0ubuntu13~16.04.1_amd64.deb
Size: 10581186
MD5sum: 08676d065c43fc6bcb450cf52c782ed5
SHA1: 8f51b8c7b0fbc3eb879e13a76286cbe13b096d34
SHA256: 0987a665d090d90b0096fbdcab7d18316cad6a1754dbc70f56ab74c36ba53c51
...

This is expected; you cannot specify options both as a flag and in the configuration file (daemon.json). If you change your DOCKER_OPTS to DOCKER_OPTS="" and restart, then it should work. We explicitly don’t “merge” these configurations.

I’ll close this issue, as I don’t think there’s a bug here, but feel free to comment 👍

This just hit me also.

Considering the default config breaks with a daemon.json containing hosts - where hosts is a supported config option (and a very common one I would think?) should we not consider this a bug still!?

//cc @thaJeztah

we use daemon.json to avoid to discriminate what file to change based on the system (upstart/systemV, etc.) But if we have to manually empty the DOCKER_OPTS, it has no sense. so dameon should have the priority

Same, I still override it. @thaJeztah is it worth reconsidering this a bug? It still affects me but I gave up trying to change your mind until people started reporting it again.

Looks like it was documented here: https://docs.docker.com/engine/admin/#troubleshoot-conflicts-between-the-daemonjson-and-startup-scripts

I’d still like to see a modification of the packaging or Docker itself so that people don’t have to work around this. Anyone using TLS over API but also the socket will need to specify their hosts, and people upgrading from another version where they used a host config in daemon.json will have Docker break until they end up finding this issue or the new doc via their favorite search engine.

Maybe docker engine should prefer settings in daemon.json over those supplied by the flag, or vice versa.

@thaJeztah, any response to @duckie? The use of -H is clearly visible in docker-ce’s systemd file, and was introduced in the last version where it wasn’t used before. This broke several working configurations.

Thanks @asbjornenge, real time saver. This really should be considered as a bug. According to the doc:

Note: You cannot set options in daemon.json that have already been set on daemon startup as a flag. On systems that use systemd to start the Docker daemon, -H is already set, so you cannot use the hosts key in daemon.json to add listening addresses. See https://docs.docker.com/engine/admin/systemd/#custom-docker-daemon-options for how to accomplish this task with a systemd drop-in file.

But when I turned to the reference, there isn’t solution for this problem.

@thaJeztah thanks for the explanation.

However, -H is definitely used by the systemd unit file provided in the official docker-ce package, at least for the RHEL family. Therefore, it should be considered a packaging bug. -H should not be used in the unit file so that it stays as generic as possible, and the default -H value should be provided in the default daemon.json file instead.

Unless there is a rationale to force the host value ? I cant see any valid reason.

The ExecStart= with no value is important here and I dont get why

When you use a systemd drop-in (“override”) file, you are specifying options to override, or add to the main systemd unit file.

However, a systemd unit file is allowed to have multiple ExecStart instructions (see the systemd man page. If you specify ExecStart in your “override” file, it is added to the existing list of ExecStart instructions in the main systemd unit file (and those ExecStart commands will be joined together).

To override / replace the ExecStart in the main systemd unit file, it is therefore needed to first reset the ExecStart, and then define the new ExecStart to replace it. So;

Example content of your override file (e.g./etc/systemd/system/docker.service.d/my-override.conf);

# This line resets / "removes" the original ExecStart as was defined in the main systemd unit file
ExecStart=

# This line defines the new ExecStart to use _instead_
ExecStart=/usr/bin/dockerd

Never edit the original systemd unit file, because doing so will mark it as “edited by user”, which means that if you upgrade Docker to a newer version (install a newer version of the rpm / deb package), and that version has an updated systemd unit-file, the file won’t be updated automatically with the new version.