moby: Conflict between default `-H fd://` systemd config and daemon.json `hosts` config

@moonbrv try the following

• overide the docker.service - add /etc/systemd/system/docker.service.d/override.conf:

  [Service]
  ExecStart=
  ExecStart=/usr/bin/dockerd
  

  Using ExecStart is not an issue. It has to be “cleaned” first, before setting a value.

• reload the systemd daemon:

  systemctl daemon-reload
  

• restart docker:

  systemctl restart docker.service
  

PS:

Below the /etc/systemd/system/multi-user.target.wants/docker.service from my machine:

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target docker.socket
Requires=docker.socket

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd://
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target

And the /etc/docker/daemon.json:

{
  "tlsverify": true,
  "tlscacert": "/etc/docker/ca.pem",
  "tlscert"  : "/etc/docker/cert.pem",
  "tlskey"   : "/etc/docker/key.pem",
  "hosts"    : ["fd://", "tcp://0.0.0.0:2376"],
  "dns"      : ["8.8.8.8","8.8.4.4"],
  "ipv6"     : false
}

@dmitriyse: don’t patch the systemd files directly but use systemd drop-in (as suggested by @czerasz 3 comments up):

# As root
mkdir -p /etc/systemd/system/docker.service.d
echo '[Service]
ExecStart=
ExecStart=/usr/bin/dockerd' > /etc/systemd/system/docker.service.d/simple_dockerd.conf
systemctl daemon-reload
service docker restart

This way you only override the ExecStart and it doesn’t get modified on every apt upgrade.

Not sure why the “workarounds” are considered the known and good practice. It’s good we have workarounds but shouldn’t this just work? Docker should update their codebase to be able to handle this; docs aside.

The “right” way to modify unit files is via systemd drop-ins: https://docs.docker.com/engine/admin/systemd/#/custom-docker-daemon-options

Yep, bug has been open for more than one year and there is not even an hint in the documentation.

Please add this tip somewhere in the documentation. It’s too difficult to find the right answer.

I try a this and works for me: Edit my daemon.json and add:

    "hosts": [
        "tcp://127.0.0.1:2375", 
        "unix:///var/run/docker.sock"
    ],

my daemon.json final:

{
    "data-root": "/mnt/dietpi_userdata/docker-data",
    "log-driver": "journald",
    "log-level": "warn",
    "hosts": [
        "tcp://127.0.0.1:2375", 
        "unix:///var/run/docker.sock"
    ],
    "debug": false
}

then:

mkdir -p /etc/systemd/system/docker.service.d
echo '[Service]
ExecStart=
ExecStart=/usr/bin/dockerd' > /etc/systemd/system/docker.service.d/simple_dockerd.conf
systemctl daemon-reload
service docker restart

And docker runs now If your type: curl -X GET http://localhost:2375/containers/json?all=1 You can retrieve the containers list using any http client

another one there. problem persists for ages! Why it is so hard to make it obvious?

if it starts by default - ok. systemd. if config persists - that is a flag, that user wants something more, so config file should have higher priority!

and no that suckish workarounds, which leads to adding unnecessary files to track.

I just hit this on a new install on Ubuntu 16.04 after running Docker for years on Ubuntu 16.04. I think that expecting people to know this much detail about systemd is a big problem because it took me quite a while to even find the relevant error message in journalctl -xe.

How about removing socket activation entirely so that daemon.json works (as documented which is recommended to be used)?

Instead of documenting how to workaround socket activation, simply document how to enable socket activation for those who want it. You can specify remote hosts and define socket activation in the hosts field of daemon.json.

While we’re at it also document what are socket activations uses/benefits because I’ve seen this term said a lot here but have yet to read any practical use for it.

Sorry if i post in wrong place I have Ubuntu 16.04 LTS and docker 1.12.1 When i try to run docker daemon, I still have problem as described above.

>> service docker status
● docker.service
   Loaded: loaded (/etc/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since чт 2016-10-20 00:39:06 EEST; 1min 33s ago
  Process: 28828 ExecStart=/usr/bin/dockerd $OPTIONS $DOCKER_STORAGE_OPTIONS $DOCKER_NETWORK_OPTIONS $BLOCK_REGISTRY $INSECURE_REGISTRY (code=exited, status=1/FAILURE)
 Main PID: 28828 (code=exited, status=1/FAILURE)

жов 20 00:39:06 moonbrv-580 systemd[1]: Started docker.service.
жов 20 00:39:06 moonbrv-580 dockerd[28828]: time="2016-10-20T00:39:06+03:00" level=fatal msg="unable to configure the Docker daemon with file /etc/docker/daemon.json: EOF\n"
жов 20 00:39:06 moonbrv-580 systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
жов 20 00:39:06 moonbrv-580 systemd[1]: docker.service: Unit entered failed state.
жов 20 00:39:06 moonbrv-580 systemd[1]: docker.service: Failed with result 'exit-code'.

In my docker.service i have -H fd:// option

>> grep 'ExecStart' /lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS

I did remove -H fd:// from this file but have the same problem. I’m really sorry for that question, Ubuntu is new thing for me, but please explain me what i need to do to fix that problem. You mention about drop-in file but I don’t understand what i need to write. My /etc/systemd/system/docker.service have next content:

[Service]
EnvironmentFile=-/etc/sysconfig/docker
EnvironmentFile=-/etc/sysconfig/docker-storage
EnvironmentFile=-/etc/sysconfig/docker-network
ExecStart=
ExecStart=/usr/bin/dockerd $OPTIONS \
          $DOCKER_STORAGE_OPTIONS \
          $DOCKER_NETWORK_OPTIONS \
          $BLOCK_REGISTRY \
          $INSECURE_REGISTRY

Text the same as in docs… I just don’t understand what wrong.