moby: docker.service Failed with result 'start-limit-hit'.

Provisioning a new node with docker-machine on Digital Ocean.

New node created OK Docker installed OK Starting Docker FAILED.

Docker-machine error output:

Error creating machine: Error running provisioning: ssh command error:
command : sudo systemctl -f start docker
err     : exit status 1
output  : Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.

Docker-machine v 0.10.0, build 76ed2a6 Docker installed on the node version Docker version 17.06.0-ce, build 02c1d87

systemctl status docker.service log

docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─10-machine.conf
   Active: failed (Result: start-limit-hit) since Mon 2017-07-03 17:48:03 UTC; 3min 38s ago
     Docs: https://docs.docker.com
  Process: 5803 ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver aufs --tlsverify --tlscacert /etc/dock
 Main PID: 5803 (code=exited, status=1/FAILURE)

Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Failed to start Docker Application Container Engine.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Unit entered failed state.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Failed with result 'exit-code'.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Stopped Docker Application Container Engine.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Start request repeated too quickly.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Failed to start Docker Application Container Engine.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Unit entered failed state.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Failed with result 'start-limit-hit'.

journalctl -xe log


Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Stopped Docker Application Container Engine.
-- Subject: Unit docker.service has finished shutting down
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.service has finished shutting down.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Closed Docker Socket for the API.
-- Subject: Unit docker.socket has finished shutting down
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.socket has finished shutting down.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Stopping Docker Socket for the API.
-- Subject: Unit docker.socket has begun shutting down
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.socket has begun shutting down.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Starting Docker Socket for the API.
-- Subject: Unit docker.socket has begun start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.socket has begun starting up.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Listening on Docker Socket for the API.
-- Subject: Unit docker.socket has finished start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.socket has finished starting up.
--
-- The start-up result is done.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Start request repeated too quickly.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.service has failed.
--
-- The result is failed.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.socket: Unit entered failed state.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Unit entered failed state.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Failed with result 'start-limit-hit'.

I have previously installed v17.03.0-ce v17.03.1-ce v17.04.0-ce v17.05.0-ce using the same method. This is the first time I install 17.06.0-ce, and first time encounter this issue.

Installation script

docker-machine create --driver digitalocean \
--digitalocean-image  ubuntu-16-10-x64 \
--digitalocean-region nyc1 \
--digitalocean-size 4gb \
--digitalocean-access-token $DOTOKEN $NODE_NAME

UPDATE: July 3, 2017 Downgrading to 17.05.0~ce-0~ubuntu-yakkety allows me to restart docker.

About this issue

Original URL
State: closed
Created 7 years ago
Reactions: 19
Comments: 61 (19 by maintainers)

Links to this issue

Most upvoted comments

I had the same error message once because of an empty /etc/docker/daemon.json file. Delete it if you don’t use it!

+125

SeWieland on Jul 12, 2018

On systemd systems, it’s bad practice to modify directly the service file As reported in control-where-the-docker-daemon-listens-for-connections:

Systemd vs daemon.json

Configuring Docker to listen for connections using both the systemd unit file and the daemon.json file causes a conflict that prevents Docker from starting.

Solution: sudo systemctl edit docker.service

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H unix:// -H tcp://127.0.0.1:2375

This will automatically create a file /etc/systemd/system/docker.service.d/override.conf

then reload and restart docker:

sudo systemctl daemon-reload
sudo systemctl restart docker.service

+37

audioscavenger on Jan 13, 2019

I got this error with a /etc/docker/daemon.json file that seemed to have a hard-coded local network IP. When the laptop was connected to a different network, docker failed to start. Removing this file solved the problem.

+14

eloff on Aug 31, 2018

Closing, because this is an issue with docker machine, not docker itself, and should be resolved. Feel free to continue the conversation after I closed

+13

thaJeztah on Jul 4, 2017

@thaJeztah thx a lot. Success. Drop-in file (/etc/systemd/system/docker.service.d/docker.conf) should have ExecStart=/usr/bin/dockerd ...... as command, not ExecStart=/usr/bin/docker daemon ......

+10

AmplifyDamage on Jul 5, 2017

have the same issue after upgrade systemd from 219-30 to systemd-219-67.el7_7.1.x86_64(reboot machine). the docker start error log:

systemd: Starting Docker Application Container Engine...
systemd: Failed at step LIMITS spawning /usr/bin/dockerd: Operation not permitted
systemd: docker.service: main process exited, code=exited, status=205/LIMITS
systemd: Failed to start Docker Application Container Engine.
systemd: Unit docker.service entered failed state.
systemd: docker.service failed.
systemd: docker.service holdoff time over, scheduling restart.
systemd: Stopped Docker Application Container Engine.
systemd: Starting Docker Application Container Engine...
systemd: Failed at step LIMITS spawning /usr/bin/dockerd: Operation not permitted
systemd: docker.service: main process exited, code=exited, status=205/LIMITS
systemd: Failed to start Docker Application Container Engine.
systemd: Unit docker.service entered failed state.
systemd: docker.service failed.

systemctl status docker

 systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Thu 2020-01-09 16:43:36 CST; 8min ago
     Docs: https://docs.docker.com
  Process: 160921 ExecStart=/usr/bin/dockerd --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} (code=exited, status=205/LIMITS)
 Main PID: 160921 (code=exited, status=205/LIMITS)

docker version: 18.06.0-ce

after comment the value in /usr/lib/systemd/system/docker.service ,the docker can start success. why? #LimitNOFILE=infinity #LimitNPROC=infinity #LimitCORE=infinity

zhxqgithub on Jan 9, 2020

FYI, after upgrading packages on one of our Ubuntu16.04 nodes, we also experienced this behavior … our ExecStart command within /lib/systemd/system/docker.service looked like this:

ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --iptables=true --dns <removed> --dns 8.8.8.8 --dns 8.8.4.4

but Docker refused to start, throwing this error:

Nov 19 09:45:13 ts-build1 systemd[1]: Failed to start Docker Application Container Engine.
Nov 19 09:45:13 ts-build1 systemd[1]: docker.service: Unit entered failed state.
Nov 19 09:45:13 ts-build1 systemd[1]: docker.service: Failed with result 'exit-code'.
Nov 19 09:45:15 ts-build1 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Nov 19 09:45:15 ts-build1 systemd[1]: Stopped Docker Application Container Engine.
Nov 19 09:45:15 ts-build1 systemd[1]: docker.service: Start request repeated too quickly.
Nov 19 09:45:15 ts-build1 systemd[1]: Failed to start Docker Application Container Engine.
Nov 19 09:45:15 ts-build1 systemd[1]: docker.service: Unit entered failed state.
Nov 19 09:45:15 ts-build1 systemd[1]: docker.service: Failed with result 'start-limit-hit'.

Found removing the -H fd:// parameter allowed Docker to start …

ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --iptables=true --dns <removed> --dns 8.8.8.8 --dns 8.8.4.4

N00bSkywalker on Nov 19, 2018

@AmplifyDamage check your systemd unit-file; it should have ExecStart=/usr/bin/dockerd ...... as command, not ExecStart=/usr/bin/docker daemon ......

thaJeztah on Jul 5, 2017

@debragail Likely fixing your systemd service config, but check the logs to see whats up… e.g. journalctl -fu docker

cpuguy83 on Nov 30, 2018

I apologize, it was my fault. I made a typo You’re correct . The issue is with docker-machine, it uploads 10-machine.conf to

/etc/systemd/system/docker.service.d

and inside it the line is

ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2376 ...

and it should be

ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 ...

@thaJeztah should this issue be referenced on docker-machine forums? What’s the best way to communicate here?

rommik on Jul 7, 2017

I had the same error because of conflict between /etc/docker/daemon.json and /etc/systemd/system/docker.service.d/options.conf

sudo journalctl --no-hostname --no-pager -b -u docker.service

Dec 31 11:45:11 dockerd[1918]: unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: iptables: (from flag: false, from file: false)
Dec 31 11:45:11 systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Dec 31 11:45:11 systemd[1]: docker.service: Failed with result 'exit-code'.
Dec 31 11:45:11 systemd[1]: Failed to start Docker Application Container Engine.

removing the option in one of the files fixed the problem

benaich on Dec 31, 2019

@AmplifyDamage what error message do you get? If you get the same error message, be sure to check you don’t have a drop-in / override file somewhere

thaJeztah on Jul 5, 2017

I had the same error message once because of an empty /etc/docker/daemon.json file. Delete it if you don’t use it!

I had extra comma at the end of json, causing problem. Its working now.

rkdrepo on Sep 14, 2020

i changed from ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock

Be really really careful when using -H tcp://0.0.0.0:2375. That configuration exposes the docker remote API on the public interface of your host, and without access control / not secured with TLS. If the machine you’re running on is accessible from the internet, that port gives unrestricted root access on your host. See Docker daemon attack surface in the documentation.

thaJeztah on May 18, 2020

@AmplifyDamage check your systemd unit-file; it should have ExecStart=/usr/bin/dockerd ...... as command, not ExecStart=/usr/bin/docker daemon ......

Thanks this helped . i was struggling since 2 days. actually i was trying test connection in jenkins but was giving error

Connection refused: /IP:2375 java.net.ConnectException: Connection refused Caused: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /IP:2375 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:714) at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:352)

i changed from ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock

to ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock then sudo systemctl daemon-reload sudo systemctl restart docker

Incase anybody faces same issues . this might help

ashishkarnad on May 18, 2020

Solved. This problem came out after I modified daemon.json. I just removed daemon.json and reboot machine, then docker auto started.

xiaozhoz on Jan 8, 2020

Same issue as https://github.com/moby/moby/issues/33931#issuecomment-439934007 and same fix (removing -H fd:// ). I’m on Ubuntu 16.04 and edit /etc/systemd/system/docker.service.d/docker.service.conf from ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS to ExecStart=/usr/bin/dockerd $DOCKER_OPTS

chuongnh3atgmaildotcom on Nov 29, 2018

@thaJeztah I updated my docker-machine as recommended, and I still have this issue. My server is new without prior usage.

lsb_release -a

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.10
Release:        16.10
Codename:       yakkety

Is it possible docker installation is wrong for Ubuntu?

Update confirmed /etc/systemd/system/docker.service.d/docker.conf contains ExecStart=/usr/bin/docker daemon and not ExecStart=/usr/bin/dockerd . Why is it the case for a fresh installation?

rommik on Jul 7, 2017