Hi, we’re having a similar issue across all of our docker machines that were recently upgraded to 1.12. I’ll try to get get stack traces out tomorrow. docker ps and all docker compose commands now hang. Going to try to rollback to previous version of docker.

I have the same issue with 1.12.0 too.

“docker ps” hangs forever and “docker stop” etc. access via swarm hangs too.

The log is absolutely quiet about this.

strace docker ps

…snip… epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=144861200, u64=140024668645392}}) = 0 getsockname(3, {sa_family=AF_LOCAL, NULL}, [2]) = 0 getpeername(3, {sa_family=AF_LOCAL, sun_path=“/var/run/docker.sock”}, [23]) = 0 futex(0xc82002c908, FUTEX_WAKE, 1) = 1 read(3, 0xc82034e000, 4096) = -1 EAGAIN (Resource temporarily unavailable) write(3, “GET /v1.24/containers/json HTTP/”…, 95) = 95 epoll_wait(4, [], 128, 0) = 0 futex(0x1326ca8, FUTEX_WAIT, 0, NULL

docker info

Containers: 11 Running: 4 Paused: 0 Stopped: 7 Images: 149 Server Version: 1.12.0 Storage Driver: aufs Root Dir: /var/lib/docker/aufs Backing Filesystem: extfs Dirs: 849 Dirperm1 Supported: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: overlay null bridge host Swarm: inactive Runtimes: runc Default Runtime: runc Security Options: apparmor seccomp Kernel Version: 4.4.0-31-generic Operating System: Ubuntu 16.04.1 LTS OSType: linux Architecture: x86_64 CPUs: 8 Total Memory: 7.795 GiB Name: content-1 ID: MHM7:O6J2:ACPA:6ZPU:SQY4:WFCG:DV7O:YJ5K:TLLL:MH5N:2GXZ:QRR2 Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ Labels: nodetype=worker-content provider=generic Cluster Store: consul://172.17.0.1:8500 Cluster Advertise: 192.168.1.13:2376 Insecure Registries: 127.0.0.0/8

We appear to be encountering similar after moving to docker 1.12.2. Restarting docker appears to be the only way to recover.

We are using an lvm thinpool devicemapper setup leveraging xfs and unfortunately, overlay is not very stable on Centos 7.2 so making that move is not currently an option. Doing so would be quite disruptive to existing workloads as well.

Here is the stack trace from dockerd: https://gist.github.com/sakserv/6aa7c7a1a8eac147e27d8c060023a36d

Docker info (note that even docker info hangs in my case):

Containers: 105
 Running: 19
 Paused: 0
 Stopped: 86
Images: 108
Server Version: 1.12.2
Storage Driver: devicemapper
 Pool Name: vg01-docker--pool
 Pool Blocksize: 524.3 kB
 Base Device Size: 274.9 GB
 Backing Filesystem: xfs
 Data file:
 Metadata file:
 Data Space Used: 853.2 GB
 Data Space Total: 5.63 TB
 Data Space Available: 4.777 TB
 Metadata Space Used: 105.9 MB
 Metadata Space Total: 16.98 GB
 Metadata Space Available: 16.87 GB
 Thin Pool Minimum Free Space: 563 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Library Version: 1.02.107-RHEL7 (2015-12-01)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: host null overlay bridge
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 3.10.0-327.13.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 32
Total Memory: 251.6 GiB
Name: foo.example.com
ID: OSYP:WEPA:N2LF:KFTJ:BZNP:L3PT:LDNV:4OBJ:A4AM:CWFB:HHIK:WN4M
Docker Root Dir: /grid/0/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Insecure Registries:
 127.0.0.0/8

Docker client strace:

-snip-
etpeername(4, {sa_family=AF_LOCAL, sun_path="/var/run/docker.sock"}, [23]) = 0
futex(0xc82004a908, FUTEX_WAKE, 1)      = 1
read(4, 0xc820327000, 4096)             = -1 EAGAIN (Resource temporarily unavailable)
write(4, "GET /v1.24/info HTTP/1.1\r\nHost: "..., 84) = 84
epoll_wait(5, {}, 128, 0)               = 0
futex(0x132cca8, FUTEX_WAIT, 0, NULL

dockerd strace:

Process 850182 attached
wait4(850188,

docker-containerd (pid 850188) strace:

Process 850188 attached
futex(0xee44c8, FUTEX_WAIT, 0, NULL

I have the same issue. I’m not using “docker exec” at all. After running “/etc/init.d/docker start” I can’t run a single docker command, even “docker version” is hung.

Version is “1.12.1_rc1 (Gentoo)”.

Tried to downgrade to 1.11, same issue.

How could I get docker to output all of the logs to one file on the host?

To keep your “current” approach, you can bind-mount a directory on your host, and have the container write to that (-v /path/on/host/:/var/log/), that way, you don’t have to exec into the container to access the log files.

However, that also doesn’t use the docker logging features. To do that, make sure the container writes logs to stdout / stderr (which can be as simple as symlinking the file, if the process expects a file to write to; see for example the nginx Dockerfile). After that, you can use a suitable logging driver for your use case to, e.g., log to syslog, collect logs with GELF, or splunk (see the logging drivers section

(sorry all for the off topic - back to the issue at hand)