moby: Docker for Mac: com.docker.docker/Data/*00000003.00000948: connect: connection refused

Output of docker version:

Client:
 Version:      1.12.0-rc2
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   906eacd
 Built:        Fri Jun 17 20:35:33 2016
 OS/Arch:      darwin/amd64
 Experimental: true

Server:
 Version:      1.12.0-rc2
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   a7119de
 Built:        Fri Jun 17 22:09:20 2016
 OS/Arch:      linux/amd64
 Experimental: true

Output of docker info:

Containers: 1
 Running: 0
 Paused: 0
 Stopped: 1
Images: 1
Server Version: 1.12.0-rc2
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 22
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: host null bridge overlay
Swarm: inactive
Runtimes: default
Default Runtime: default
Security Options: seccomp
Kernel Version: 4.4.13-moby
Operating System: Alpine Linux v3.4
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.788 GiB
Name: moby
ID: XW4O:XEF3:TSZX:XJHQ:W7QB:675K:S5YA:FJ7E:P46X:BBFM:YN53:P5HL
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 21
 Goroutines: 29
 System Time: 2016-06-28T15:22:19.160120135Z
 EventsListeners: 1
No Proxy: *.local, 169.254/16
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
 127.0.0.0/8

Additional environment details (AWS, VirtualBox, physical, etc.):

Steps to reproduce the issue:

  1. docker pull choerst/centoscppbuild:latest
  2. ‘docker run --rm -it -v “$PWD”:/workspace -w /workspace/project choerst/centoscppbuild:latest ant’

Describe the results you received: I received an error message stating the following:

Error response from daemon: dial unix /Users/johker/Library/Containers/com.docker.docker/Data/*00000003.00000948: connect: connection refused

Describe the results you expected: I expected the ant build to complete without any issues. It worked well when using virtualbox via docker toolbox, but now after switching to Docker for Mac, I experience issues.

Additional information you deem important (e.g. issue happens only occasionally): Docker is set to use 4 CPUs and 8GB of memory. The ant build first builds a large C++ project by calling make, then runs rpmbuild to package the project as an RPM.

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Reactions: 17
  • Comments: 42 (15 by maintainers)

Most upvoted comments

Is there a solution to this issue yet?

@warik your crashes look like those that @tailhook identified in docker/hyperkit#50, thanks for posting your diagnostic.

Other passersby, your issue may not have docker/hyperkit#50 as its root cause. It is best to upload your diagnostics and report the diagnostic ID or confirm yourself that the errors in your logs look like the error in docker/hyperkit#50. The high level symptom could be caused by any number of underlying problems.

this is a huge issue for us - (one of many at the moment!)

we have a container running Ubuntu 16.04 LTS and Nginx where code is fetch from mounted volume (mounted via -v /a:/b)

Version 1.12.1 (build: 12133)
2d5b4d9c3daa089e3869e6355a47dd96dbf39856
Error response from daemon: dial unix /Users/user/Library/Containers/com.docker.docker/Data/*00000003.00000948: connect: connection refused

Update:

I’ve seen this before with VBox and VMWare on Mac where multiple CPUs allocated cause a deadlock. Reasoning still a bit unclear, but tested out the theory and appears to hold true.

Solution - drop to 1 CPU core. Not great, but appears to be solve the problem.

screen shot 2016-09-28 at 15 43 55

@jaywhy13 yours is the closest I’ve seen to a workable reproduction. I created Dockerfile and docker-compose.yml with the content you posted and ran docker-compose up. However when I visit localhost:8000 all I get is a The requested resource / was not found on this server.. I expect this is because the Dockerfile has COPY . /code but I have no code. If you could provide a basic skeleton which reproduces the issue for you with 1.12.1-beta27 then that would be very useful.

I’m also unsure what you mean by and perform the installation but maybe that would be clear if I had some content? In any case if you could walk me through something which reproduces it step by step I’d be grateful (I know nothing about wordpress I’m afraid).

We have just released v1.21.1-beta27 which includes some upgrades and improvements to the vsock transport code. Note that when switching between beta and stable you need to full uninstall, see Q: Can I switch back and forth between stable and beta versions of Docker for Mac?

Please could everyone test with that and if you are still seeing issues open a fresh ticket against https://github.com/docker/for-mac with the full output of diagnostics (from the whale menu). Providing reproduction steps of any case would also be very useful in tracking this down.

I understand that many of the repro cases are when building proprietary software but if someone was able to par their case down to just the basic build system skeleton required to trigger the issue (removing all the proprietary stuff) or find a subset of their code which they were willing/able to post then we’ll stand a much better chance of nailing this once and for all.

edited to add pointer to instructions for switching between stable and beta

@gluxon can you provide instructions how to reproduce the patch? I.e. how to build hyperkit from sources and replace it.

That binary patch is actually not the result of compiling hyperkit from sources myself. I created it from disassembling the source of com.docker.hyperkit and finding the assembly instruction that correlated with if (sc->reply_prod == VTSOCK_REPLYRINGSZ). The resulting opcode was 81f900020000 and I changed that to 81f900040000.

If you would prefer to manually change the file yourself with a hex editor instead of applying the patch using the bspatch utility in homebrew, you can find the opcode above at offset 000000010001fb71.

I’d just like to reiterate what @dsheets said above (https://github.com/docker/docker/issues/24045#issuecomment-237901919), from the information posted here so far only @warik, @gluxon and @tailhook appear to be suffering from docker/hyperkit#50.

For everyone else (at least @johker-wst, @davidgibbons, @jaronoff97, @theartoflogic and @vielmetti at this point) there is not enough information to conclude one way or another if you are seeing the same issue, the com.docker.docker/Data/*00000003.00000948: connect: connection refused symptom is common to any number of underlying failures (essentially failure to communicate for any reason will show up like this). To reiterate what @dsheets says upthread (https://github.com/docker/docker/issues/24045#issuecomment-237823247):

It is very hard to diagnose this issue without either or both of:

  • Reproduction
  • Docker for Mac diagnostic ID

We currently have a UI issue (internal docker/pinata#4551) with the diagnostic upload process so you will need to post your diagnostic ID and at least click the Open Issues button in the Diagnose & Feedback window. You needn’t open a new issue but, currently, diagnostic upload is linked to that button.

IMHO it would probably be best/least confusing if you were to each create a fresh issue in the docker/for-mac issue tracker (by following the Docker for Mac UI under Diagnose & Feedback) with your individual diagnostic ID and your reproduction steps. It is much easier to reconcile multiple duplicate tickets than it is to split a single ticker which contains multiple issues (even more so when those issues are superficially similar but turn out to have very different root causes).

Thanks, Ian.

It is very hard to diagnose this issue without either or both of:

  1. Reproduction
  2. Docker for Mac diagnostic ID

We currently have a UI issue (internal docker/pinata#4551) with the diagnostic upload process so you will need to post your diagnostic ID and at least click the Open Issues button in the Diagnose & Feedback window. You needn’t open a new issue but, currently, diagnostic upload is linked to that button.

It is quite likely that there are multiple different underlying causes for the defects reported in this issue. Diagnostic upload and posting of IDs will help us to understand exactly which components are ultimately at fault (many scary log messages are downstream failure cascades from the source).

Thanks!

I just saw this error on my Docker for Mac.

Exception in thread Thread-2:
Traceback (most recent call last):
  File "threading.py", line 810, in __bootstrap_inner
  File "threading.py", line 763, in run
  File "compose/cli/log_printer.py", line 149, in tail_container_logs
  File "compose/cli/log_printer.py", line 179, in wait_on_exit
  File "compose/container.py", line 239, in wait
  File "site-packages/docker/utils/decorators.py", line 21, in wrapped
  File "site-packages/docker/api/container.py", line 441, in wait
  File "site-packages/docker/client.py", line 174, in _raise_for_status
APIError: 500 Server Error: Internal Server Error ("dial unix /Users/emv/Library/Containers/com.docker.docker/Data/*00000003.00000948: connect: connection refused")

I don’t know if this does anything more than say “also crashed for me”, this is a first time crash in this particular environment.