testcontainers-java: "Can not connect to Ryuk" on macOS with Docker for Mac 2.4.0.0

I’ve seen this exact error come and go a fair bit on here, but it’s just today started happening for me.

I’m using testcontainers-java 1.12.0, but updating to the latest version has no impact at all.

When running my test, I get the following:

2020-08-28 17:19:18.778,INFO ,[main]:,o.t.d.DockerClientProviderStrategy,,,,,Loaded org.testcontainers.dockerclient.UnixSocketClientProviderStrategy from ~/.testcontainers.properties, will try it first
2020-08-28 17:19:18.833,INFO ,[main]:,o.t.d.DockerClientProviderStrategy,,,,,Will use 'okhttp' transport
2020-08-28 17:19:19.490,INFO ,[main]:,o.t.d.UnixSocketClientProviderStrategy,,,,,Accessing docker with local Unix socket
2020-08-28 17:19:19.491,INFO ,[main]:,o.t.d.DockerClientProviderStrategy,,,,,Found Docker environment with local Unix socket (unix:///var/run/docker.sock)
2020-08-28 17:19:19.492,INFO ,[main]:,o.t.DockerClientFactory,,,,,Docker host IP address is localhost
2020-08-28 17:19:19.680,INFO ,[main]:,o.t.DockerClientFactory,,,,,Connected to docker:
  Server Version: 19.03.13-beta2
  API Version: 1.40
  Operating System: Docker Desktop
  Total Memory: 2996 MB
2020-08-28 17:19:19.988,INFO ,[main]:,o.t.u.RegistryAuthLocator,,,,,Credential helper/store (docker-credential-desktop) does not have credentials for quay.io

And then a seemingly infinite amount of the following:

2020-08-28 17:19:20.733,WARN ,[testcontainers-ryuk]:,o.t.u.ResourceReaper,,,,,Can not connect to Ryuk at localhost:32781
java.net.ConnectException: Connection refused (Connection refused)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at java.net.Socket.connect(Socket.java:538)
	at java.net.Socket.<init>(Socket.java:434)
	at java.net.Socket.<init>(Socket.java:211)
	at org.testcontainers.utility.ResourceReaper.lambda$start$1(ResourceReaper.java:113)
	at java.lang.Thread.run(Thread.java:748)

This is running with Java 1.8.0_222 from Amazon Corretto, running on macOS 10.15.6.

This definitely worked yesterday. I did have an update to the Docker Engine today, and I’m assuming that’s relevant, but everything else I’m doing with Docker is all working fine - including a different project that uses the Rust version of TestContainers. (Though I notice that doesn’t use Ryuk, which might be relevant)

Cheers

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 59
  • Comments: 39 (11 by maintainers)

Commits related to this issue

Most upvoted comments

Docker for Mac 2.4.0.0 was released yesterday, pushing this issue into the stable version.

We released 1.15.0-rc2 which includes a fix (#3159) for compatibility - please upgrade.

image Ok just to confirm the flick of a gRPC switch has worked for now until all the libraries come back to their synergy.

@vinay-thakkar you can:

  1. use Testcontainers 1.15.0-rc2 that includes the fix
  2. Disable gRPC FUSE in Docker for Mac 2.4.0
  3. Downgrade to Docker for Mac 2.3.x

I’m also experiencing the same issue

Docker for Mac 2.4.0.0 was released yesterday, pushing this issue into the stable version.

We released 1.15.0-rc2 which includes a fix (#3159) for compatibility - please upgrade.

No-one has asked me, but I confirm that 1.15.0-rc2 works 😉

Same here: 1.15.0-rc2 solves the problem. Thanks for the fast reaction 👍

Thanks @rnorth it does indeed fix one issue it seems. However, I’m now facing a different issue that used to work before this.

I’m creating a PostgreSQLContainer based on a postgis image like this:

PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer("postgis/postgis:12-3.0-alpine")
                .withDatabaseName("panoramix_db")
                .withUsername("sa")
                .withPassword("sa");

Before I updated my Docker Desktop this used to work but now I’m getting this error:

Caused by: java.lang.IllegalStateException: Failed to verify that image 'postgis/postgis:12-3.0-alpine' is a compatible substitute for 'postgres'. This generally means that you are trying to use an image that Testcontainers has not been designed to use. If this is deliberate, and if you are confident that the image is compatible, you should declare compatibility in code using the `asCompatibleSubstituteFor` method. For example:
   DockerImageName myImage = DockerImageName.parse("postgis/postgis:12-3.0-alpine").asCompatibleSubstituteFor("postgres");

EDIT Figured it out. So to fix the above code snippet it would become something like this:

PostgreSQLContainer postgreSQLContainer =
                new PostgreSQLContainer(
                        DockerImageName.parse("postgis/postgis:12-3.0-alpine").asCompatibleSubstituteFor("postgres"))
                        .withDatabaseName("panoramix_db")
                        .withUsername("sa")
                        .withPassword("sa");

Since I am using spring boot, I updated the version in my pom.xml by overriding the testcontainers.version property.

Edit. This comment was incorrect. I was overriding a parent pom created by my own org. The testcontainers.version property is not part of spring boot.

@vinay-thakkar The same problem came to me when I updated the docker version. Does this problem occur only for the desktop?

Not sure about other env. For me it occurred only for ‘Docker for Mac’. Our circle CI pipelines are pinned to older versions of Docker so no idea there. And don’t have windows machine in team to confirm.

@schophil

When will 1.15.0 be released?

Very soon - we mainly want to get #3102 over the line and into the 1.15.0 final release, as part of mitigation for the upcoming Docker Hub rate limits (#3099)

What is the impact of not using ryuk?

Ryuk does final cleanup of containers that might have escaped being killed. Testcontainers code tries its best, but in case of a sudden JVM termination or a bug there’s a limit to what the Java code can do. Ryuk is external to the JVM process and provides a more surefire method of cleaning up. Having Ryuk enabled is something we strongly recommend.

For the recent Docker Desktop 2.4.x Release on macOS, see #3159.

@bsideup you were right, my mistake, sorry for the inconvenience. Thanks!

@astubbs we will close it when 1.15.0 is out 😃

@avaz perhaps you disabled Ryuk to workaround the issue and forgot to enable it back?

NoClassDefFoundError (com/fasterxml/jackson/annotation/JsonMerge)

@tonymurphy please make sure that you’re not forcing an old version of jackson-annotations.

Thanks @bsideup , much appreciated. Fixed the problem

I have the same issue.

Using 1.15.0-rc2 could be a workaround. It’s available in maven central.

https://mvnrepository.com/artifact/org.testcontainers/testcontainers/1.15.0-rc2

@sazzer Have you found a solution in the meantime? I have the same issue after updating Docker Engine today.

Edit: “Downgraded” from Docker Desktop Edge to Docker Desktop Stable and the issue has gone. Looks like a problem in combination with Docker 2.3.6.0.