testcontainers-java: Intermittent failure in CI environment

I have a simple Redis integration test:

public class RedisResourceTest {

 private static final String DOCKER_IMAGE = "redis:3.2.9";
   private static final int TIMEOUT = 120;

  @ClassRule
  public static GenericContainer redis = new GenericContainer(DOCKER_IMAGE).withExposedPorts(6379)
      .withStartupTimeout(Duration.ofSeconds(TIMEOUT));

  @Test
  public void testResourceCanSendAndRecieve() throws ResourceInitializationException {
    String ipAddredd = redis.getContainerIpAddress()),
    int redisPort = redis.getMappedPort(6379)))};
   
    //TEST code  
  
  }
}

This passes every time running locally, however, when running in Jenkins (BlueOcean) it intermittently fails with the exception:

java.lang.AssertionError: A port exposed by a docker container should be accessible: 'Connection refused (Connection refused)' does not equal expected 'hello'
	at org.rnorth.visibleassertions.VisibleAssertions.fail(VisibleAssertions.java:440)
	at org.rnorth.visibleassertions.VisibleAssertions.assertEquals(VisibleAssertions.java:174)
	at org.testcontainers.DockerClientFactory.checkExposedPort(DockerClientFactory.java:190)
	at org.testcontainers.DockerClientFactory.lambda$client$2(DockerClientFactory.java:125)
	at org.testcontainers.DockerClientFactory.runInsideDocker(DockerClientFactory.java:226)
	at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:114)
	at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:116)
	at io.committed.RedisResourceTest.<clinit>(RedisResourceTest.java:27)
	at sun.misc.Unsafe.ensureClassInitialized(Native Method)
	at sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
	at sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:156)
	at java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088)
	at java.lang.reflect.Field.getFieldAccessor(Field.java:1069)
	at java.lang.reflect.Field.get(Field.java:393)
	at org.junit.runners.model.FrameworkField.get(FrameworkField.java:73)
	at org.junit.runners.model.TestClass.getAnnotatedFieldValues(TestClass.java:230)
	at org.junit.runners.ParentRunner.classRules(ParentRunner.java:255)
	at org.junit.runners.ParentRunner.withClassRules(ParentRunner.java:244)
	at org.junit.runners.ParentRunner.classBlock(ParentRunner.java:194)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:362)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

I’ve tried larger timeout values, but that doesn’t seem to help. The docker sock port is exposed to the machine, the logs report lots of activity and the fact it sometimes passes must imply it’s not simply that it can’t interact with Docker.

I realize that this must be something to do with the environment, maybe as the build is already run in a Docker Container, so I don’t expect you to have a fix for this, however, my question is, how do I start to debug such an issue? Maybe there is a way to see the logs from the container?

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 32 (25 by maintainers)

Commits related to this issue

Most upvoted comments

Yes, I’ll make a PR. I’ll keep you updated.

@agibalov 1.4.3 is old, the latest is 1.6.0, please update

I created a shell script on the Jenkins Cloud Slave which executed, in for loop for like 1.000.000 times, the ‘ip route’ command and it always returned a valid value. So I don’t think it’s related to ECS-native networking.

I found out the LogToStringContainerCallback class doesn’t receive the output of the Docker container. I’m afraid there’s something wrong with the capturing of the stdout messages (probably in combination with ECS)