gradle-docker-plugin: DockerStartContainer problem

I am trying to follow the example in the user guide to emulate docker build + docker run (as suggested in #857) but the Dockerfile ENTRYPOINT is not run, and there is no indication that anything is wrong.

Expected Behavior

I expect the container to run.

Current Behavior

The container does not run. And there is no indication that anything went wrong.

Context

Your Environment

macOS 10.14.6, Docker version 19.03.2, build 6a30dfc.

Steps to Reproduce (for bugs)

The following transcript shows that docker build + docker run behaves as expected, but the corresponding Gradle tasks just quietly does not run the container entry point.

My files:

-bash$ find . '(' -name gradle -prune -or -not -path '*/\.*' -not -name gradlew -not -name gradlew.bat -type f -exec printf '###### START %s:\n' '{}' \; -exec cat '{}' \; -exec printf '###### END %s\n\n' '{}' \; ')'
###### START ./Dockerfile:
FROM alpine:latest
ENTRYPOINT ["/bin/sh", "-c", "echo 'The ENTRYPOINT is faking a test-error' ; echo 'Test results' > /tmp/my_test_outputs/test_outputs.txt; ls -l /tmp/my_test_outputs/; exit 37"]

###### END ./Dockerfile

###### START ./build.gradle:
import com.bmuschko.gradle.docker.tasks.container.DockerCreateContainer
import com.bmuschko.gradle.docker.tasks.container.DockerStartContainer
import com.bmuschko.gradle.docker.tasks.container.DockerStopContainer
import com.bmuschko.gradle.docker.tasks.image.DockerBuildImage

plugins {
    id 'base'
    id 'com.bmuschko.docker-remote-api' version '4.10.0'
}

task buildTestImage(type: DockerBuildImage) {
    inputDir.set(file("$projectDir"))
    dockerFile.set(file("Dockerfile"))

    tags.add('my-unit-tests:latest')
}

task createTestContainer(type: DockerCreateContainer) {
    dependsOn buildTestImage
    targetImageId buildTestImage.getImageId()
    
    autoRemove.set(true)

    volumes.add("$buildDir:/tmp/my_test_output")
}

task startTestContainer(type: DockerStartContainer) {
    dependsOn createTestContainer
    targetContainerId createTestContainer.getContainerId()
}

task stopTestContainer(type: DockerStopContainer) {
    dependsOn createTestContainer
    targetContainerId createTestContainer.getContainerId()
}

task myTest {
    dependsOn startTestContainer
    finalizedBy stopTestContainer
}


###### END ./build.gradle

###### START ./settings.gradle:
rootProject.name = 'docker-test'
###### END ./settings.gradle

Using docker build + docker run manages to run the ENTRYPOINT and pass the exit code back to the host:

-bash$ rm -rf build && docker build -t my-unit-tests-manual:latest . && docker run --rm -v $PWD/build:/tmp/my_test_outputs my-unit-tests-manual; echo "Docker run exited with $?"; echo 'The contents of ./build/ is:'; ls -l ./build/
Sending build context to Docker daemon  149.5kB
Step 1/2 : FROM alpine:latest
 ---> 961769676411
Step 2/2 : ENTRYPOINT ["/bin/sh", "-c", "echo 'The ENTRYPOINT is faking a test-error' ; echo 'Test results' > /tmp/my_test_outputs/test_outputs.txt; ls -l /tmp/my_test_outputs/; exit 37"]
 ---> Running in ebdb926e289b
Removing intermediate container ebdb926e289b
 ---> c56474213903
Successfully built c56474213903
Successfully tagged my-unit-tests-manual:latest
The ENTRYPOINT is faking a test-error
total 4
-rw-r--r--    1 root     root            13 Sep 19 13:51 test_outputs.txt
Docker run exited with 37
The contents of ./build/ is:
total 8
-rw-r--r--  1 perm  staff  13 Sep 19 15:51 test_outputs.txt

Trying to do the same things with gradle, following the example in the user guide, quietly does not run the container ENTRYPOINT:

-bash$ rm -rf build && ./gradlew myTest; echo "Gradle exited with $?"; echo 'The contents of ./build/ is:'; ls -la ./build/.docker/

> Task :buildTestImage
Building image using context '/Users/perm/docker-test'.
Using Dockerfile '/Users/perm/docker-test/Dockerfile'
Using tags 'my-unit-tests:latest' for image.
Step 1/2 : FROM alpine:latest
 ---> 961769676411
Step 2/2 : ENTRYPOINT ["/bin/sh", "-c", "echo 'The ENTRYPOINT is faking a test-error' ; echo 'Test results' > /tmp/my_test_outputs/test_outputs.txt; ls -l /tmp/my_test_outputs/; exit 37"]
 ---> Using cache
 ---> c56474213903
Successfully built c56474213903
Successfully tagged my-unit-tests:latest
Created image with ID 'c56474213903'.

> Task :createTestContainer
Created container with ID 'c1e9476bc7aa12cc63704c5eb88cf7964ea35cfa02968b627b2ce96747e6693b'.

> Task :startTestContainer
Starting container with ID 'c1e9476bc7aa12cc63704c5eb88cf7964ea35cfa02968b627b2ce96747e6693b'.

> Task :stopTestContainer
Stopping container with ID 'c1e9476bc7aa12cc63704c5eb88cf7964ea35cfa02968b627b2ce96747e6693b'.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.6.2/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 1s
4 actionable tasks: 4 executed
Gradle exited with 0
The contents of ./build/ is:
total 8
drwxr-xr-x  3 perm  staff  96 Sep 19 15:52 .
drwxr-xr-x  3 perm  staff  96 Sep 19 15:52 ..
-rw-r--r--  1 perm  staff  12 Sep 19 15:52 buildTestImage-imageId.txt
-bash$

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 24 (2 by maintainers)

Most upvoted comments

@PerMildner It might be not be apparent but almost all of the communication work from this plugin to the Docker Daemon is performed by the Docker Java library. We are not actively suppressing any error message. If we do, then it’s a bug in our plugin. The same goes for providing relevant input values. Most of the time, we are just calling off to the API of Docker Java with the same property names.

There are some differences between the behavior of the docker CLI and the Docker Daemon that we are aware of. If you feel that the docker CLI provide more convenient behavior or error messaging then there’s nothing speaking against calling docker directly with an Exec command.

Documentation is always something we could improve on. I’ve been trying to add more documentation and Javadocs over the past months. It’ll take a more concerted effort (also by contributors) to improve on it. Generally speaking, it would be very similar to the Javadocs in Docker Java classes e.g. CreateContainerCmd.java. If you have a deeper look at the Javadocs of Docker Java, you will find they are also not perfect when it comes down to the docs.

Thanks for raising those issues again. They have come up in the past. IMHO they mostly boil down to better documentation, in the user guide and the Javadocs. I am hoping that we can get some additional help from the community to improve on it. Reminder from my end: This is an open source project and the main committers are spending their free-time on it.

bmuschko on Sep 30, 2019

Yes, there are some remaining issues.

The main issue is that DockerStartContainer does not give an error when the binds parameter to the corresponding DockerCreateContainer cannot be used. As I showed above, it looks as if the Dockers daemon does give an error in this case (“Error response from daemon: Mounts denied”). Quietly suppressing errors is a usability problem.

Similarly, the volumes parameter could give an error when the volume name looks like an absolute path (are Docker volume names allowed to contain the / directory separator?) (*).

The other issue is that the documentation for DockerCreateContainer does not make it sufficiently clear that the volumes property is unrelated to the docker create --volume option. This is especially unfortunate since the docker command treats --volumes as more or less equivalent with its --mount option (but with different syntax). This is what caused me to initially use the wrong property with DockerCreateContainer (*).

Also, it does not help that the docker create option --mount corresponds to the DockerCreateContainer property with the different namebinds.

Looking again, DockerCreateContainer is completely undocumented, as far as I can tell. It is only present without comment (and without volumes or mounts) in an example in the user manual. The JavaDoc also has no relevant documentation.

(*) Fixing either, or both, of these two issues would have helped me avoid the problems that caused me to create this ticket.

PerMildner on Sep 30, 2019

@PerMildner Are you still experiencing problem? If not, let’s close this issue.

bmuschko on Sep 28, 2019

“why do you assume that DockerStartContainer should throw exception when exit code is non-zero?”

Because it is what docker run ... does (which, was what I really wanted to emulate). However, you are right, the (almost undocumented) docker start ... also does not report an error in this case, so this is OK.

“Regarding non-existing folder in volume, if Docker deamon doesn’t complain why should we?”

Because this is what docker start ... does, see below. It seems reasonable to expect DockerStartContainer to behave similarly to docker start. I do now know what the Docker daemon does. The transcript below indicate that the daemon does complain (“Error response from daemon: Mounts denied”), perhaps it is the Docker Java library that ignores the error. In either case the docker command line tool, not the daemon, is what normal users come in contact with, and in this case the docker command would give a helpful error.

bash$ docker build .
...
Successfully built f164d9a9d1ba
bash$ docker create f164d9a9d1ba
c870560e97861dcb1cc0b6eb10fb78e516742f7be8bac27f197acca62566218e
bash$ docker create -v '/this/folder/does/not/exist:/tmp/container_path' f164d9a9d1ba
48e1fb25b904948fea8652bb5a6f23b05b559abfce160aaa1a2aeffa72a3397b
bash$ docker start 48e1fb25b904948fea8652bb5a6f23b05b559abfce160aaa1a2aeffa72a3397b
Error response from daemon: Mounts denied: 
The path /this/folder/does/not/exist
is not shared from OS X and is not known to Docker.
You can configure shared paths from Docker -> Preferences... -> File Sharing.
See https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.
.
Error: failed to start containers: 48e1fb25b904948fea8652bb5a6f23b05b559abfce160aaa1a2aeffa72a3397b
bash$ cat Dockerfile
FROM alpine:latest
ENTRYPOINT ["/bin/sh", "-c", "echo 'The ENTRYPOINT is faking a test-error' ; echo 'Test results' > /tmp/my_test_outputs/test_outputs.txt; ls -l /tmp/my_test_outputs/ >> /tmp/my_test_outputs/test_outputs.txt; exit 37"]

bash$

PerMildner on Sep 26, 2019

@PerMildner why do you assume that DockerStartContainer should throw exception when exit code is non-zero? I guess you can handle it by yourself using onComplete handler.

Regarding non-existing folder in volume, if Docker deamon doesn’t complain why should we?

orzeh on Sep 26, 2019

@orzeh Thanks! Indeed I got confused by the fact that the DockerCreateContainer volumes property is unrelated to the docker --volume option. Also neither volumes nor bind is documented in the plugin user guide, as far as I can tell.

However (here he goes again), although my original analysis was incorrect (the ENTRYPOINT is run), two issues remains: 1. The non-zero exit code from the ENTRYPOINT is quietly ignored by DockerStartContainer instead of raising an exception, and 2. The non-existing source folder specified in the volumes of DockerCreateContainer is also ignored and does not cause any indication that something is wrong. Quietly ignoring erroneous input or error exits seems like a bad idea.

PerMildner on Sep 26, 2019

@PerMildner everything works as expected. In createTestContainer task you define volumes but you probably want to setup binds. This might be confusing, see the docs.

This worked for me:


task createTestContainer(type: DockerCreateContainer) {
    dependsOn buildTestImage
    targetImageId buildTestImage.getImageId()
    
    autoRemove.set(true)
    binds = ["$buildDir":"/tmp/my_test_outputs"]
}

Please note that file will be created in build directory.

I guess you can submit PR against our User Guide to cover everything you learned 😃

orzeh on Sep 25, 2019