jib: jib doesn't update images when included Gradle projects change

Description of the issue: gradle jibDockerBuild isn’t creating new Docker images when code in Gradle included builds changes.

For the time being, I’m calling gradle build jibDockerBuild to work around this but that’s counter-intuitive.

Expected behavior: When code in an included build changes and gradle jibDockerBuild is re-run, a new image should be built containing the latest code from both the project and its included builds.

Steps to reproduce: Assume a composite Gradle build where an app’s build.gradle declares a dependency on a library using

    implementation("foo:some-lib:1.0.0") { changing = true }

and its settings.gradle contains

includeBuild '../some-lib'

If I intentionally introduce a compile-time error in some-lib and run gradle jibDockerBuild on the app, the build fails, so I know the right library code is being compiled.
If I make a non-breaking change in some-lib, jibDockerBuild succeeds but the Docker image it constructs is identical to the one before the change, it just has a new LastTagTime.
If I make a change to both the app and to the library and run gradle jibDockerBuild, a new image is built, but it still executes old some-lib code.
If I open a shell, I can see some-lib.jar in /apps/libs, which isn’t getting updated.
If I run gradle build before gradle jibDockerBuild, the library is updated.

Environment: macOS, JDK8, Gradle

jib-gradle-plugin Configuration:

jib {
    from {
        image = "gcr.io/distroless/java:debug"
    }
    to {
        image = "foo/bar"
    }
    container {
        jvmFlags = ["-Duser.timezone=UTC"]
        ports = ["80"]
    }
}

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 16 (9 by maintainers)

Most upvoted comments

I think I found the culprit, and it’s not Spring.

Our some-libs use the Java Library plugin instead of the JAR plugin to reduce compile classpath sizes and to avoid inadvertent transient dependencies, as recommended by Gradle. During compilation, If i’m reading this right and judging by the console output, JARs aren’t built when using the Java Library plugin … “consumers will use the output classes directory of this project directly on their compile classpath, instead of the jar file if the project uses the Java plugin”. I assume “use” means reference, not actually copy.

My guess is that jibDockerBuild isn’t seeing the updated .class files of the included builds during the “Building classes layer…” step and creating stale images as a result. But it is seeing the updated JAR files of the included builds that are created by a build or assemble task during the Building dependencies layer... step and updating the image correctly.

If that’s right, this could be a bigger issue, because it would affect all projects using Java Library plugins, irrespective of included builds.

emersonf on Jun 6, 2019

I have a similar issue to #1776, where running jibDockerBuild fails because it can’t find the jar files for sub-project dependencies. I tried the solution suggested by @loosebazooka above and could only get the build to pass when I added a dependency on tasks.assemble. Depending on tasks.jar was not sufficient.

From the logs, it seems the reason is that with the java-library plugin tasks.jar only builds the current project’s jar file and does not create any dependencies on the jar tasks of sub-project dependencies. Whereas tasks.assemble does create a dependency on the jar tasks of all sub-project dependencies.

For reference, this is what I ended up adding in my build.gradle (modified from @loosebazooka’s suggestion in order to work with gradle’s new task configuration avoidance API):

tasks.matching {
    it.name.startsWith('jib')
}.configureEach {
    it.dependsOn tasks.named('assemble')
}

garyp on Sep 19, 2019

@emersonf, right, I was just curious why you were using it. We sometimes encounter users making confusion decisions, it’s a little presumptuous of me to make that assumption wholesale, but we usually need to make sure people know what they’re doing before investing time on strange issues.

It turns out we expect gradle to behave predictably and it doesn’t seem to be doing so here: For whatever reason gradle is not resolving substituted dependencies as projectDependencies but instead still as externalModuleDependencies when we read them, so our logic to handle projectDependencies isn’t kicking in. I don’t know at what point it does make this substitution. (<- turns out this is irrelevant except for skaffold intergration) I will take a deeper look today.

loosebazooka on May 30, 2019