orc: [JAVA] mvn package fails if test compiling was skipped

I would like to run mvn -Dmaven.test.skip=true clean package, but maven-dependency-plugin complains Unused declared dependencies for some libraries used by the test code, which breaks the compilation. Please check the attached logs.

I’m not a java expert, but I’m guessing the cause of the problem is the misuse of analyze-only in the package phase. According to the documentation, the analyze-only goal is meant to be used during the test-compile phase. In our case, the test class was not compiled, so the dependency analyzer treated some libraries as unused. I don’t know the right way to fix it though.

The setting of maven-dependency-plugin: https://github.com/apache/orc/blob/8cf1047f9ace3799df12f24d2a5096b17a9a6ed0/java/pom.xml#L373-L388

Logs:

$ cd orc/java

$ mvn -Dmaven.test.skip=true clean package
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] Apache ORC                                                         [pom]
[INFO] ORC Shims                                                          [jar]
[INFO] ORC Core                                                           [jar]
[INFO] ORC MapReduce                                                      [jar]
[INFO] ORC Tools                                                          [jar]
[INFO] ORC Examples                                                       [jar]
[INFO]
[INFO] -------------------------< org.apache.orc:orc >-------------------------
[INFO] Building Apache ORC 1.9.0-SNAPSHOT                                 [1/6]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.2.0:clean (default-clean) @ orc ---
[INFO] Deleting /Users/x/Documents/playground/orc/java/target
[INFO]
[INFO] --- maven-enforcer-plugin:3.1.0:enforce (enforce-maven-version) @ orc ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.1.0:enforce (enforce-java-version) @ orc ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.1.0:enforce (enforce-maven) @ orc ---
[INFO]
[INFO] --- maven-remote-resources-plugin:1.7.0:process (process-resource-bundles) @ orc ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- maven-antrun-plugin:3.1.0:run (setup-test-dirs) @ orc ---
[INFO] Executing tasks
[INFO]     [mkdir] Created dir: /Users/x/Documents/playground/orc/java/target/testing-tmp
[INFO] Executed tasks
[INFO]
[INFO] --- maven-site-plugin:3.12.0:attach-descriptor (attach-descriptor) @ orc ---
[INFO] No site descriptor found: nothing to attach.
[INFO]
[INFO] --- maven-source-plugin:3.2.1:jar-no-fork (create-source-jar) @ orc ---
[INFO]
[INFO] --- maven-source-plugin:3.2.1:test-jar-no-fork (create-source-jar) @ orc ---
[INFO]
[INFO] --- reproducible-build-maven-plugin:0.15:strip-jar (default) @ orc ---
[INFO]
[INFO] ----------------------< org.apache.orc:orc-shims >----------------------
[INFO] Building ORC Shims 1.9.0-SNAPSHOT                                  [2/6]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.2.0:clean (default-clean) @ orc-shims ---
[INFO] Deleting /Users/x/Documents/playground/orc/java/shims/target
[INFO]
[INFO] --- maven-enforcer-plugin:3.1.0:enforce (enforce-maven-version) @ orc-shims ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.1.0:enforce (enforce-java-version) @ orc-shims ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.1.0:enforce (enforce-maven) @ orc-shims ---
[INFO]
[INFO] --- build-helper-maven-plugin:3.3.0:add-source (add-source) @ orc-shims ---
[INFO] Source directory: /Users/x/Documents/playground/orc/java/shims/target/generated-sources added.
[INFO]
[INFO] --- maven-remote-resources-plugin:1.7.0:process (process-resource-bundles) @ orc-shims ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @ orc-shims ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] skip non existing resourceDirectory /Users/x/Documents/playground/orc/java/shims/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.10.1:compile (default-compile) @ orc-shims ---
[INFO] Compiling 13 source files to /Users/x/Documents/playground/orc/java/shims/target/classes
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) @ orc-shims ---
[INFO] Not copying test resources
[INFO]
[INFO] --- maven-antrun-plugin:3.1.0:run (setup-test-dirs) @ orc-shims ---
[INFO] Executing tasks
[INFO]     [mkdir] Created dir: /Users/x/Documents/playground/orc/java/shims/target/testing-tmp
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.10.1:testCompile (default-testCompile) @ orc-shims ---
[INFO] Not compiling test sources
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @ orc-shims ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:3.3.0:jar (default-jar) @ orc-shims ---
[INFO] Building jar: /Users/x/Documents/playground/orc/java/shims/target/orc-shims-1.9.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-site-plugin:3.12.0:attach-descriptor (attach-descriptor) @ orc-shims ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-source-plugin:3.2.1:jar-no-fork (create-source-jar) @ orc-shims ---
[INFO] Building jar: /Users/x/Documents/playground/orc/java/shims/target/orc-shims-1.9.0-SNAPSHOT-sources.jar
[INFO]
[INFO] --- maven-source-plugin:3.2.1:test-jar-no-fork (create-source-jar) @ orc-shims ---
[INFO] Building jar: /Users/x/Documents/playground/orc/java/shims/target/orc-shims-1.9.0-SNAPSHOT-test-sources.jar
[INFO]
[INFO] --- reproducible-build-maven-plugin:0.15:strip-jar (default) @ orc-shims ---
[INFO] Stripping /Users/x/Documents/playground/orc/java/shims/target/orc-shims-1.9.0-SNAPSHOT-test-sources.jar
[INFO] Stripping /Users/x/Documents/playground/orc/java/shims/target/orc-shims-1.9.0-SNAPSHOT-sources.jar
[INFO] Stripping /Users/x/Documents/playground/orc/java/shims/target/orc-shims-1.9.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-dependency-plugin:3.1.2:analyze-only (default) @ orc-shims ---
[WARNING] Unused declared dependencies found:
[WARNING]    org.junit.jupiter:junit-jupiter-api:jar:5.9.0:test
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Apache ORC 1.9.0-SNAPSHOT:
[INFO]
[INFO] Apache ORC ......................................... SUCCESS [  4.108 s]
[INFO] ORC Shims .......................................... FAILURE [  4.817 s]
[INFO] ORC Core ........................................... SKIPPED
[INFO] ORC MapReduce ...................................... SKIPPED
[INFO] ORC Tools .......................................... SKIPPED
[INFO] ORC Examples ....................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  9.189 s
[INFO] Finished at: 2022-10-13T15:07:06+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.1.2:analyze-only (default) on project orc-shims: Dependency problems found -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :orc-shims

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 20 (13 by maintainers)

Commits related to this issue

ORC-1294: Fix build error when skip tests build ### What changes were proposed in this pull request? This PR aims to fix build error when skip tests build. It has two modifications: - Add `ignoreNonC... — committed to apache/orc by deshanxiao 2 years ago
ORC-1294: Fix build error when skip tests build ### What changes were proposed in this pull request? This PR aims to fix build error when skip tests build. It has two modifications: - Add `ignoreNonC... — committed to apache/orc by deshanxiao 2 years ago

Most upvoted comments

Okay. Please make a PR. 😃

dongjoon-hyun on Oct 20, 2022

Agree. I think the purpose of introducing Apache Maven Dependency Plugin is to reduce some unused dependencies into the package. For Runtime/Provided/Test/System scope, it will not be packed into the final package. So the cost of introducing this tag looks like acceptable.

Parquet-MR: https://github.com/apache/parquet-mr/blob/62b774cd0f0c60cfbe540bbfa60bee15929af5d4/pom.xml#L554

deshanxiao on Oct 20, 2022

Ah, I deleted my previous post. Sorry for that. It seems that the drawback (Ignoring all Runtime/Provided/Test/System scope) is too big.

dongjoon-hyun on Oct 20, 2022

It’s great for discovering new dependency problems in this way. But the flag seems to entirely skip analyzing for the test scope (precisely non-compile scopes), so we cannot find unused test-only dependencies too. I don’t know if it’s acceptable.

zjx20 on Oct 20, 2022

Sorry about that, I didn’t test that and thought it should just work. It seems that mvn -Dmaven.test.skip=true is not executed the way I thought it would. The approach is not working and I have no more ideas.

zjx20 on Oct 17, 2022