cloud-opensource-java: LinkageCheckerMain fails to check beam-sdks-java-extensions-sql-zetasql

suztomo-macbookpro44:beam suztomo$ ./gradlew -Ppublishing -PjavaLinkageArtifactIds=beam-sdks-java-extensions-sql-zetasql :checkJavaLinkage 
Configuration on demand is an incubating feature.

> Task :runners:apex:buildDependencyTree
See the report at: file:///Users/suztomo/beam/runners/apex/build/classes/java/main/org/apache/beam/runners/apex/dependency-tree

> Task :checkJavaLinkage
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Jan 03, 2020 10:23:32 PM com.google.cloud.tools.opensource.dependencies.DependencyGraphBuilder levelOrder
SEVERE: Infinite recursion resolving jaxen:jaxen:jar:1.1-beta-4 (compile). Likely cycle in [org.apache.beam:beam-sdks-java-extensions-sql-zetasql:jar:2.19.0-SNAPSHOT (compile), org.apache.beam:beam-sdks-java-extensions-sql:jar:2.19.0-SNAPSHOT (compile), com.alibaba:fastjson:jar:1.2.49 (compile), org.springframework:spring-webmvc:jar:4.3.7.RELEASE (provided?), org.codehaus.groovy:groovy-all:jar:2.4.9 (compile?), com.thoughtworks.xstream:xstream:jar:1.4.7 (compile?), dom4j:dom4j:jar:1.6.1 (compile?), jaxen:jaxen:jar:1.1-beta-6 (compile?), dom4j:dom4j:jar:1.5.2 (compile), jaxen:jaxen:jar:1.1-beta-4 (compile), dom4j:dom4j:jar:1.5.2 (compile)]

...

org.apache.beam:beam-sdks-java-extensions-sql-zetasql:jar:2.19.0-SNAPSHOT (compile)
 / org.apache.beam:beam-sdks-java-extensions-sql:jar:2.19.0-SNAPSHOT (compile)
 / org.apache.beam:beam-sdks-java-io-parquet:jar:2.19.0-SNAPSHOT (provided)
 / org.apache.parquet:parquet-avro:jar:1.10.0 (compile)
 / org.apache.parquet:parquet-hadoop:jar:1.10.0 (compile)
 / org.apache.hadoop:hadoop-client:jar:2.7.3 (provided)
 / org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.3 (compile)
 / org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.3 (compile)
 / org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.3 (compile)
 / org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.3 (compile)
 / org.apache.hadoop:hadoop-yarn-common:jar:2.7.3 (compile)
 / org.apache.hadoop:hadoop-yarn-api:jar:2.7.3 (compile)
 / org.apache.hadoop:hadoop-common:jar:2.7.3 (provided)
 / org.apache.hadoop:hadoop-auth:jar:2.7.3 (compile)
 / org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15 (compile)
 / net.sf.ehcache:ehcache-core:jar:2.4.4 (compile)
 / org.hibernate:hibernate-core:jar:3.5.1-Final (provided)
 / org.jboss.javaee:jboss-jacc-api_JDK4:jar:1.1.0 (provided)
 / org.jboss.javaee:jboss-servlet-api_3.0:jar:1.0-SNAPSHOT (compile)
  : org.eclipse.aether.resolution.DependencyResolutionException:
     Could not find artifact org.jboss.javaee:jboss-servlet-api_3.0:jar:1.0-SNAPSHOT in central (http://repo1.maven.org/maven2/)

        at com.google.cloud.tools.opensource.dependencies.DependencyGraphBuilder.levelOrder(DependencyGraphBuilder.java:341)
        at com.google.cloud.tools.opensource.dependencies.DependencyGraphBuilder.getStaticLinkageCheckDependencyGraph(DependencyGraphBuilder.java:192)
        at com.google.cloud.tools.opensource.classpath.ClassPathBuilder.artifactsToDependencyPaths(ClassPathBuilder.java:86)
        at com.google.cloud.tools.opensource.classpath.ClassPathBuilder.artifactsToClasspath(ClassPathBuilder.java:56)
        at com.google.cloud.tools.opensource.classpath.LinkageCheckerArguments.getInputClasspath(LinkageCheckerArguments.java:187)
        at com.google.cloud.tools.opensource.classpath.LinkageCheckerMain.main(LinkageCheckerMain.java:50)

> Task :checkJavaLinkage FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':checkJavaLinkage'.
> Process 'command '/Library/Java/JavaVirtualMachines/jdk1.8.0_221.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 14m 30s
885 actionable tasks: 156 executed, 729 up-to-date

Any artifact that has hadoop-client fails with this problem. Failure at beam-sdks-java-extensions-sql-datacatalog:

org.apache.beam:beam-sdks-java-extensions-sql-datacatalog:jar:2.19.0-SNAPSHOT (compile) / org.apache.beam:beam-sdks-java-extensions-sql:jar:2.19.0-SNAPSHOT (provided) / org.apache.beam:beam-sdks-java-io-parquet:jar:2.19.0-SNAPSHOT (provided) / org.apache.parquet:parquet-avro:jar:1.10.0 (compile) / org.apache.parquet:parquet-hadoop:jar:1.10.0 (compile) / org.apache.hadoop:hadoop-client:jar:2.7.3 (provided) / org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.3 (compile) / org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.3 (compile) / org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.3 (compile) / org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.3 (compile) / org.apache.hadoop:hadoop-yarn-common:jar:2.7.3 (compile) / org.apache.hadoop:hadoop-yarn-api:jar:2.7.3 (compile) / org.apache.hadoop:hadoop-common:jar:2.7.3 (provided) / org.apache.hadoop:hadoop-auth:jar:2.7.3 (compile) / org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15 (compile) / net.sf.ehcache:ehcache-core:jar:2.4.4 (compile) / org.hibernate:hibernate-core:jar:3.5.1-Final (provided) / org.jboss.javaee:jboss-jacc-api_JDK4:jar:1.1.0 (provided) / org.jboss.javaee:jboss-servlet-api_3.0:jar:1.0-SNAPSHOT (compile): org.eclipse.aether.resolution.DependencyResolutionException: Could not find artifact org.jboss.javaee:jboss-servlet-api_3.0:jar:1.0-SNAPSHOT in central (http://repo1.maven.org/maven2/)

        at com.google.cloud.tools.opensource.dependencies.DependencyGraphBuilder.levelOrder(DependencyGraphBuilder.java:341)
        at com.google.cloud.tools.opensource.dependencies.DependencyGraphBuilder.getStaticLinkageCheckDependencyGraph(DependencyGraphBuilder.java:192)
        at com.google.cloud.tools.opensource.classpath.ClassPathBuilder.artifactsToDependencyPaths(ClassPathBuilder.java:86)
        at com.google.cloud.tools.opensource.classpath.ClassPathBuilder.artifactsToClasspath(ClassPathBuilder.java:56)
        at com.google.cloud.tools.opensource.classpath.LinkageCheckerArguments.getInputClasspath(LinkageCheckerArguments.java:187)
        at com.google.cloud.tools.opensource.classpath.LinkageCheckerMain.main(LinkageCheckerMain.java:50)

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 15 (11 by maintainers)

Most upvoted comments

For Apache Beam, --auto-add-provided-and-optional-dependencies would be more useful if you also add --exclude-dependencies rg.jboss.javaee:jboss-servlet-api_3.0:jar:1.0,org.apache.hadoop:hadoop-client:jar:2.7.3 since it could be that we provide two optional dependencies that both supply the same functionality and ask the user to choose one.

Other users could have value in --provide-dependencies.

Apache Beam doesn’t provide an OSGi bundle but some of its libraries contain the metadata and the linkage checker attempts to find the OSGi metadata classes even though they are only used by an OSGi capable runtime. I know to ignore them when the linkage checker complains about them. If I remember which one complained, I’ll try to get a repro for you.

lukecwik on Jan 7, 2020