grpc-java: Could not find TLS ALPN provider; no working netty-tcnative, Conscrypt, or Jetty NPN/ALPN available

Please answer these questions before submitting your issue.

What version of gRPC are you using?

1.13.1

What did you expect to see?

The jar should be running fine.

I am using java 8 to build an executable jar. Below is the java version:

$ /usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/bin/java -version
openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-8u171-b11-2~14.04-b11)
OpenJDK 64-Bit Server VM (build 25.171-b11, mixed mode)

I am using gradle 3.4.1 to generate the jar as follows: /opt/gradle-3.4.1/bin/gradle jar -Dorg.gradle.java.home=/usr/lib/jvm/java-1.8.0-openjdk-amd64/ Below are the dependencies defined in the module-level build.gradle file:

dependencies {
  compile files("$TOOLCHAIN_VERSION_DIR/lib/commons-io-2.6.jar")
  compile files("$TOOLCHAIN_VERSION_DIR/lib/grpc-all.jar")
  compile files("$TOOLCHAIN_VERSION_DIR/lib/java-protobuf.jar")
  compile files("$TOOLCHAIN_VERSION_DIR/lib/jetty.jar")
  compile files("$TOOLCHAIN_VERSION_DIR/lib/log4j-core-2.8.jar")
  compile files("$TOOLCHAIN_VERSION_DIR/lib/log4j-slf4j-impl-2.8.jar")
  compile files("$TOOLCHAIN_VERSION_DIR/lib/netty-tcnative-boringssl-static-2.0.20.Final.jar")
  compile files("$TOOLCHAIN_VERSION_DIR/lib/netty-tcnative-2.0.20.Final.jar")
  compile files("$TOOLCHAIN_VERSION_DIR/lib/picocli-3.8.2.jar")
  compile group: 'com.google.guava', name: 'guava', version: '20.0'
  compile files("$TOOLCHAIN_VERSION_DIR/lib/javassist-3.19.0-GA.jar")
  compile project(':annotation')
}

After building, I am running the jar on an AIX7.2 machine as follows: /usr/java8_64/jre/bin/java -jar agent-1.0.jar

The java version on the AIX machine is as follows:

$ /usr/java8_64/jre/bin/java -version           
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 8.0.5.26 - pap6480sr5fp26-20181115_03(SR5 FP26))
IBM J9 VM (build 2.9, JRE 1.8.0 AIX ppc64-64-Bit Compressed References 20181106_401576 (JIT enabled, AOT enabled)
OpenJ9   - fde1d6f
OMR      - d8c3617
IBM      - 5c4a9f0)
JCL - 20181022_01 based on Oracle jdk8u191-b26

But I am getting the following error while running:

Feb 17, 2019 3:11:33 AM io.grpc.netty.GrpcSslContexts defaultSslProvider
INFO: netty-tcnative unavailable (this may be normal)
java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_aix_ppc_64, netty_tcnative_ppc_64, netty_tcnative]
        at io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:93)
        at io.netty.handler.ssl.OpenSsl.loadTcNative(OpenSsl.java:430)
        at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:97)
        at io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:242)
        at io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:171)
        at io.grpc.netty.GrpcSslContexts.forServer(GrpcSslContexts.java:151)
        at io.grpc.netty.NettyServerBuilder.useTransportSecurity(NettyServerBuilder.java:456)
        at io.grpc.netty.NettyServerBuilder.useTransportSecurity(NettyServerBuilder.java:55)
        at com.ankit.agents.AgentMain.<init>(AgentMain.java:91)
        at com.ankit.agents.AgentMain.main(AgentMain.java:132)
        Suppressed: java.lang.UnsatisfiedLinkError: could not load a native library: netty_tcnative_aix_ppc_64
                at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:205)
                at io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:85)
                ... 9 more
        Caused by: java.io.FileNotFoundException: META-INF/native/libnetty_tcnative_aix_ppc_64.a
                at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
                ... 10 more
                Suppressed: java.lang.UnsatisfiedLinkError: netty_tcnative_aix_ppc_64 (Not found in java.library.path)
                        at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:1425)
                        at java.lang.ClassLoader.loadLibraryWithClassLoader(ClassLoader.java:1395)
                        at java.lang.System.loadLibrary(System.java:565)
                        at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
                        at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
                        at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
                        ... 10 more
                        Suppressed: java.lang.UnsatisfiedLinkError: netty_tcnative_aix_ppc_64 (Not found in java.library.path)
                                at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:1425)
                                at java.lang.ClassLoader.loadLibraryWithClassLoader(ClassLoader.java:1395)
                                at java.lang.System.loadLibrary(System.java:565)
                                at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
                                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
                                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
                                at java.lang.reflect.Method.invoke(Method.java:508)
                                at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
                                at java.security.AccessController.doPrivileged(AccessController.java:647)
                                at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
                                at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
                                ... 11 more
        Suppressed: java.lang.UnsatisfiedLinkError: could not load a native library: netty_tcnative_ppc_64
                at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:205)
                at io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:85)
                ... 9 more
        Caused by: java.io.FileNotFoundException: META-INF/native/libnetty_tcnative_ppc_64.a
                at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
                ... 10 more
                Suppressed: java.lang.UnsatisfiedLinkError: netty_tcnative_ppc_64 (Not found in java.library.path)
                        at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:1425)
                        at java.lang.ClassLoader.loadLibraryWithClassLoader(ClassLoader.java:1395)
                        at java.lang.System.loadLibrary(System.java:565)
                        at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
                        at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
                        at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
                        ... 10 more
                        Suppressed: java.lang.UnsatisfiedLinkError: netty_tcnative_ppc_64 (Not found in java.library.path)
                                at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:1425)
                                at java.lang.ClassLoader.loadLibraryWithClassLoader(ClassLoader.java:1395)
                                at java.lang.System.loadLibrary(System.java:565)
                                at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
                                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
                                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
                                at java.lang.reflect.Method.invoke(Method.java:508)
                                at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
                                at java.security.AccessController.doPrivileged(AccessController.java:647)
                                at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
                                at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
                                ... 11 more
        Suppressed: java.lang.UnsatisfiedLinkError: could not load a native library: netty_tcnative
                at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:205)
                at io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:85)
                ... 9 more
        Caused by: java.io.FileNotFoundException: META-INF/native/libnetty_tcnative.a
                at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
                ... 10 more
                Suppressed: java.lang.UnsatisfiedLinkError: netty_tcnative (Not found in java.library.path)
                        at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:1425)
                        at java.lang.ClassLoader.loadLibraryWithClassLoader(ClassLoader.java:1395)
                        at java.lang.System.loadLibrary(System.java:565)
                        at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
                        at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
                        at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
                        ... 10 more
                        Suppressed: java.lang.UnsatisfiedLinkError: netty_tcnative (Not found in java.library.path)
                                at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:1425)
                                at java.lang.ClassLoader.loadLibraryWithClassLoader(ClassLoader.java:1395)
                                at java.lang.System.loadLibrary(System.java:565)
                                at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
                                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
                                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
                                at java.lang.reflect.Method.invoke(Method.java:508)
                                at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
                                at java.security.AccessController.doPrivileged(AccessController.java:647)
                                at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
                                at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
                                ... 11 more

Feb 17, 2019 3:11:33 AM io.grpc.netty.GrpcSslContexts defaultSslProvider
INFO: Conscrypt not found (this may be normal)
Feb 17, 2019 3:11:33 AM io.grpc.netty.GrpcSslContexts defaultSslProvider
INFO: Jetty ALPN unavailable (this may be normal)
java.lang.ClassNotFoundException: org.eclipse.jetty.alpn.ALPN
        at java.lang.Class.forNameImpl(Native Method)
        at java.lang.Class.forName(Class.java:403)
        at io.grpc.netty.JettyTlsUtil.isJettyAlpnConfigured(JettyTlsUtil.java:64)
        at io.grpc.netty.JettyTlsUtil.getJettyAlpnUnavailabilityCause(JettyTlsUtil.java:75)
        at io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:255)
        at io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:171)
        at io.grpc.netty.GrpcSslContexts.forServer(GrpcSslContexts.java:151)
        at io.grpc.netty.NettyServerBuilder.useTransportSecurity(NettyServerBuilder.java:456)
        at io.grpc.netty.NettyServerBuilder.useTransportSecurity(NettyServerBuilder.java:55)
        at com.ankit.agents.AgentMain.<init>(AgentMain.java:91)
        at com.ankit.agents.AgentMain.main(AgentMain.java:132)

Exception in thread "main" java.lang.IllegalStateException: Could not find TLS ALPN provider; no working netty-tcnative, Conscrypt, or Jetty NPN/ALPN available
        at io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:256)
        at io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:171)
        at io.grpc.netty.GrpcSslContexts.forServer(GrpcSslContexts.java:151)
        at io.grpc.netty.NettyServerBuilder.useTransportSecurity(NettyServerBuilder.java:456)
        at io.grpc.netty.NettyServerBuilder.useTransportSecurity(NettyServerBuilder.java:55)
        at com.ankit.agents.AgentMain.<init>(AgentMain.java:91)
        at com.ankit.agents.AgentMain.main(AgentMain.java:132)

However, it is running fine on an Ubuntu machine. I am running the jar as follows: /usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/bin/java -jar agent-1.0.jar

The java version here is:

$ /usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/bin/java -version
openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-8u171-b11-2~14.04-b11)
OpenJDK 64-Bit Server VM (build 25.171-b11, mixed mode)

What is the problem here and how to solve this?

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 35 (8 by maintainers)

Most upvoted comments

I had the same problem and I was using a Docker image with a kernel that was incompatible with the netty native libraries needed by my app. I changed the image in my Dockerfile(in my case from “FROM openjdk:8-jre-alpine” to “FROM openjdk:8”) and the issue was fixed.

+32
crengutam on May 17, 2019

It must be 64 bit

On Wed, Jul 31, 2019, 2:20 PM Marek Meyer notifications@github.com wrote:

I still faced the issue while using openjdk:8 and managed to fix it by upgrading to openjdk:11 😊 Java 11 has ALPN available by default.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/grpc/grpc-java/issues/5369?email_source=notifications&email_token=AA2RM3I2PS5XT6TSC7X2Q3LQCFG5ZA5CNFSM4GX6BP3KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3GR2LI#issuecomment-516758829, or mute the thread https://github.com/notifications/unsubscribe-auth/AA2RM3KZDF2LCOD6JUVNDILQCFG5ZANCNFSM4GX6BP3A .

+2
Ravipatel401 on Jul 31, 2019

You need to have an (exact) version match for the netty and tcnative versions to work together. See also the table at https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty

grpc-version: 1.13.1 tcnative-version: 2.0.20.Final (but should be 2.0.8.Final)

Off-Topic: Could we add a headline/link anchor there so that the table is easier to link?

+2
ST-DDT on Feb 18, 2019

I had the same issue and I resolved changing the java version to 64 bit!

+1
mrojasabregu on Dec 11, 2019

Hey Guys,

Tying to identifying this issues from last 4 days but found the perfect solution that is just checked when

Experiment 1. :- Java 8 32bit with eclipse 32 bit :---- Failed to load any of the given libraries: [netty_tcnative_windows_x86_32, netty_tcnative_x86_32, netty_tcnative] Experiment 2.:- ### Java 8 64bit with Eclipse Luna 64 bit 👍 :- Worked Without Error.

Thanks

+1
Ravipatel401 on Jun 8, 2019

Can you paste the logs if you just use the dependencies as described in the readme (+plus the non-grpc/non-netty/non-tcnative libs)?

compile 'io.grpc:grpc-netty-shaded:1.18.0'
compile 'io.grpc:grpc-protobuf:1.18.0'
compile 'io.grpc:grpc-stub:1.18.0'
+1
ST-DDT on Feb 18, 2019

@ST-DDT I tried to use grpc 1.18.0 which matches tcnative-version 2.0.20.Final. But it is giving the same error. My guess is that since I have built it on an ubuntu machine using openjdk, it is creating problem for IBM JRE. By the way, is there any option other than netty-tcnative that can be used in my use case? Are you aware of some? I would appreciate any help.

+1
ankitshubham97 on Feb 18, 2019
Read more comments on GitHub
← grpc-java: Binary backwards-incompatibily in 1.33.0
grpc: java-csharp interop failed with INTERNAL: Connection closed with unknown cause →