grpc-java: UNAVAILABLE: HTTP/2 error code: NO_ERROR Received Goaway
What version of gRPC-Java are you using?
1.30.2
What is your environment?
Linux xxxxx016 3.10.0-1062.18.1.el7.x86_64 #1 SMP Tue Mar 17 23:49:17 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux java version “1.8.0_91” Java™ SE Runtime Environment (build 1.8.0_91-b14) Java HotSpot™ 64-Bit Server VM (build 25.91-b14, mixed mode)
What did you expect to see?
What did you see instead?
io.grpc.StatusRuntimeException: UNAVAILABLE: HTTP/2 error code: NO_ERROR
Received Goaway
at io.grpc.Status.asRuntimeException(Status.java:533)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:460)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:426)
at io.grpc.internal.ClientCallImpl.access$500(ClientCallImpl.java:66)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:689)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$900(ClientCallImpl.java:577)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:751)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:740)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Steps to reproduce the bug
The java grpc client connects to the go grpc service (service grpc version 1.27.1). The above exception occurs. The service uses container deployment and nginx for load balancing. There is a high probability of this abnormal situation. Of course, we are also working hard to find a solution, but there is no progress. Hope to get your guidance and help. I will be very grateful.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 20 (11 by maintainers)
Commits related to this issue
- netty: Refine workaround for Netty header processing for transparent retries Nginx and C core don't do graceful GOAWAY and retries have matured such that transparent retries may soon be on by default... — committed to ejona86/grpc-java by ejona86 3 years ago
- netty: Refine workaround for Netty header processing for transparent retries Nginx and C core don't do graceful GOAWAY and retries have matured such that transparent retries may soon be on by default... — committed to grpc/grpc-java by ejona86 3 years ago
- netty: Refine workaround for Netty header processing for transparent retries Nginx and C core don't do graceful GOAWAY and retries have matured such that transparent retries may soon be on by default... — committed to ejona86/grpc-java by ejona86 3 years ago
I think we can change the workaround to only trigger if the HTTP/2 error-code is indicates an error (anything but NO_ERROR). That would workaround the Netty problem but allow transparent retries to function on servers that use abrupt GOAWAYs.