grpc-node: Using client after dropped connection leads to freezing of the whole NodeJs Process

Problem description

Please correct me if I’m wrong, this grpc client should automatically reconnect if the connection becomes invalid. I see this behavior in most cases, however there is one error which makes the client unusable and the whole NodeJs app gets frozen. It’ s “Connection dropped”. Other errors such as “Channel has been shut down”, “Received RST_STREAM with code 2” don’t lead to such behavior.

Reproduction steps

I create a client at a start of my app, then I occasionally use this grpc client. It works for a few hours but then all of the sudden my app gets frozen. And the last log is always this:

Connection dropped","metadata":{"internalRepr":{},"options":{}},"level":"warn","message":"Failed to execute grpc callConnection dropped","stack":"Error: Connection dropped\n at Object.callErrorFromStatus (/usr/src/app/node_modules/@grpc/grpc-js/src/call.ts:81:24)\n at Object.onReceiveStatus (/usr/src/app/node_modules/@grpc/grpc-js/src/client.ts:334:36)\n at Object.onReceiveStatus (/usr/src/app/node_modules/@grpc/grpc-js/src/client-interceptors.ts:434:34)\n at Object.onReceiveStatus (/usr/src/app/node_modules/@grpc/grpc-js/src/client-interceptors.ts:397:48)\n at Http2CallStream.outputStatus (/usr/src/app/node_modules/@grpc/grpc-js/src/call-stream.ts:230:22)\n at Http2CallStream.maybeOutputStatus (/usr/src/app/node_modules/@grpc/grpc-js/src/call-stream.ts:280:14)\n at Http2CallStream.endCall (/usr/src/app/node_modules/@grpc/grpc-js/src/call-stream.ts:263:12)\n at Http2CallStream.disconnectListener (/usr/src/app/node_modules/@grpc/grpc-js/src/call-stream.ts:215:12)\n at Socket.session.socket.once (/usr/src/app/node_modules/@grpc/grpc-js/src/subchannel.ts:518:13)\n at Object.onceWrapper (events.js:286:20)\n at Socket.emit (events.js:203:15)\n at Socket.EventEmitter.emit (domain.js:448:20)\n at TCP._handle.close (net.js:607:12)

Temporarily fix

Now if I get an exception during executing a grpc call I just close the current client and create a new one. But I wonder if it’s the right approach.

Environment

• Debian GNU/Linux 9 (stretch)
• Node 10
• Node installation method: apt
• “@grpc/grpc-js”: “^1.1.1”,

Additional context

Add any other context about the problem here. If possible, attach full logs. Do not try to omit anything for brevity, but instead include absolutely everything. Please try and set your operating system’s locale to English, so that logs contain error messages in the English language.