firebase-functions: Firestore/triggers grpc-js No connection established errors
Related issues
https://github.com/googleapis/nodejs-firestore/issues/768
https://github.com/firebase/firebase-js-sdk/issues/1783
https://github.com/grpc/grpc-node/issues/1027
Version info
node: 10.15.1
firebase-functions: 3.3.0
firebase-tools: 7.6.0
firebase-admin: 8.6.0
Test case
exports.addDriverLocation = functions.region(region).firestore
.document('drivers/{userId}/locations/{locationId}')
.onCreate((snap, context) => {
const { userId = -3 } = context.params;
const record = snap.data();
const { location = {}} = record;
const {
uuid,
coords: {
latitude,
longitude
} = {},
extras
} = location;
return snap.ref.set({
userId,
// server timestamp
sts: admin.firestore.Timestamp.fromDate(new Date()),
// client timestamp
cts: admin.firestore.Timestamp.fromDate(new Date(timestamp)),
latitude,
longitude,
extras
});
Steps to reproduce
- Create a firestore trigger function
Expected behavior
Every call should be get executed without gRPC related errors.
Actual behavior
Every now and then we get a No connection established error:
Error: No connection established
at Http2CallStream.call.on (/srv/functions/node_modules/@grpc/grpc-js/build/src/client.js:96:45)
at Http2CallStream.emit (events.js:194:15)
at Http2CallStream.EventEmitter.emit (domain.js:459:23)
at process.nextTick (/srv/functions/node_modules/@grpc/grpc-js/build/src/call-stream.js:75:22)
at process._tickCallback (internal/process/next_tick.js:61:11)
It encountered 8600 exact errors for the past week:
For every other firestore operation, we used old grpc
as described here https://github.com/googleapis/nodejs-firestore/issues/768#issuecomment-537311719:
const Firestore = require('@google-cloud/firestore');
const grpc = require('grpc');
const firestore = new Firestore({ grpc });
but we don’t know how to set old grpc for functions.region(region).firestore
to get rid of all these errors.
We are not sure if here is the right place to post this issue or the https://github.com/googleapis/nodejs-firestore. If this is the case, the please close this issue and we’ll go and report it on the nodejs-firestore repo.
Were you able to successfully deploy your functions?
No errors during functions deploy.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 5
- Comments: 52
I had a rogue reference in my package-lock.json
After nuking that and starting from a clean slate I was able to get the correct grpc-js version. For anyone else running into this issue right now, you need to delete BOTH your node_modules and your package-lock.json
For those interested it was under google-gax as a required dependency. It should read:
@grpc/grpc-js": "0.6.9"
I have since had three scheduled calls go off with no failures. Fingers crossed this is the one, thanks for the timely responses from all and the community support!
@adamkpurdy It seems to be sporadically failing on our project still.
Hey peoples:
I just did a dep upgrade after reading grpc/grpc-node#1027:
I’ve had three successful triggers fired. Anybody else wanna try and post their results?
Same error here. No solution yet?
This started popping up again today for me inside a cloud function.
I’m also getting the same issue using
firebase-admin v8.10.0
. Fails consistently, then works consistently a few times, then continues to fail.This issue is not solved. Deleting package-lock.json and npm modules doesn’t solve the problem.
Just for a record, I started to experience the same exact problem 3 days ago. Clean slate does not really help. Spontaneously some function will run but the majority will fail. The logs output snippet is this: "events.js:174
Error: 14 UNAVAILABLE: No connection established at Object.callErrorFromStatus (/Users/milan/Projects/firebase/motus/functions/node_modules/@grpc/grpc-js/build/src/call.js:31:26) at Object.onReceiveStatus (/Users/milan/Projects/firebase/motus/functions/node_modules/@grpc/grpc-js/build/src/client.js:327:49) at Object.onReceiveStatus (/Users/milan/Projects/firebase/motus/functions/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:305:181) at Http2CallStream.outputStatus (/Users/milan/Projects/firebase/motus/functions/node_modules/@grpc/grpc-js/build/src/call-stream.js:117:74) at Http2CallStream.maybeOutputStatus (/Users/milan/Projects/firebase/motus/functions/node_modules/@grpc/grpc-js/build/src/call-stream.js:156:22) at Http2CallStream.endCall (/Users/milan/Projects/firebase/motus/functions/node_modules/@grpc/grpc-js/build/src/call-stream.js:142:18) at Http2CallStream.cancelWithStatus (/Users/milan/Projects/firebase/motus/functions/node_modules/@grpc/grpc-js/build/src/call-stream.js:467:14) at ChannelImplementation.tryPick (/Users/milan/Projects/firebase/motus/functions/node_modules/@grpc/grpc-js/build/src/channel.js:238:32) at ChannelImplementation._startCallStream (/Users/milan/Projects/firebase/motus/functions/node_modules/@grpc/grpc-js/build/src/channel.js:268:14) at Http2CallStream.start (/Users/milan/Projects/firebase/motus/functions/node_modules/@grpc/grpc-js/build/src/call-stream.js:445:22) Emitted ‘error’ event at: at Object.onReceiveStatus (/Users/milan/Projects/firebase/motus/functions/node_modules/@grpc/grpc-js/build/src/client.js:327:28) at Object.onReceiveStatus (/Users/milan/Projects/firebase/motus/functions/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:305:181)
No workaround found yet that works for rme.
I don’t know if it helps anyone, because I’m not using grpc-js, but I got an error in my Firebase function logs:
It looks like the function had spontaneously stopped working around 11:30pm last night.
I found this issue thread and ran
npm update
to update the dependencies. I redeployed, and it started working again.These were the packages that got updated:
firebase-functions
from “^3.6.1” to “^3.7.0”typescript
from “^3.8.0” to “^3.9.5”@types
packages, but I don’t think that would have affected it.Still having this issue. Clearing the
node_modules
and reinstalling the deps didn’t work. Any idea on what’s causing this?@jhk115 only Firestore
@jhk115 I’m not doing anything with storage. Only Firestore.
Nope, you were right to do your due diligence. I hadn’t gotten a fully clean state. Looks like the workaround does work for me.
I still experience this, running
firebase-functions
3.3.0 andfirebase-admin
8.6.0. I tried deletingnode_modules
andpackage-lock.json
and reinstalling, but still gotgrpc-js
0.6.4…Updating from
"firebase-functions": "^3.2.0"
to"firebase-functions": "^3.3.0"
and re deploy worked for me ! Thanks everyone.It looks promising on our end! Thanks, everyone.
@adamkpurdy Upgraded also, working for me so far!
@jhk115 This is because of a dependency that firestore is using called
grpc-js
as mentioned by @clytrasThere have been workarounds referenced if you look further back in the thread and to the linked issues on the original comment, however there are still situations where you can not manage this dependency (to our knowledge so far)
@mrmodelo this occurs to every kind of function that involves firestore and it’s related to firestore changing
grpc
togrpc-js
. There is a workaround for other kind of functions as you can see on my initial post, but I can’t find a way to set old grpc for firestrore funtion triggers.