extensions: [firestore-counter] Counter extension run locally: Transaction to aggregate shards failed. TypeError: Cannot read properties of undefined (reading 'length')

Configuration

Extension name: Distributed counter
Extension version: 0.2.4 (19 Oct 2021)
Configuration values (redact info where appropriate): LOCATION=“europe-west3” INTERNAL_STATE_PATH= “firebase_ext/sharded_counter” SCHEDULE_FREQUENCY= 1

Problem

This is a follow-up bug report to this question

During the Implementation of Distributed-Counter-Extension for local emulator I was able to run the scheduled function (controllerCore();) via the firebase shell and it will only work for the first aggregation. The next one will throw the following error:

Transaction to aggregate shards failed. 
TypeError: Cannot read properties of undefined (reading 'length')\n    at /Users/**/Documents/**/**-Backend/functions/internal/counter/controller.js:118:44\n .

What I have encountered is, that when removing _firebase_ext_ from the local firestore it will work again for one aggregation again (and the directory will be created again). When running the controllerCore again after the error I will get a timeout warning of my worker function.

functions: Your function timed out after ~60s. To configure this timeout, see
      https://firebase.google.com/docs/functions/manage-functions#set_timeout_and_memory_allocation.
>  /usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:618
>              throw new Error("Function timed out.");

Steps to reproduce:

Implement the Counter extension to the local emulator with the following export changes in the index file as described here:

exports.controllerCore = functions.pubsub.schedule(`every ${Config.counter.SCHEDULE_FREQUENCY} minutes`).onRun(async () => {}

exports.worker = functions.firestore.document(`${Config.counter.INTERNAL_STATE_PATH}/workers/{workerId}`).onWrite(async (change, context) => {}

exports.onWrite = functions.firestore.document(`${Config.counter.INTERNAL_STATE_PATH}/workers/{workerId}`).onWrite(async (change, context) => {}

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 16 (15 by maintainers)

Commits related to this issue

Resolve local emulator error "Transaction to aggregate shards failed" This is a proposed pull request for [this](https://github.com/firebase/extensions/issues/876#issue-1124877924) issue. Note that t... — committed to polarby/extensions by polarby 2 years ago

Most upvoted comments

Give me a week for the longest, I will make it sure to notify you about the results.

polarby on Jul 14, 2022

Wooof, this is a tricky one. Glad to hear that you found a workaround, and AFAIK we’ve never seen this behavior in production. I spent some time digging through the code, and here’s my theory on why you see it only in the emulator:

The ‘workers’ collection gets created within a transaction here: https://github.com/firebase/extensions/blob/9f8d7fd6048bcaa7b5bc505cebe2e90494359e33/firestore-counter/functions/src/controller.ts#L298 That transaction includes a couple reads/writes to the same collection.
The Firestore emulator doesn’t have great support for transactions today (https://firebase.google.com/docs/emulator-suite/connect_firestore#how_the_emulator_differs_from_production), and writing many things to one document can cause lockups - this may be why you were seeing timeouts on the worker function. You may be able to work around this by increasing the timeout of the worker function. This limitation of the Firestore Emulator is actively being worked on, and hopefully you will be fixed in the coming months.

joehan on Feb 8, 2022

Instant temporary solution to bug report

It seems that the worker array doesn’t get initialized after the first iteration.

To solve the error message and make the extension work properly replace the following line in the contoller.js file:

t.set(this.controllerDocRef, { timestamp: firebase_admin_1.firestore.FieldValue.serverTimestamp() }, { merge: true });

if (controllerDoc.exists) { //only update timestamp
   t.set(this.controllerDocRef, { timestamp: firebase_admin_1.firestore.FieldValue.serverTimestamp() }, { merge: true });
} else { //init worker list as well
  t.set(this.controllerDocRef, { workers: [], timestamp: firebase_admin_1.firestore.FieldValue.serverTimestamp() }, { merge: true });

}

I am not sure if I was missing something during the implementation, but it seems this error doesn’t appear in the cloud?! Is this a known error?

polarby on Feb 5, 2022