extensions: Error when running fs-bq-import-collection. PartialFailureError

@joaqcid this was a big help!!!

git clone git@github.com:joaqcid/extensions.git
cd extensions
git checkout run-manual-import
cd firestore-bigquery-export/firestore-bigquery-change-tracker
npm i
cd ../scripts/import
npm i
npm run import

hi all,

i creted a branch “run-manual-import” on my fork for to run it manually, https://github.com/joaqcid/extensions

from there, you need to

• npm install in firestore-bigquery-change-tracker folder (dont mind the errors)
• npm install in scripts/import folder
• npm run import

hope this helps

run-manual-import

@joaqcid Do I want to use the fix-bq-import fork or the run-manual-import fork?

Hi, the package has been updated on NPM, sorry for the delay!

@sourcec0de , got compilation errors during install:

$ npm i

> @firebaseextensions/firestore-bigquery-change-tracker@1.0.1 prepare /home/astar/dev/extensions/firestore-bigquery-export/firestore-bigquery-change-tracker
> npm run build


> @firebaseextensions/firestore-bigquery-change-tracker@1.0.1 build /home/astar/dev/extensions/firestore-bigquery-export/firestore-bigquery-change-tracker
> npm run clean && npm run compile


> @firebaseextensions/firestore-bigquery-change-tracker@1.0.1 clean /home/astar/dev/extensions/firestore-bigquery-export/firestore-bigquery-change-tracker
> rimraf functions/lib


> @firebaseextensions/firestore-bigquery-change-tracker@1.0.1 compile /home/astar/dev/extensions/firestore-bigquery-export/firestore-bigquery-change-tracker
> tsc

src/test/bigquery/snapshot.spec.ts:47:1 - error TS2582: Cannot find name 'describe'. Do you need to install type definitions for a test runner? Try `npm i @types/jest` or `npm i @types/mocha`.

47 describe("latest snapshot view sql generation", () => {
   ~~~~~~~~

src/test/bigquery/snapshot.spec.ts:48:3 - error TS2582: Cannot find name 'it'. Do you need to install type definitions for a test runner? Try `npm i @types/jest` or `npm i @types/mocha`.

48   it("should generate the epxected sql", async () => {
     ~~

src/test/bigquery/snapshot.spec.ts:60:3 - error TS2582: Cannot find name 'it'. Do you need to install type definitions for a test runner? Try `npm i @types/jest` or `npm i @types/mocha`.

60   it("should generate correct sql with no groupBy columns", async () => {
     ~~

src/test/bigquery/snapshot.spec.ts:72:3 - error TS2582: Cannot find name 'it'. Do you need to install type definitions for a test runner? Try `npm i @types/jest` or `npm i @types/mocha`.

72   it("should throw an error for empty group by columns", async () => {
     ~~

src/test/bigquery/snapshot.spec.ts:79:3 - error TS2582: Cannot find name 'it'. Do you need to install type definitions for a test runner? Try `npm i @types/jest` or `npm i @types/mocha`.

79   it("should throw an error for empty timestamp field", async () => {
     ~~


Found 5 errors.

I ignored those installation errors and proceed to import. Which worked well. Thx.

@joaqcid no errors when doing npm install at scripts/import, but still get this error: 13 verbose stack Error: @firebaseextensions/fs-bq-import-collection@0.1.3 import: node ./lib/index.js13 verbose stack Exit status 1 13 verbose stack at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16) 13 verbose stack at EventEmitter.emit (events.js:223:5) 13 verbose stack at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14) 13 verbose stack at ChildProcess.emit (events.js:223:5) 13 verbose stack at maybeClose (internal/child_process.js:1021:16) 13 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:283:5) 14 verbose pkgid @firebaseextensions/fs-bq-import-collection@0.1.3

MacBook-Pro:import User$ npm run import

@firebaseextensions/fs-bq-import-collection@0.1.3 import /Users/user/node_modules/extensions-run-manual-import/firestore-bigquery-export/scripts/import node ./lib/scripts/import/src/index.js

internal/modules/cjs/loader.js:796 throw err; ^

Error: Cannot find module ‘/Users/user/node_modules/extensions-run-manual-import/firestore-bigquery-export/scripts/import/lib/scripts/import/src/index.js’ at Function.Module._resolveFilename (internal/modules/cjs/loader.js:793:17) at Function.Module._load (internal/modules/cjs/loader.js:686:27) at Function.Module.runMain (internal/modules/cjs/loader.js:1043:10) at internal/main/run_main_module.js:17:11 { code: ‘MODULE_NOT_FOUND’, requireStack: [] } npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! @firebaseextensions/fs-bq-import-collection@0.1.3 import: node ./lib/scripts/import/src/index.js npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the @firebaseextensions/fs-bq-import-collection@0.1.3 import script. npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in: npm ERR! /Users/user/.npm/_logs/2020-02-14T10_34_01_803Z-debug.log

@joaqcid i tried your solution

getting this error , it will be great if you can look into this

@tptshoe i cant understand the error in the logs, try clearing all node_modules folders from the firebase/extensions and re-run npm install only under /firestore-bigquery-export/firestore-bigquery-change-tracker /firestore-bigquery-export/scripts/import

then run npm import under

/firestore-bigquery-export/scripts/import

@joaqcid. Thanks a heap, worked a treat…

I made the changes suggested in the PR to the index.js file (not using Typescript) and I updated the dependency to 4.7 but I’m still getting the error.

I called ngx with the --no-install flag so that it would have to use the modified file.

Hi @joaqcid

You seem to have solved this, but I wasn’t able to replicate what you did. I cloned the repo and updated the dependecy to 4.7, but that didn’t seem to help when I ran it again.

Could you do an idiots guide version of your above comment please?