extensions: "Provided schema does not match table" when running the fs-bq-schema-view script

Extension name: firetore-bigquery-export (the gen-schema-views script)
Extension version: firetore-bigquery-export@0.1.26, latest `gen-schema-views script´ (I´ve also tried with 0.4.2)

We’re updating our schema, and after deleting the previously generated views (as instructed by #423 and #646) running npx @firebaseextensions/fs-bq-schema-views fails with the following output:

{"code":400,"errors":[{"message":"Provided Schema does not match Table audit-backend:audit.audit_schema_fields_changelog. Cannot add fields (field: old_data)","domain":"global","reason":"invalid"}],"response":{"headers":{"alt-svc":"h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"","cache-control":"private","content-encoding":"gzip","content-type":"application/json; charset=UTF-8","date":"Wed, 16 Nov 2022 17:21:45 GMT","etag":"338qmb/WriADWLSvCyjq2w==","server":"ESF","transfer-encoding":"chunked","vary":"Origin, X-Origin, Referer","x-content-type-options":"nosniff","x-frame-options":"SAMEORIGIN","x-xss-protection":"0"}},"message":"Provided Schema does not match Table audit-backend:audit.audit_schema_fields_changelog. Cannot add fields (field: old_data)"}
Provided Schema does not match Table audit-backend:audit.audit_schema_fields_changelog. Cannot add fields (field: old_data)

The script does create the audit_schema_fields_changelog view with most of the fields (it apparently stops while trying to add the old_data field), but it does not create the other one.

I’ve updated the schema a couple of times in the past following the exact same procedure without any issues. I’m using the exact same command as before:

npx @firebaseextensions/fs-bq-schema-views \
  --non-interactive \
  --project=audit-backend \
  --dataset=audit \
  --table-name-prefix=audit \
  --schema-files=./fields.json

I’ve also explicitly set the version to the previous one (npx @firebaseextensions/fs-bq-schema-views@0.4.2 with the same result

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 9
Comments: 20 (5 by maintainers)

Most upvoted comments

Same issue there - I have tried to upgrade extension via Firebase Console to 0.1.27 (oldData is mentioned for this release as “added feature” - whatever it is - as there are no docs) - upgrade went ok, then I deleted both views in BQ, run again npx scripts…

Unfortunately the same error

alapeta on Nov 24, 2022

I am still getting this error with 0.4.7. It would be great if one could update (add/delete columns) existing schemas without manually deleting the tables, or if we could have an option in the CLI to delete the existing tables when recreating the schema views.

JarnoRFB on Dec 18, 2023

@gustavorochakv @FakhruddinAbdi

Can you confirm that this is still an issue with the latest version.

Previously, I am assuming that you would not need to recreate the views when introducing a new field? But this is now a consistent issue?

Yes, it is still an issue with updating the schema directly It doesn’t allow to update the schema, instead, you have to remove the view manually, then run the script to recreate it

abdifardin on Dec 5, 2022

An update has been added and published for this issue.

Please use to the latest version 0.4.4.

Source: https://www.npmjs.com/package/@firebaseextensions/fs-bq-schema-views)

dackers86 on Nov 29, 2022