lakeFS: [Bug]: Getting error to copy Glue metastore using lakectl
What happened?
Getting errors to run following commands:
lakectl metastore copy \ --catalog-id 11111111111 \ --from-client-type glue \ --from-schema amit_glue_hive_demo \ --from-table customers \ --to-client-type glue \ --to-schema amit_glue_hive_demo_glue_etl_branch \ --to-table customers \ --to-branch glue-etl-branch
Error message:
InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, GetTableInput.CatalogId.
lakectl metastore copy-schema \ --catalog-id 1111111111 \ --from-client-type glue \ --from-schema amit_glue_hive_demo \ --to-client-type glue \ --to-schema amit_glue_hive_demo_glue_etl_branch \ --to-branch glue-etl-branch
failed to get database on copy from 'amit_glue_hive_demo': InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, GetDatabaseInput.CatalogId.
lakectl metastore copy-all \ --from-client-type glue \ --from-address amit_glue_hive_demo \ --to-client-type glue \ --to-address amit_glue_hive_demo_glue_etl_branch \ --branch glue-etl-branch
InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, GetDatabasesInput.CatalogId.
Expected behavior
No response
lakeFS version
0.104.0
How lakeFS is installed
lakeFS Cloud
Affected clients
No response
Relevant log output
No response
Contact details
amit.kesarwani@treeverse.io
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 16 (16 by maintainers)
Hi @kesarwam, the
--from-addressand--to-addressfields inlakectl metastore copy-allmight be confusing. They are used only when the type ishive. Thecopy allcommand should be used to copy all data between different metastores, that is (hive->glue,glue->hive,hive:address1->hive:address2)It seems like there are two things to fix to make things clear:
--from-addressor--to-addresswhen the type isgluecopy-allon the same metastore return “copy-all can’t be done within the same metastore”