superset: upgrade from 2.0.1 to 2.1.0.rc3 fails

A clear and concise description of what the bug is.

How to reproduce the bug

pip based install on Debian Bullseye 11.6

  1. pip install apache-superset==2.1.0rc3 - Successful !

  2. superset db upgrade /superset_root/lib/python3.9/site-packages/flask_appbuilder/models/sqla/interface.py:67: SAWarning: relationship ‘SqlaTable.slices’ will copy column tables.id to column slices.datasource_id, which conflicts with relationship(s): ‘Slice.table’ (copies tables.id to slices.datasource_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter ‘overlaps=“table”’ to the ‘SqlaTable.slices’ relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx) for prop in class_mapper(obj).iterate_properties: INFO [alembic.runtime.migration] Context impl MySQLImpl. INFO [alembic.runtime.migration] Will assume transactional DDL. INFO [alembic.runtime.migration] Running upgrade e09b4ae78457 -> f3afaf1f11f0, add_unique_name_desc_rls INFO [alembic.runtime.migration] Running upgrade f3afaf1f11f0 -> 7fb8bca906d2, permalink_rename_filterState INFO [alembic.runtime.migration] Running upgrade 7fb8bca906d2 -> cdcf3d64daf4, Add user_id and dttm composite index to Log model INFO [alembic.runtime.migration] Running upgrade cdcf3d64daf4 -> c747c78868b6, Migrating legacy TreeMap INFO [alembic.runtime.migration] Running upgrade c747c78868b6 -> 06e1e70058c7, Migrating legacy Area INFO [alembic.runtime.migration] Running upgrade 06e1e70058c7 -> a39867932713, query_context_to_mediumtext INFO [alembic.runtime.migration] Running upgrade a39867932713 -> 409c7b420ab0, add created_by_fk as owner INFO [alembic.runtime.migration] Running upgrade 409c7b420ab0 -> ffa79af61a56, rename report_schedule.extra to extra_json INFO [alembic.runtime.migration] Running upgrade ffa79af61a56 -> 6d3c6f9d665d, fix_table_chart_conditional_formatting_colors INFO [alembic.runtime.migration] Running upgrade 6d3c6f9d665d -> 291f024254b5, drop_column_allow_multi_schema_metadata_fetch INFO [alembic.runtime.migration] Running upgrade 291f024254b5 -> deb4c9d4a4ef, parameters in saved queries INFO [alembic.runtime.migration] Running upgrade deb4c9d4a4ef -> 4ce1d9b25135, remove_filter_bar_orientation INFO [alembic.runtime.migration] Running upgrade 4ce1d9b25135 -> f3c2d8ec8595, create_ssh_tunnel_credentials_tbl

  3. superset init Loaded your LOCAL configuration at [/superset_root/superset_config.py] logging was configured successfully 2023-03-24 08:39:16,598:INFO:superset.utils.logging_configurator:logging was configured successfully 2023-03-24 08:39:16,620:INFO:root:Configured event logger of type <class ‘superset.utils.log.DBEventLogger’> We haven’t found any Content Security Policy (CSP) defined in the configurations. Please make sure to configure CSP using the TALISMAN_ENABLED and TALISMAN_CONFIG keys or any other external software. Failing to configure CSP have serious security implications. Check https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP for more information. You can disable this warning using the CONTENT_SECURITY_POLICY_WARNING key. 2023-03-24 08:39:16,622:WARNING:superset.initialization:We haven’t found any Content Security Policy (CSP) defined in the configurations. Please make sure to configure CSP using the TALISMAN_ENABLED and TALISMAN_CONFIG keys or any other external software. Failing to configure CSP have serious security implications. Check https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP for more information. You can disable this warning using the CONTENT_SECURITY_POLICY_WARNING key. /superset_root/lib/python3.9/site-packages/flask_appbuilder/models/sqla/interface.py:67: SAWarning: relationship ‘SqlaTable.slices’ will copy column tables.id to column slices.datasource_id, which conflicts with relationship(s): ‘Slice.table’ (copies tables.id to slices.datasource_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter ‘overlaps=“table”’ to the ‘SqlaTable.slices’ relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx) for prop in class_mapper(obj).iterate_properties: 2023-03-24 08:39:20,274:ERROR:flask_appbuilder.security.sqla.manager:Add View Menu Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_view_menu_id_seq’”) [SQL: INSERT INTO ab_view_menu (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘AvailableDomains’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,292:ERROR:flask_appbuilder.security.sqla.manager:Add View Menu Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_view_menu_id_seq’”) [SQL: INSERT INTO ab_view_menu (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘AvailableDomains’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,301:ERROR:flask_appbuilder.security.sqla.manager:Creation of Permission View Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_view_menu_id_seq’”) [SQL: INSERT INTO ab_view_menu (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘AvailableDomains’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,516:ERROR:flask_appbuilder.security.sqla.manager:Add Permission: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_id_seq’”) [SQL: INSERT INTO ab_permission (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘can_get_or_create_dataset’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,522:ERROR:flask_appbuilder.security.sqla.manager:Creation of Permission View Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_id_seq’”) [SQL: INSERT INTO ab_permission (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘can_get_or_create_dataset’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,553:ERROR:flask_appbuilder.security.sqla.manager:Add Permission: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_id_seq’”) [SQL: INSERT INTO ab_permission (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘can_duplicate’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,559:ERROR:flask_appbuilder.security.sqla.manager:Creation of Permission View Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_id_seq’”) [SQL: INSERT INTO ab_permission (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘can_duplicate’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,703:ERROR:flask_appbuilder.security.sqla.manager:Add Permission: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_id_seq’”) [SQL: INSERT INTO ab_permission (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘can_get_column_values’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,705:ERROR:flask_appbuilder.security.sqla.manager:Creation of Permission View Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_id_seq’”) [SQL: INSERT INTO ab_permission (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘can_get_column_values’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,760:ERROR:flask_appbuilder.security.sqla.manager:Add View Menu Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_view_menu_id_seq’”) [SQL: INSERT INTO ab_view_menu (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘Explore’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,765:ERROR:flask_appbuilder.security.sqla.manager:Add View Menu Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_view_menu_id_seq’”) [SQL: INSERT INTO ab_view_menu (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘Explore’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,767:ERROR:flask_appbuilder.security.sqla.manager:Creation of Permission View Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_view_menu_id_seq’”) [SQL: INSERT INTO ab_view_menu (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘Explore’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,824:ERROR:flask_appbuilder.security.sqla.manager:Add Permission: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_id_seq’”) [SQL: INSERT INTO ab_permission (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘can_execute_sql_query’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,826:ERROR:flask_appbuilder.security.sqla.manager:Creation of Permission View Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_id_seq’”) [SQL: INSERT INTO ab_permission (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘can_execute_sql_query’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,831:ERROR:flask_appbuilder.security.sqla.manager:Add Permission: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_id_seq’”) [SQL: INSERT INTO ab_permission (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘can_export_csv’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,833:ERROR:flask_appbuilder.security.sqla.manager:Creation of Permission View Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_id_seq’”) [SQL: INSERT INTO ab_permission (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘can_export_csv’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,838:ERROR:flask_appbuilder.security.sqla.manager:Add Permission: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_id_seq’”) [SQL: INSERT INTO ab_permission (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘can_get_results’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,840:ERROR:flask_appbuilder.security.sqla.manager:Creation of Permission View Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_id_seq’”) [SQL: INSERT INTO ab_permission (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘can_get_results’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,992:ERROR:flask_appbuilder.security.sqla.manager:Add Permission: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_id_seq’”) [SQL: INSERT INTO ab_permission (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘can_samples’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:20,994:ERROR:flask_appbuilder.security.sqla.manager:Creation of Permission View Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_id_seq’”) [SQL: INSERT INTO ab_permission (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘can_samples’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:21,031:ERROR:flask_appbuilder.security.sqla.manager:Add View Menu Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_view_menu_id_seq’”) [SQL: INSERT INTO ab_view_menu (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘Explore’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:21,035:ERROR:flask_appbuilder.security.sqla.manager:Add View Menu Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_view_menu_id_seq’”) [SQL: INSERT INTO ab_view_menu (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘Explore’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:21,037:ERROR:flask_appbuilder.security.sqla.manager:Creation of Permission View Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_view_menu_id_seq’”) [SQL: INSERT INTO ab_view_menu (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘Explore’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:21,040:ERROR:flask_appbuilder.security.sqla.manager:Add View Menu Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_view_menu_id_seq’”) [SQL: INSERT INTO ab_view_menu (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘Explore’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:21,044:ERROR:flask_appbuilder.security.sqla.manager:Add View Menu Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_view_menu_id_seq’”) [SQL: INSERT INTO ab_view_menu (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘Explore’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:21,047:ERROR:flask_appbuilder.security.sqla.manager:Creation of Permission View Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_view_menu_id_seq’”) [SQL: INSERT INTO ab_view_menu (id, name) VALUES (%s, %s)] [parameters: [{‘name’: ‘Explore’}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-03-24 08:39:21,192:ERROR:flask_appbuilder.security.sqla.manager:Creation of Permission View Error: (sqlalchemy.exc.OperationalError) (MySQLdb.OperationalError) (4091, “Unknown SEQUENCE: ‘ab_permission_view_id_seq’”) [SQL: INSERT INTO ab_permission_view (id, permission_id, view_menu_id) VALUES (%s, %s, %s)] [parameters: [{‘permission_id’: 39, ‘view_menu_id’: 50}]] (Background on this error at: https://sqlalche.me/e/14/e3q8) Syncing role definition 2023-03-24 08:39:21,329:INFO:superset.security.manager:Syncing role definition Syncing Admin perms 2023-03-24 08:39:21,340:INFO:superset.security.manager:Syncing Admin perms Syncing Alpha perms 2023-03-24 08:39:21,456:INFO:superset.security.manager:Syncing Alpha perms Syncing Gamma perms 2023-03-24 08:39:21,590:INFO:superset.security.manager:Syncing Gamma perms Syncing granter perms 2023-03-24 08:39:21,746:INFO:superset.security.manager:Syncing granter perms Syncing sql_lab perms 2023-03-24 08:39:22,201:INFO:superset.security.manager:Syncing sql_lab perms Traceback (most recent call last): File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1880, in _execute_context self.dialect.do_executemany( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/dialects/mysql/mysqldb.py”, line 180, in do_executemany rowcount = cursor.executemany(statement, parameters) File “/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py”, line 239, in executemany self.rowcount = sum(self.execute(query, arg) for arg in args) File “/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py”, line 239, in <genexpr> self.rowcount = sum(self.execute(query, arg) for arg in args) File “/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py”, line 206, in execute res = self._query(query) File “/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py”, line 319, in _query db.query(q) File “/superset_root/lib/python3.9/site-packages/MySQLdb/connections.py”, line 254, in query _mysql.connection.query(self, query) MySQLdb.ProgrammingError: (1146, “Table ‘superset.ab_permission_view_role_id_seq’ doesn’t exist”)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File “/superset_root/bin/superset”, line 11, in <module> load_entry_point(‘apache-superset==2.1.0rc3’, ‘console_scripts’, ‘superset’)() File “/superset_root/lib/python3.9/site-packages/click/core.py”, line 1130, in call return self.main(*args, **kwargs) File “/superset_root/lib/python3.9/site-packages/flask/cli.py”, line 567, in main return super().main(*args, **kwargs) File “/superset_root/lib/python3.9/site-packages/click/core.py”, line 1055, in main rv = self.invoke(ctx) File “/superset_root/lib/python3.9/site-packages/click/core.py”, line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File “/superset_root/lib/python3.9/site-packages/click/core.py”, line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File “/superset_root/lib/python3.9/site-packages/click/core.py”, line 760, in invoke return __callback(*args, **kwargs) File “/superset_root/lib/python3.9/site-packages/click/decorators.py”, line 26, in new_func return f(get_current_context(), *args, **kwargs) File “/superset_root/lib/python3.9/site-packages/flask/cli.py”, line 407, in decorator return __ctx.invoke(f, *args, **kwargs) File “/superset_root/lib/python3.9/site-packages/click/core.py”, line 760, in invoke return __callback(*args, **kwargs) File “/superset_root/lib/python3.9/site-packages/click/decorators.py”, line 26, in new_func return f(get_current_context(), *args, **kwargs) File “/superset_root/lib/python3.9/site-packages/flask/cli.py”, line 407, in decorator return __ctx.invoke(f, *args, **kwargs) File “/superset_root/lib/python3.9/site-packages/click/core.py”, line 760, in invoke return __callback(*args, **kwargs) File “/superset_root/lib/python3.9/site-packages/superset/cli/main.py”, line 62, in init security_manager.sync_role_definitions() File “/superset_root/lib/python3.9/site-packages/superset/security/manager.py”, line 792, in sync_role_definitions self.set_role(“sql_lab”, self._is_sql_lab_pvm) File “/superset_root/lib/python3.9/site-packages/superset/security/manager.py”, line 885, in set_role self.get_session.merge(role) File “<string>”, line 2, in merge File “/superset_root/lib/python3.9/site-packages/sqlalchemy/orm/session.py”, line 3047, in merge self.autoflush() File “/superset_root/lib/python3.9/site-packages/sqlalchemy/orm/session.py”, line 2261, in autoflush util.raise(e, with_traceback=sys.exc_info()[2]) File “/superset_root/lib/python3.9/site-packages/sqlalchemy/util/compat.py”, line 211, in raise raise exception File “/superset_root/lib/python3.9/site-packages/sqlalchemy/orm/session.py”, line 2250, in _autoflush self.flush() File “/superset_root/lib/python3.9/site-packages/sqlalchemy/orm/session.py”, line 3446, in flush self._flush(objects) File “/superset_root/lib/python3.9/site-packages/sqlalchemy/orm/session.py”, line 3586, in flush transaction.rollback(capture_exception=True) File “/superset_root/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py”, line 70, in exit compat.raise( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/util/compat.py”, line 211, in raise raise exception File “/superset_root/lib/python3.9/site-packages/sqlalchemy/orm/session.py”, line 3546, in _flush flush_context.execute() File “/superset_root/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py”, line 456, in execute rec.execute(self) File “/superset_root/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py”, line 579, in execute self.dependency_processor.process_saves(uow, states) File “/superset_root/lib/python3.9/site-packages/sqlalchemy/orm/dependency.py”, line 1182, in process_saves self._run_crud( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/orm/dependency.py”, line 1245, in _run_crud connection.execute(statement, secondary_insert) File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1380, in execute return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) File “/superset_root/lib/python3.9/site-packages/sqlalchemy/sql/elements.py”, line 334, in _execute_on_connection return connection._execute_clauseelement( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1572, in _execute_clauseelement ret = self._execute_context( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1943, in _execute_context self.handle_dbapi_exception( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 2124, in handle_dbapi_exception util.raise( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/util/compat.py”, line 211, in raise raise exception File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1880, in _execute_context self.dialect.do_executemany( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/dialects/mysql/mysqldb.py”, line 180, in do_executemany rowcount = cursor.executemany(statement, parameters) File “/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py”, line 239, in executemany self.rowcount = sum(self.execute(query, arg) for arg in args) File “/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py”, line 239, in <genexpr> self.rowcount = sum(self.execute(query, arg) for arg in args) File “/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py”, line 206, in execute res = self._query(query) File “/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py”, line 319, in _query db.query(q) File “/superset_root/lib/python3.9/site-packages/MySQLdb/connections.py”, line 254, in query _mysql.connection.query(self, query) sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely) (MySQLdb.ProgrammingError) (1146, “Table ‘superset.ab_permission_view_role_id_seq’ doesn’t exist”) [SQL: INSERT INTO ab_permission_view_role (id, permission_view_id, role_id) VALUES (nextval(ab_permission_view_role_id_seq), %s, %s)] [parameters: ((147, 6), (153, 6), (63, 6), (68, 6), (66, 6), (98, 6), (114, 6), (126, 6) … displaying 10 of 14 total bound parameter sets … (151, 6), (152, 6))] (Background on this error at: https://sqlalche.me/e/14/f405)

  1. /superset_root/bin/gunicorn -w 4 -k gevent --timeout 120 -b 0.0.0.0:6666 --limit-request-line 0 --limit-request-field_size 0 “superset.app:create_app()”

  2. See error

After login to UI there is an alert: There was an issue fetching your recent activity

At this point the charts and dashboards work as expected. However when editing a chart there is a “Missing dataset Warning” offering swap dataset.

Database connection Test works as expected.

SQL Lab query returns “Database Error” Forbidden with exception being thrown…

(MySQLdb.IntegrityError) (1452, 'Cannot add or update a child row: a foreign key constraint fails (superset.tab_state, CONSTRAINT tab_state_latest_query_id_fkey FOREIGN KEY (latest_query_id) REFERENCES query (client_id) ON DELETE SET NULL)') [SQL: UPDATE tab_state SET changed_on=%s, latest_query_id=%s, changed_by_fk=%s WHERE tab_state.id = %s] [parameters: (datetime.datetime(2023, 3, 24, 8, 46, 28, 440763), 'ztiX2scW7', 7, 3)] (Background on this error at: https://sqlalche.me/e/14/gkpj) Traceback (most recent call last): File "/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context self.dialect.do_execute( File "/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute cursor.execute(statement, parameters) File "/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py", line 206, in execute res = self._query(query) File "/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py", line 319, in _query db.query(q) File "/superset_root/lib/python3.9/site-packages/MySQLdb/connections.py", line 254, in query _mysql.connection.query(self, query) MySQLdb.IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (superset.tab_state, CONSTRAINT tab_state_latest_query_id_fkey FOREIGN KEY (latest_query_id) REFERENCES query (client_id`) ON DELETE SET NULL)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File “/superset_root/lib/python3.9/site-packages/flask/app.py”, line 1517, in full_dispatch_request rv = self.dispatch_request() File “/superset_root/lib/python3.9/site-packages/flask/app.py”, line 1503, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) File “/superset_root/lib/python3.9/site-packages/flask_appbuilder/security/decorators.py”, line 181, in wraps return f(self, *args, **kwargs) File “/superset_root/lib/python3.9/site-packages/superset/views/sql_lab/views.py”, line 219, in put db.session.query(TabState).filter_by(id=tab_state_id).update(fields) File “/superset_root/lib/python3.9/site-packages/sqlalchemy/orm/query.py”, line 3306, in update result = self.session.execute( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/orm/session.py”, line 1714, in execute result = conn._execute_20(statement, params or {}, execution_options) File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1705, in _execute_20 return meth(self, args_10style, kwargs_10style, execution_options) File “/superset_root/lib/python3.9/site-packages/sqlalchemy/sql/elements.py”, line 334, in _execute_on_connection return connection._execute_clauseelement( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1572, in _execute_clauseelement ret = self._execute_context( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1943, in _execute_context self.handle_dbapi_exception( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 2124, in handle_dbapi_exception util.raise( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/util/compat.py”, line 211, in raise raise exception File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1900, in _execute_context self.dialect.do_execute( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/default.py”, line 736, in do_execute cursor.execute(statement, parameters) File “/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py”, line 206, in execute res = self._query(query) File “/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py”, line 319, in _query db.query(q) File “/superset_root/lib/python3.9/site-packages/MySQLdb/connections.py”, line 254, in query _mysql.connection.query(self, query) sqlalchemy.exc.IntegrityError: (MySQLdb.IntegrityError) (1452, ‘Cannot add or update a child row: a foreign key constraint fails (superset.tab_state, CONSTRAINT tab_state_latest_query_id_fkey FOREIGN KEY (latest_query_id) REFERENCES query (client_id) ON DELETE SET NULL)’) [SQL: UPDATE tab_state SET changed_on=%s, latest_query_id=%s, changed_by_fk=%s WHERE tab_state.id = %s] [parameters: (datetime.datetime(2023, 3, 24, 8, 46, 28, 440763), ‘ztiX2scW7’, 7, 3)] (Background on this error at: https://sqlalche.me/e/14/gkpj) 2023-03-24 08:46:28,441:ERROR:superset.views.base:(MySQLdb.IntegrityError) (1452, ‘Cannot add or update a child row: a foreign key constraint fails (superset.tab_state, CONSTRAINT tab_state_latest_query_id_fkey FOREIGN KEY (latest_query_id) REFERENCES query (client_id) ON DELETE SET NULL)’) [SQL: UPDATE tab_state SET changed_on=%s, latest_query_id=%s, changed_by_fk=%s WHERE tab_state.id = %s] [parameters: (datetime.datetime(2023, 3, 24, 8, 46, 28, 440763), ‘ztiX2scW7’, 7, 3)] (Background on this error at: https://sqlalche.me/e/14/gkpj) Traceback (most recent call last): File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1900, in _execute_context self.dialect.do_execute( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/default.py”, line 736, in do_execute cursor.execute(statement, parameters) File “/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py”, line 206, in execute res = self._query(query) File “/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py”, line 319, in _query db.query(q) File “/superset_root/lib/python3.9/site-packages/MySQLdb/connections.py”, line 254, in query _mysql.connection.query(self, query) MySQLdb.IntegrityError: (1452, ‘Cannot add or update a child row: a foreign key constraint fails (superset.tab_state, CONSTRAINT tab_state_latest_query_id_fkey FOREIGN KEY (latest_query_id) REFERENCES query (client_id) ON DELETE SET NULL)’)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File “/superset_root/lib/python3.9/site-packages/flask/app.py”, line 1517, in full_dispatch_request rv = self.dispatch_request() File “/superset_root/lib/python3.9/site-packages/flask/app.py”, line 1503, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) File “/superset_root/lib/python3.9/site-packages/flask_appbuilder/security/decorators.py”, line 181, in wraps return f(self, *args, **kwargs) File “/superset_root/lib/python3.9/site-packages/superset/views/sql_lab/views.py”, line 219, in put db.session.query(TabState).filter_by(id=tab_state_id).update(fields) File “/superset_root/lib/python3.9/site-packages/sqlalchemy/orm/query.py”, line 3306, in update result = self.session.execute( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/orm/session.py”, line 1714, in execute result = conn._execute_20(statement, params or {}, execution_options) File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1705, in _execute_20 return meth(self, args_10style, kwargs_10style, execution_options) File “/superset_root/lib/python3.9/site-packages/sqlalchemy/sql/elements.py”, line 334, in _execute_on_connection return connection._execute_clauseelement( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1572, in _execute_clauseelement ret = self._execute_context( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1943, in _execute_context self.handle_dbapi_exception( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 2124, in handle_dbapi_exception util.raise( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/util/compat.py”, line 211, in raise raise exception File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1900, in _execute_context self.dialect.do_execute( File “/superset_root/lib/python3.9/site-packages/sqlalchemy/engine/default.py”, line 736, in do_execute cursor.execute(statement, parameters) File “/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py”, line 206, in execute res = self._query(query) File “/superset_root/lib/python3.9/site-packages/MySQLdb/cursors.py”, line 319, in _query db.query(q) File “/superset_root/lib/python3.9/site-packages/MySQLdb/connections.py”, line 254, in query _mysql.connection.query(self, query) sqlalchemy.exc.IntegrityError: (MySQLdb.IntegrityError) (1452, ‘Cannot add or update a child row: a foreign key constraint fails (superset.tab_state, CONSTRAINT tab_state_latest_query_id_fkey FOREIGN KEY (latest_query_id) REFERENCES query (client_id) ON DELETE SET NULL)’) [SQL: UPDATE tab_state SET changed_on=%s, latest_query_id=%s, changed_by_fk=%s WHERE tab_state.id = %s] [parameters: (datetime.datetime(2023, 3, 24, 8, 46, 28, 440763), ‘ztiX2scW7’, 7, 3)] (Background on this error at: https://sqlalche.me/e/14/gkpj)`

Environment

(please complete the following information):

  • browser type and version: Firefox and Chrome

  • superset version: superset version

  • -=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Superset 2.1.0rc3 -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

  • python version: python --version

  • Python 3.9.2

  • node.js version: node -v

  • any feature flags active: FEATURE_FLAGS = { ‘DASHBOARD_RBAC’:True, “DASHBOARD_NATIVE_FILTERS”: True, “DASHBOARD_NATIVE_FILTERS_SET”: True, “DASHBOARD_CROSS_FILTERS”: True, “DASHBOARD_FILTERS_EXPERIMENTAL”: True, “DRILL_TO_DETAIL”:True, “ALLOW_ADHOC_SUBQUERY”: True, }

Checklist

Make sure to follow these steps before submitting your issue - thank you!

  • I have checked the superset logs for python stacktraces and included it here as text if there are any.
  • I have reproduced the issue with at least the latest released version of superset.
  • I have checked the issue tracker for the same issue and I haven’t found one similar.

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 32 (9 by maintainers)

Most upvoted comments

i’m getting same issue with postgres backend

There seems to be an issue with sqlalchemy 1.4 which was bumped in Superset 2.1.0 and MariaDB’s functionality of Sequence. Here’s a related thread: https://github.com/apache/airflow/issues/24247#issuecomment-1147581509 Some people show their workarounds for MariaDB to get around running the sequence commands. I was able to find an update in sqlalchemy-continuum thread about fixing the issue, perhaps in sqlalchemy 2.0. Let us know if for now you’re able to fix the issue with any of the workarounds from the suggestions in the airflow thread.

Just noting here that this still shows up for me trying to upgrade from 2.0.0 to 3.1.0. All four sequences are missing.

I’m also interested to know whether normal operations that does inserts into these tables would continue with the auto increment field for mariadb or start using the sequences? Because if it does, then this would break again at some point.

I’ve found another missing sequence in my upgraded 2.1.1 instance. I attempted to add the Alpha role to my account and received the following error. image

Oct 12 16:15:15 an-tool1005 superset[3608822]: 2023-10-12 16:15:15,189:ERROR:flask_appbuilder.models.sqla.interface:Edit record error: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
Oct 12 16:15:15 an-tool1005 superset[3608822]: (sqlalchemy.exc.ProgrammingError) (MySQLdb.ProgrammingError) (1146, "Table 'superset_staging.ab_user_role_id_seq' doesn't exist")
Oct 12 16:15:15 an-tool1005 superset[3608822]: [SQL: INSERT INTO ab_user_role (id, user_id, role_id) VALUES (%s, %s, %s)]
Oct 12 16:15:15 an-tool1005 superset[3608822]: [parameters: [{'user_id': 447, 'role_id': 3}]]
Oct 12 16:15:15 an-tool1005 superset[3608822]: (Background on this error at: https://sqlalche.me/e/14/f405)
Oct 12 16:15:15 an-tool1005 superset[3608822]: Traceback (most recent call last):
Oct 12 16:15:15 an-tool1005 superset[3608822]:   File "/srv/deployment/analytics/superset/venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1897, in _cursor_execute
Oct 12 16:15:15 an-tool1005 superset[3608822]:     self.dialect.do_execute(cursor, statement, parameters, context)
Oct 12 16:15:15 an-tool1005 superset[3608822]:   File "/srv/deployment/analytics/superset/venv/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
Oct 12 16:15:15 an-tool1005 superset[3608822]:     cursor.execute(statement, parameters)
Oct 12 16:15:15 an-tool1005 superset[3608822]:   File "/srv/deployment/analytics/superset/venv/lib/python3.9/site-packages/MySQLdb/cursors.py", line 179, in execute
Oct 12 16:15:15 an-tool1005 superset[3608822]:     res = self._query(mogrified_query)
Oct 12 16:15:15 an-tool1005 superset[3608822]:   File "/srv/deployment/analytics/superset/venv/lib/python3.9/site-packages/MySQLdb/cursors.py", line 330, in _query
Oct 12 16:15:15 an-tool1005 superset[3608822]:     db.query(q)
Oct 12 16:15:15 an-tool1005 superset[3608822]:   File "/srv/deployment/analytics/superset/venv/lib/python3.9/site-packages/MySQLdb/connections.py", line 255, in query
Oct 12 16:15:15 an-tool1005 superset[3608822]:     _mysql.connection.query(self, query)
Oct 12 16:15:15 an-tool1005 superset[3608822]: MySQLdb.ProgrammingError: (1146, "Table 'superset_staging.ab_user_role_id_seq' doesn't exist")

Presumably I can just create the sequence like I did for ab_permission_id_seq, ab_view_menu_id_seq, ab_permission_view_id_seq and ab_permission_view_role_id_seq, but it would be better if we could fix the broken migrations with the mariadb dialect.

Is there an easy way to check what sequences should be present?

@eschutho : Thanks for the airflow link that you have provided. It helped me to get the idea and fix the issue. i am able to upgrade superset db from 1.5.2 to 2.1.1 being mariadb as my metastore db.

Below are the steps I followed: 1. I created all the the missing sequence as below.

DROP SEQUENCE IF EXISTS ab_permission_id_seq; CREATE SEQUENCE ab_permission_id_seq start with 1 minvalue 1 maxvalue 9223372036854775806 increment by 1 cache 1000 nocycle ENGINE = InnoDB; SELECT SETVAL(ab_permission_id_seq, 105, 0);

DROP SEQUENCE IF EXISTS ab_view_menu_id_seq; CREATE SEQUENCE ab_view_menu_id_seq start with 1 minvalue 1 maxvalue 9223372036854775806 increment by 1 cache 1000 nocycle ENGINE = InnoDB; SELECT SETVAL(ab_view_menu_id_seq, 8014, 0);

DROP SEQUENCE IF EXISTS ab_permission_view_id_seq; CREATE SEQUENCE ab_permission_view_id_seq start with 1 minvalue 1 maxvalue 9223372036854775806 increment by 1 cache 1000 nocycle ENGINE = InnoDB; SELECT SETVAL(ab_permission_view_id_seq, 8195, 0);

DROP SEQUENCE IF EXISTS ab_permission_view_role_id_seq; CREATE SEQUENCE ab_permission_view_role_id_seq start with 1 minvalue 1 maxvalue 9223372036854775806 increment by 1 cache 1000 nocycle ENGINE = InnoDB; SELECT SETVAL(ab_permission_view_role_id_seq, 1238, 0);

For each table ab_permission_view, ab_permission_view_role, ab_view_menu and ab_permission I extracted the max id columns value and same I placed in the create sequence command as the start index. (else will return a duplicate value.).

Cheers !!!