airflow: AssertionError: Dependency rule tried to blank-out primary key column 'ab_permission.id' on instance '

Apache Airflow version

main (development)

What happened

airflow webserver cannot start due to error

Traceback (most recent call last):
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/gunicorn/util.py", line 412, in import_app
    app = app(*args, **kwargs)
  File "/Users/ping_zhang/airlab/repos/airflow/airflow/www/app.py", line 149, in cached_app
    app = create_app(config=config, testing=testing)
  File "/Users/ping_zhang/airlab/repos/airflow/airflow/www/app.py", line 136, in create_app
    sync_appbuilder_roles(flask_app)
  File "/Users/ping_zhang/airlab/repos/airflow/airflow/www/app.py", line 66, in sync_appbuilder_roles
    flask_app.appbuilder.sm.sync_roles()
  File "/Users/ping_zhang/airlab/repos/airflow/airflow/www/security.py", line 544, in sync_roles
    self.clean_perms()
  File "/Users/ping_zhang/airlab/repos/airflow/airflow/www/security.py", line 406, in clean_perms
    sesh.commit()
  File "<string>", line 2, in commit
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1423, in commit
    self._transaction.commit(_to_root=self.future)
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 829, in commit
    self._prepare_impl()
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl
    self.session.flush()
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3255, in flush
    self._flush(objects)
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3395, in _flush
    transaction.rollback(_capture_exception=True)
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    compat.raise_(
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3355, in _flush
    flush_context.execute()
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 453, in execute
    rec.execute(self)
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 574, in execute
    self.dependency_processor.process_deletes(uow, states)
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/sqlalchemy/orm/dependency.py", line 552, in process_deletes
    self._synchronize(
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/sqlalchemy/orm/dependency.py", line 610, in _synchronize
    sync.clear(dest, self.mapper, self.prop.synchronize_pairs)
  File "/Users/ping_zhang/miniforge3/envs/apache-airflow/lib/python3.9/site-packages/sqlalchemy/orm/sync.py", line 86, in clear
    raise AssertionError(
AssertionError: Dependency rule tried to blank-out primary key column 'ab_view_menu.id' on instance '<Resource at 0x111da6b50>'

What you expected to happen

airflow webserver cannot start due to error

How to reproduce

Error in CI: https://github.com/apache/airflow/runs/5101435727?check_suite_focus=true#step:7:2424

Error from my local:

i have the latest airflow main commit: 471e368eacbcae1eedf9b7e1cb4290c385396ea9

python version: 3.9.7 Flask related pkg:

❯ pip list | grep Flas
Flask                           1.1.4
Flask-AppBuilder                3.4.4
Flask-Babel                     2.0.0
Flask-Caching                   1.10.1
Flask-JWT-Extended              3.25.1
Flask-Login                     0.4.1
Flask-OpenID                    1.3.0
Flask-SQLAlchemy                2.5.1
Flask-WTF                       0.14.3

❯ pip list | grep SQ
Flask-SQLAlchemy                2.5.1
SQLAlchemy                      1.4.9
SQLAlchemy-JSONField            1.0.0
SQLAlchemy-Utils                0.38.2

database: mysql

mysql version:

❯ mysql -V
mysql  Ver 8.0.28 for macos12.0 on arm64 (Homebrew)

the aiflow.cfg and webserver conf are auto generated, i did not change except sql_alchemy_conn

Operating System

Apple M1 Max, version: 12.2

Versions of Apache Airflow Providers

No response

Deployment

Other

Deployment details

local dev

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project’s Code of Conduct