airflow: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
Apache Airflow version: 1.10.10
Kubernetes version (if you are using kubernetes) (use kubectl version
): -
Environment:
- Cloud provider or hardware configuration: MacBook Pro (2019), i5, 16gp RAM
- OS (e.g. from /etc/os-release): Debian GNU/Linux 9 (stretch)
- Kernel (e.g.
uname -a
): Linux 9d5d03cba741 4.19.76-linuxkit #1 SMP Thu Oct 17 19:31:58 UTC 2019 x86_64 GNU/Linux - Install tools: Run Airflow in Docker containers
- Others: What happened: When use CeleryExecutor + Dag Serialization triggering DAGs via Web UI gets error:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python3.7/site-packages/airflow/www_rbac/decorators.py", line 121, in wrapper
return f(self, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/security/decorators.py", line 109, in wraps
return f(self, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/www_rbac/decorators.py", line 56, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 74, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/www_rbac/views.py", line 1050, in trigger
external_trigger=True
File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 74, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/models/dag.py", line 1818, in create_dagrun
return self.get_dag().create_dagrun(run_id=run_id,
AttributeError: 'NoneType' object has no attribute 'create_dagrun'
What you expected to happen: DAG can be triggered via Web UI without exception.
How to reproduce it:
- Deploy Airflow Cluster via docker-compose (example below)
- Using CeleryExecutor in separated Docker containers for webserver, scheduler, and worker
- Disable mount DAG files on webserver container and configure DAG Serialisation:
AIRFLOW__CORE__STORE_SERIALIZED_DAGS=True
AIRFLOW__CORE__MIN_SERIALIZED_DAG_UPDATE_INTERVAL=30
AIRFLOW__CORE__STORE_DAG_CODE=True
- Open Web UI and try to trigger any DAG.
- Gets error
AttributeError: 'NoneType' object has no attribute 'create_dagrun'
Anything else we need to know:
Used docker-compose.yml (see NOTE!!
comment):
version: "3.7"
services:
postgres:
container_name: airflow_postgres
image: postgres:10
environment:
POSTGRES_HOST: "postgres"
POSTGRES_PORT: "5432"
POSTGRES_DB: "airflow"
POSTGRES_USER: "airflow"
POSTGRES_PASSWORD: "airflow"
redis:
container_name: airflow_redis
image: 'redis:5.0.5'
webserver:
container_name: airflow_webserver
image: airflow-dev:latest
depends_on:
- postgres
- redis
env_file:
.env
ports:
- "8080:8080"
command: webserver
# NOTE!! If uncomment mount bellow, trigger dag via web UI works.
#volumes:
#- ./dags:/usr/local/airflow/dags
scheduler:
container_name: airflow_scheduler
image: airflow-dev:latest
depends_on:
- webserver
env_file:
.env
command: scheduler
volumes:
- ./dags:/usr/local/airflow/dags
flower:
container_name: airflow_flower
image: airflow-dev:latest
restart: always
depends_on:
- redis
env_file:
.env
environment:
CELERY_CONFIG_MODULE: airflow.config_templates.default_celery
ports:
- 5555:5555
command: flower
worker:
container_name: airflow_worker
image: airflow-dev:latest
restart: always
ports: ['8793:8793']
depends_on:
- webserver
- scheduler
env_file:
.env
command: worker -q tech
volumes:
- ./dags:/usr/local/airflow/dags
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 15 (9 by maintainers)
Commits related to this issue
- BugFix: DAG trigger via UI error in RBAC UI resolves https://github.com/apache/airflow/issues/8247 — committed to kaxil/airflow by kaxil 4 years ago
- BugFix: DAG trigger via UI error in RBAC UI (#8411) resolves https://github.com/apache/airflow/issues/8247 — committed to apache/airflow by kaxil 4 years ago
- BugFix: DAG trigger via UI error in RBAC UI (#8411) resolves https://github.com/apache/airflow/issues/8247 (cherry picked from commit cf433940d25c38ea7a3fa34ff62cd26d7226386f) — committed to astronomer/airflow by kaxil 4 years ago
- BugFix: DAG trigger via UI error in RBAC UI (#8411) resolves https://github.com/apache/airflow/issues/8247 GitOrigin-RevId: cf433940d25c38ea7a3fa34ff62cd26d7226386f — committed to GoogleCloudPlatform/composer-airflow by kaxil 4 years ago
- BugFix: DAG trigger via UI error in RBAC UI (#8411) resolves https://github.com/apache/airflow/issues/8247 GitOrigin-RevId: cf433940d25c38ea7a3fa34ff62cd26d7226386f — committed to GoogleCloudPlatform/composer-airflow by kaxil 4 years ago
- BugFix: DAG trigger via UI error in RBAC UI (#8411) resolves https://github.com/apache/airflow/issues/8247 — committed to cfei18/incubator-airflow by kaxil 4 years ago
- BugFix: DAG trigger via UI error in RBAC UI (#8411) resolves https://github.com/apache/airflow/issues/8247 GitOrigin-RevId: cf433940d25c38ea7a3fa34ff62cd26d7226386f — committed to GoogleCloudPlatform/composer-airflow by kaxil 4 years ago
- BugFix: DAG trigger via UI error in RBAC UI (#8411) resolves https://github.com/apache/airflow/issues/8247 GitOrigin-RevId: cf433940d25c38ea7a3fa34ff62cd26d7226386f — committed to GoogleCloudPlatform/composer-airflow by kaxil 4 years ago
- BugFix: DAG trigger via UI error in RBAC UI (#8411) resolves https://github.com/apache/airflow/issues/8247 GitOrigin-RevId: cf433940d25c38ea7a3fa34ff62cd26d7226386f — committed to GoogleCloudPlatform/composer-airflow by kaxil 4 years ago
- BugFix: DAG trigger via UI error in RBAC UI (#8411) resolves https://github.com/apache/airflow/issues/8247 connected to Internal bug Change-Id: Iabbaf5f8fb88c6ec1c9b3928354efbcb74ff58cc GitOrigin-R... — committed to GoogleCloudPlatform/composer-airflow by kaxil 4 years ago
- BugFix: DAG trigger via UI error in RBAC UI (#8411) resolves https://github.com/apache/airflow/issues/8247 connected to Internal bug Change-Id: Iabbaf5f8fb88c6ec1c9b3928354efbcb74ff58cc GitOrigin-R... — committed to GoogleCloudPlatform/composer-airflow by kaxil 4 years ago
- BugFix: DAG trigger via UI error in RBAC UI (#8411) resolves https://github.com/apache/airflow/issues/8247 connected to Internal bug Change-Id: I8af7e2be75ca37c9b0ea24debccd40e90169482d GitOrigin-R... — committed to GoogleCloudPlatform/composer-airflow by deleted user 4 years ago
- BugFix: DAG trigger via UI error in RBAC UI (#8411) resolves https://github.com/apache/airflow/issues/8247 connected to Internal bug Change-Id: Iabbaf5f8fb88c6ec1c9b3928354efbcb74ff58cc GitOrigin-R... — committed to GoogleCloudPlatform/composer-airflow by kaxil 4 years ago
- BugFix: DAG trigger via UI error in RBAC UI (#8411) resolves https://github.com/apache/airflow/issues/8247 connected to Internal bug Change-Id: Iabbaf5f8fb88c6ec1c9b3928354efbcb74ff58cc GitOrigin-R... — committed to GoogleCloudPlatform/composer-airflow by kaxil 4 years ago
- BugFix: DAG trigger via UI error in RBAC UI (#8411) resolves https://github.com/apache/airflow/issues/8247 connected to Internal bug Change-Id: Iabbaf5f8fb88c6ec1c9b3928354efbcb74ff58cc GitOrigin-R... — committed to GoogleCloudPlatform/composer-airflow by kaxil 4 years ago
Meanwhile, can you try the following docker image and let me know if that solves the issue for you:
Only use the non-rbac UI