airflow: Airflow fails to initdb with cattrs 1.1.0
Apache Airflow version:
1.10.12
Kubernetes version (if you are using kubernetes) (use kubectl version
):
N/A
Environment:
- Cloud provider or hardware configuration:
- OS (e.g. from /etc/os-release): Centos 7
- Kernel (e.g.
uname -a
): 3.10.0-229.el7.x86_64 - Install tools: pip 9.0.3 from /usr/lib/python3.6/site-packages (python 3.6)
- Others:
What happened:
Following the instructions here, I encountered an issue at the airflow initdb
stage:
[xxx@xxx ~]# airflow initdb
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 26, in <module>
from airflow.bin.cli import CLIFactory
File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 94, in <module>
api_module = import_module(conf.get('cli', 'api_client')) # type: Any
File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.6/site-packages/airflow/api/client/local_client.py", line 24, in <module>
from airflow.api.common.experimental import delete_dag
File "/usr/local/lib/python3.6/site-packages/airflow/api/common/experimental/delete_dag.py", line 26, in <module>
from airflow.models.serialized_dag import SerializedDagModel
File "/usr/local/lib/python3.6/site-packages/airflow/models/serialized_dag.py", line 35, in <module>
from airflow.serialization.serialized_objects import SerializedDAG
File "/usr/local/lib/python3.6/site-packages/airflow/serialization/serialized_objects.py", line 28, in <module>
import cattr
File "/usr/local/lib/python3.6/site-packages/cattr/__init__.py", line 1, in <module>
from .converters import Converter, GenConverter, UnstructureStrategy
File "/usr/local/lib/python3.6/site-packages/cattr/converters.py", line 16, in <module>
from attr import fields, resolve_types
ImportError: cannot import name 'resolve_types'
What you expected to happen:
I expected the Airflow DB to be initialised as per the instructions.
How to reproduce it:
On a fresh installation, follow the Quick Start guide until the apache initdb
stage.
Anything else we need to know:
Investigation suggests that this was caused by a Python dependency; specifically cattrs==1.1.0
, which was released yesterday (2020-10-29). Downgrading cattrs
manually to 1.0.0 does fix the issue and allows the Airflow database to be initialised:
[xxx@xxx ~]# pip3 install cattrs==1.0.0
Collecting cattrs==1.0.0
Downloading https://files.pythonhosted.org/packages/17/5b/6afbdaeb066ecf8ca28d85851048103ac80bb169491a54a14bd39823c422/cattrs-1.0.0-py2.py3-none-any.whl
Requirement already satisfied: attrs>=17.3 in /usr/local/lib/python3.6/site-packages (from cattrs==1.0.0)
Installing collected packages: cattrs
Found existing installation: cattrs 1.1.0
Uninstalling cattrs-1.1.0:
Successfully uninstalled cattrs-1.1.0
Successfully installed cattrs-1.0.0
[xxx@xxx ~]# airflow initdb
DB: sqlite:////path/to/airflow.db
[2020-10-30 09:36:05,431] {db.py:378} INFO - Creating tables
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade -> e3a246e0dc1, current schema
INFO [alembic.runtime.migration] Running upgrade e3a246e0dc1 -> 1507a7289a2f, create is_encrypted
/usr/local/lib/python3.6/site-packages/alembic/ddl/sqlite.py:44: UserWarning: Skipping unsupported ALTER for creation of implicit constraintPlease refer to the batch mode feature which allows for SQLite migrations using a copy-and-move strategy.
...
Done.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 31
- Comments: 29 (16 by maintainers)
Commits related to this issue
- Bump attrs and cattrs dependencies `cattrs` now depends on `attrs >= 20.1.0`, because of `attr.resolve_types`. Source: https://github.com/Tinche/cattrs/blob/master/HISTORY.rst#110-2020-10-29 closes... — committed to astronomer/airflow by kaxil 4 years ago
- Bump attrs and cattrs dependencies (#11969) `cattrs` now depends on `attrs >= 20.1.0`, because of `attr.resolve_types`. Source: https://github.com/Tinche/cattrs/blob/master/HISTORY.rst#110-2020-10... — committed to apache/airflow by kaxil 4 years ago
- fix airflow instrall per: https://github.com/apache/airflow/issues/11965#issuecomment-719488558 Signed-off-by: Julien Le Dem <julien@apache.org> — committed to MarquezProject/marquez-airflow by julienledem 4 years ago
- fix airflow instrall per: https://github.com/apache/airflow/issues/11965#issuecomment-719488558 (#100) Signed-off-by: Julien Le Dem <julien@apache.org> — committed to MarquezProject/marquez-airflow by julienledem 4 years ago
- Bump attrs and cattrs dependencies (#11969) `cattrs` now depends on `attrs >= 20.1.0`, because of `attr.resolve_types`. Source: https://github.com/Tinche/cattrs/blob/master/HISTORY.rst#110-2020-10... — committed to szn/airflow by kaxil 4 years ago
- Bump attrs and cattrs dependencies (#11969) `cattrs` now depends on `attrs >= 20.1.0`, because of `attr.resolve_types`. Source: https://github.com/Tinche/cattrs/blob/master/HISTORY.rst#110-2020-10... — committed to astronomer/airflow by kaxil 4 years ago
- Bump attrs and cattrs dependencies (#11969) `cattrs` now depends on `attrs >= 20.1.0`, because of `attr.resolve_types`. Source: https://github.com/Tinche/cattrs/blob/master/HISTORY.rst#110-2020-10-2... — committed to apache/airflow by kaxil 4 years ago
- Bump attrs and cattrs dependencies (#11969) `cattrs` now depends on `attrs >= 20.1.0`, because of `attr.resolve_types`. Source: https://github.com/Tinche/cattrs/blob/master/HISTORY.rst#110-2020-10-2... — committed to apache/airflow by kaxil 4 years ago
- Bump attrs and cattrs dependencies (#11969) `cattrs` now depends on `attrs >= 20.1.0`, because of `attr.resolve_types`. Source: https://github.com/Tinche/cattrs/blob/master/HISTORY.rst#110-2020-10-2... — committed to apache/airflow by kaxil 4 years ago
- Bump attrs and cattrs dependencies (#11969) `cattrs` now depends on `attrs >= 20.1.0`, because of `attr.resolve_types`. Source: https://github.com/Tinche/cattrs/blob/master/HISTORY.rst#110-2020-10-2... — committed to apache/airflow by kaxil 4 years ago
- Bump attrs and cattrs dependencies (#11969) `cattrs` now depends on `attrs >= 20.1.0`, because of `attr.resolve_types`. Source: https://github.com/Tinche/cattrs/blob/master/HISTORY.rst#110-2020-10-2... — committed to astronomer/airflow by kaxil 4 years ago
- Bump attrs and cattrs dependencies (#11969) `cattrs` now depends on `attrs >= 20.1.0`, because of `attr.resolve_types`. Source: https://github.com/Tinche/cattrs/blob/master/HISTORY.rst#110-2020-10-2... — committed to cfei18/incubator-airflow by kaxil 4 years ago
- Skips flakky airflow test on macos. Recent versions of Tensorflow and Airflow makes test fails with segmentation faults. Airflow doesn't support macos as an execution environment.[1] [1] https://git... — committed to tensorflow/tfx by deleted user 3 years ago
- Skips a flakky airflow test on macos. Recent versions of Tensorflow and Airflow makes test fails with segmentation faults. Airflow doesn't support macos as an execution environment.[1] [1] https://g... — committed to tensorflow/tfx by deleted user 3 years ago
- Bump attrs and cattrs dependencies (#11969) `cattrs` now depends on `attrs >= 20.1.0`, because of `attr.resolve_types`. Source: https://github.com/Tinche/cattrs/blob/master/HISTORY.rst#110-2020-10-2... — committed to operasoftware-osp/incubator-airflow by kaxil 4 years ago
- Skips a flakky airflow test on macos. Recent versions of Tensorflow and Airflow makes test fails with segmentation faults. Airflow doesn't support macos as an execution environment.[1] [1] https://g... — committed to tensorflow/tfx by deleted user 3 years ago
- Skips a flakky airflow test on macos. Recent versions of Tensorflow and Airflow makes test fails with segmentation faults. Airflow doesn't support macos as an execution environment.[1] [1] https://g... — committed to tensorflow/tfx by deleted user 3 years ago
You can use the following for now, this will be fixed in 1.10.13
great! I have the some question, when i use pip3 install cattrs==1.0.0. fix my problem
Just update both
attrs
andcattrs
that too should work.or just downgrade
cattrs
This issue seems to be fixed in 1.10.13.
@kaxil @potiuk ah, i’m sorry i didn’t describe it clearly 😅 the constraint file
constraints-3.8.txt
did solve theattr
and/orcattrs
issue, but in order to use it i had to do the following changes due to https://github.com/apache/airflow/issues/8467#issuecomment-720712897:original:
my updates:
It worked! Thanks!