airflow: Airflow fails to initdb with cattrs 1.1.0

Apache Airflow version:

1.10.12

Kubernetes version (if you are using kubernetes) (use kubectl version):

N/A

Environment:

  • Cloud provider or hardware configuration:
  • OS (e.g. from /etc/os-release): Centos 7
  • Kernel (e.g. uname -a): 3.10.0-229.el7.x86_64
  • Install tools: pip 9.0.3 from /usr/lib/python3.6/site-packages (python 3.6)
  • Others:

What happened:

Following the instructions here, I encountered an issue at the airflow initdb stage:

[xxx@xxx ~]# airflow initdb
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 26, in <module>
    from airflow.bin.cli import CLIFactory
  File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 94, in <module>
    api_module = import_module(conf.get('cli', 'api_client'))  # type: Any
  File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/local/lib/python3.6/site-packages/airflow/api/client/local_client.py", line 24, in <module>
    from airflow.api.common.experimental import delete_dag
  File "/usr/local/lib/python3.6/site-packages/airflow/api/common/experimental/delete_dag.py", line 26, in <module>
    from airflow.models.serialized_dag import SerializedDagModel
  File "/usr/local/lib/python3.6/site-packages/airflow/models/serialized_dag.py", line 35, in <module>
    from airflow.serialization.serialized_objects import SerializedDAG
  File "/usr/local/lib/python3.6/site-packages/airflow/serialization/serialized_objects.py", line 28, in <module>
    import cattr
  File "/usr/local/lib/python3.6/site-packages/cattr/__init__.py", line 1, in <module>
    from .converters import Converter, GenConverter, UnstructureStrategy
  File "/usr/local/lib/python3.6/site-packages/cattr/converters.py", line 16, in <module>
    from attr import fields, resolve_types
ImportError: cannot import name 'resolve_types'

What you expected to happen:

I expected the Airflow DB to be initialised as per the instructions.

How to reproduce it:

On a fresh installation, follow the Quick Start guide until the apache initdb stage.

Anything else we need to know:

Investigation suggests that this was caused by a Python dependency; specifically cattrs==1.1.0, which was released yesterday (2020-10-29). Downgrading cattrs manually to 1.0.0 does fix the issue and allows the Airflow database to be initialised:

[xxx@xxx ~]# pip3 install cattrs==1.0.0
Collecting cattrs==1.0.0
  Downloading https://files.pythonhosted.org/packages/17/5b/6afbdaeb066ecf8ca28d85851048103ac80bb169491a54a14bd39823c422/cattrs-1.0.0-py2.py3-none-any.whl
Requirement already satisfied: attrs>=17.3 in /usr/local/lib/python3.6/site-packages (from cattrs==1.0.0)
Installing collected packages: cattrs
  Found existing installation: cattrs 1.1.0
    Uninstalling cattrs-1.1.0:
      Successfully uninstalled cattrs-1.1.0
Successfully installed cattrs-1.0.0
[xxx@xxx ~]# airflow initdb            
DB: sqlite:////path/to/airflow.db
[2020-10-30 09:36:05,431] {db.py:378} INFO - Creating tables
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> e3a246e0dc1, current schema
INFO  [alembic.runtime.migration] Running upgrade e3a246e0dc1 -> 1507a7289a2f, create is_encrypted
/usr/local/lib/python3.6/site-packages/alembic/ddl/sqlite.py:44: UserWarning: Skipping unsupported ALTER for creation of implicit constraintPlease refer to the batch mode feature which allows for SQLite migrations using a copy-and-move strategy.
...
Done.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 31
  • Comments: 29 (16 by maintainers)

Commits related to this issue

Most upvoted comments

You can use the following for now, this will be fixed in 1.10.13

pip install apache-airflow==1.10.12 \
 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.7.txt"

great! I have the some question, when i use pip3 install cattrs==1.0.0. fix my problem

@jodmoreira oh how i envy you, the constraint file constraints-3.8.txt doesn’t work for my 1.10.12 😿 #8467 (comment)

Just update both attrs and cattrs that too should work.

or just downgrade cattrs

This issue seems to be fixed in 1.10.13.

@kaxil @potiuk ah, i’m sorry i didn’t describe it clearly 😅 the constraint file constraints-3.8.txt did solve the attr and/or cattrs issue, but in order to use it i had to do the following changes due to https://github.com/apache/airflow/issues/8467#issuecomment-720712897:

original:

oauthlib==3.1.0
requests-oauthlib==1.3.0

my updates:

oauthlib==2.1.0
requests-oauthlib==1.1.0

You can use the following for now, this will be fixed in 1.10.13

pip install apache-airflow==1.10.12 \
 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.7.txt"

It worked! Thanks!