airflow: Wrong timeout value for ExternalTaskSensor running in deferrable mode

Apache Airflow version

2.10.3

If “Other Airflow 2 version” selected, which one?

No response

What happened?

The WorkflowTrigger used by ExternalTaskSensor should have a time limit set from timeout attribute instead of execution_timeout

https://github.com/apache/airflow/blob/bb234dc316f8421760255d85a37a10fd508de14f/airflow/sensors/external_task.py#L349

What you think should happen instead?

No response

How to reproduce

  1. Start an ExternalTaskSensor in deferrable mode to monitor a non-running DAG, using default arguments.
  2. The Sensor never timeout because the monitored DAGs do not update, and the timeout value is wrongly set from execution_timeout instead of timeout

Operating System

Linux

Versions of Apache Airflow Providers

No response

Deployment

Google Cloud Composer

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 17 (4 by maintainers)

Most upvoted comments

It’s partially overlap with #33718, which is making Deferrable sensor to not retry when timeout is reach.

However, even with that fix in place, the sensor still needs to call defer with the correct timeout parameter, which is not the case with ExternalTaskSensor: it’s passing execution_timeout instead of timeout