airflow: Links to specific dagruns don't work when older than 25 runs
Apache Airflow version
2.7.1
What happened
- Browse to
AIRFLOW/dagrun/list/?_flt_3_dag_id=my_dag - Click any link from the Run Id column, below the most recent 25 runs.
It should take you to
AIRFLOW/dags/mydag/graph?run_id=scheduled__2023-09-21T00%3A00%3A00%2B00%3A00&execution_date=2023-09-21+00%3A00%3A00%2B00%3A00and it does so correctly - The graph view opens but no run is selected. When clicking on any item in the graph, the LATEST run is selected in the grid. If the user is not careful to observe this, they are mislead to thinking the results they are browsing in the graph is the ones they selected in step 2.
As workaround, one can append &num_runs=365 to the url, this will take you to the correct run from step 2. But eventually you will exhaust 365 runs as well.
This problem isn’t only applicable to the dagrun list. Any links to the /dags/ graph are affected. We have external systems linking to individual dagruns that now get unusable after 25 runs.
What you think should happen instead
When navigating to a run via link, it should always present the run that was provided in run_id querystring.
How to reproduce
See above
Operating System
Debian GNU/Linux 11 (bullseye)
Versions of Apache Airflow Providers
No response
Deployment
Other Docker-based deployment
Deployment details
No response
Anything else
This was not a problem in Airflow 2.5, regression in 2.7, probably introduced by new embedded graph view (which otherwise is great 😃)
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project’s Code of Conduct
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 16
- Comments: 15 (9 by maintainers)
Commits related to this issue
- Base date for fetching dag grid view must include selected run_id Previously, if user set dag_run_id parameter in the url, that refers to a old run, which doesn't fit in the most recent 25 runs, then... — committed to hterik/airflow by hterik 3 years ago
- Base date for fetching dag grid view must include selected run_id (#34887) Previously, if user set dag_run_id parameter in the url, that refers to a old run, which doesn't fit in the most recent 25 ... — committed to apache/airflow by hterik 3 years ago
- Base date for fetching dag grid view must include selected run_id (#34887) Previously, if user set dag_run_id parameter in the url, that refers to a old run, which doesn't fit in the most recent 25 r... — committed to apache/airflow by hterik 3 years ago
Also eagerly waiting for this fix. For the time being, is there a workaround to filter + navigate to a dagrun?
Sorry for pinging but is there any way we can move this forward? I’ve already provided a fix in https://github.com/apache/airflow/pull/34887, waiting for review. IMO not being able to permalink to specific dagruns is quite severe bug.
Hello,
This was Ok in airflow 2.6.2. This possibility to get logs on the history (more than 25 run ago) is very important for us. It’s way too hard to hack the url each time for us. We would probably rollback waiting for this issue.
Alexandre Gué, Lead DevOps CastorDoc
I did a test with an example DAG, triggered this manually >25 times. As the GRID view shows 25 items per default all is fine if you click on one of the recent 25 items.
If you click on an item >25 runs, GRID view will not adjust the selection of the 25runs to include in the column section your run. In my setup it shows the grid view, but clicking on the “Details” tab it shows the DAG overview, not details of the (previously) selected DAG runs. So in total it is hard to navigate from a DAG search result to the GRID view t check for details. (Especially because in GRID view the search options are limited compared to the DAG search screen)
This still happens in 2.9.1 - very frustrating experience troubleshooting a failed fag
cc: @bbovenzi @pierrejeambrun - would be cool to take a look and fix that one as it seems to be an important feature for those who complain about loosing the old grid view (see #36884) and it seems a bit of an architectural decision on the grid view behaviour.
Thanks @jermaine151, that seems to work indeed. It doesn’t visually highlight a dagrun in the grid view (see Screenshot), but if you navigate to the Graph tab, then click on a Task it does lead you to the dagrun from the URL.
That last bit is important because in our case multiple runs are triggered on the same second, the gridview doesn’t distinguish between those.
I found a workaround to this problem in v2.7.3. If you add
&base_date=<same as execution date>to the URL, it will locate runs older than the latest 25. Here’s an example link that is working for me:<base_url>/dags/<dag_id>/grid?execution_date=2023-12-14+19%3A10%3A24.218280%2B00%3A00&base_date=2023-12-14+19%3A10%3A24.218280%2B00%3A00&tab=graph&dag_run_id=<run_id>