airflow: Links to specific dagruns don't work when older than 25 runs

Apache Airflow version

2.7.1

What happened

  1. Browse to AIRFLOW/dagrun/list/?_flt_3_dag_id=my_dag
  2. Click any link from the Run Id column, below the most recent 25 runs. It should take you to AIRFLOW/dags/mydag/graph?run_id=scheduled__2023-09-21T00%3A00%3A00%2B00%3A00&execution_date=2023-09-21+00%3A00%3A00%2B00%3A00 and it does so correctly
  3. The graph view opens but no run is selected. When clicking on any item in the graph, the LATEST run is selected in the grid. If the user is not careful to observe this, they are mislead to thinking the results they are browsing in the graph is the ones they selected in step 2.

As workaround, one can append &num_runs=365 to the url, this will take you to the correct run from step 2. But eventually you will exhaust 365 runs as well.

This problem isn’t only applicable to the dagrun list. Any links to the /dags/ graph are affected. We have external systems linking to individual dagruns that now get unusable after 25 runs.

What you think should happen instead

When navigating to a run via link, it should always present the run that was provided in run_id querystring.

How to reproduce

See above

Operating System

Debian GNU/Linux 11 (bullseye)

Versions of Apache Airflow Providers

No response

Deployment

Other Docker-based deployment

Deployment details

No response

Anything else

This was not a problem in Airflow 2.5, regression in 2.7, probably introduced by new embedded graph view (which otherwise is great 😃)

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 16
  • Comments: 15 (9 by maintainers)

Commits related to this issue

Most upvoted comments

Also eagerly waiting for this fix. For the time being, is there a workaround to filter + navigate to a dagrun?

Sorry for pinging but is there any way we can move this forward? I’ve already provided a fix in https://github.com/apache/airflow/pull/34887, waiting for review. IMO not being able to permalink to specific dagruns is quite severe bug.

Hello,

This was Ok in airflow 2.6.2. This possibility to get logs on the history (more than 25 run ago) is very important for us. It’s way too hard to hack the url each time for us. We would probably rollback waiting for this issue.

Alexandre Gué, Lead DevOps CastorDoc

I did a test with an example DAG, triggered this manually >25 times. As the GRID view shows 25 items per default all is fine if you click on one of the recent 25 items.

If you click on an item >25 runs, GRID view will not adjust the selection of the 25runs to include in the column section your run. In my setup it shows the grid view, but clicking on the “Details” tab it shows the DAG overview, not details of the (previously) selected DAG runs. So in total it is hard to navigate from a DAG search result to the GRID view t check for details. (Especially because in GRID view the search options are limited compared to the DAG search screen)

This still happens in 2.9.1 - very frustrating experience troubleshooting a failed fag

cc: @bbovenzi @pierrejeambrun - would be cool to take a look and fix that one as it seems to be an important feature for those who complain about loosing the old grid view (see #36884) and it seems a bit of an architectural decision on the grid view behaviour.

Thanks @jermaine151, that seems to work indeed. It doesn’t visually highlight a dagrun in the grid view (see Screenshot), but if you navigate to the Graph tab, then click on a Task it does lead you to the dagrun from the URL.

image

That last bit is important because in our case multiple runs are triggered on the same second, the gridview doesn’t distinguish between those.

I found a workaround to this problem in v2.7.3. If you add &base_date=<same as execution date> to the URL, it will locate runs older than the latest 25. Here’s an example link that is working for me:

<base_url>/dags/<dag_id>/grid?execution_date=2023-12-14+19%3A10%3A24.218280%2B00%3A00&base_date=2023-12-14+19%3A10%3A24.218280%2B00%3A00&tab=graph&dag_run_id=<run_id>