You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "hterik (via GitHub)" <gi...@apache.org> on 2023/10/03 05:51:37 UTC
[I] Links to specific dagruns don't work when older than 25 runs [airflow]
hterik opened a new issue, #34723:
URL: https://github.com/apache/airflow/issues/34723
### Apache Airflow version
2.7.1
### What happened
1. Browse to `AIRFLOW/dagrun/list/?_flt_3_dag_id=my_dag`
2. Click any link from the **Run Id** column, below the most recent 25 runs.
It should take you to `AIRFLOW/dags/mydag/graph?run_id=scheduled__2023-09-21T00%3A00%3A00%2B00%3A00&execution_date=2023-09-21+00%3A00%3A00%2B00%3A00` and it does so correctly
3. The graph view opens but **no run is selected.**
When clicking on any item in the graph, the LATEST run is selected in the grid. If the user is not careful to observe this, they are mislead to thinking the results they are browsing in the graph is the ones they selected in step 2.
As workaround, one can append `&num_runs=365` to the url, this will take you to the correct run from step 2. But eventually you will exhaust 365 runs as well.
This problem isn't only applicable to the dagrun list. Any links to the /dags/ graph are affected. We have external systems linking to individual dagruns that now get unusable after 25 runs.
### What you think should happen instead
When navigating to a run via link, it should **always** present the run that was provided in run_id querystring.
### How to reproduce
See above
### Operating System
Debian GNU/Linux 11 (bullseye)
### Versions of Apache Airflow Providers
_No response_
### Deployment
Other Docker-based deployment
### Deployment details
_No response_
### Anything else
This was not a problem in Airflow 2.5, regression in 2.7, probably introduced by new embedded graph view (which otherwise is great :))
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "QuintenBruynseraede (via GitHub)" <gi...@apache.org>.
QuintenBruynseraede commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-1805875502
Also eagerly waiting for this fix. For the time being, is there a workaround to filter + navigate to a dagrun?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "hterik (via GitHub)" <gi...@apache.org>.
hterik commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-1748051842
I've done some debugging:
**Problem 1.**
Dag list is linking to the legacy `/graph` endpoint. This endpoint is only for backwards compatibility purposes and redirects to the new `/grid` endpoint.
This old function does actually calculate the `base_date` based on `run_id`, inside `dt_nr_dr_data = get_date_time_num_runs_dag_runs_form_data(request, session, dag)`
https://github.com/apache/airflow/blob/0c8e30e43b70e9d033e1686b327eb00aab82479c/airflow/www/views.py#L216-230
But then it forgets to forward the `dt_nr_dr_data["base_date"]` in the returned kwargs:
https://github.com/apache/airflow/blob/0c8e30e43b70e9d033e1686b327eb00aab82479c/airflow/www/views.py#L2964-2969
SolutionA: Forward base_date
SolutionB: Change dag list to use new url
**Problem 2**
The new grid endpoint, receives the `run_id` parameter correctly, after the redirection described above.
However, it then performs a Javascript fetch call to the `grid_data` API. In this API call, the `run_id` is NOT included.
The `grid_data` function that serves this request has not implemented handling of run_id parameter, in the same way that the old `get_date_time_num_runs_dag_runs_form_data` function did.
https://github.com/apache/airflow/blob/0c8e30e43b70e9d033e1686b327eb00aab82479c/airflow/www/views.py#L3475-3478
Solution C: Implement run_id handling in grid_data and forward the parameter from the frontend.
I think both solution A and C are must haves here. B is optional, more nice to have to get rid of legacy in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "shlomiken (via GitHub)" <gi...@apache.org>.
shlomiken commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-2713420675
This still happens in 2.9.1 - very frustrating experience troubleshooting a failed fag
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "jermaine151 (via GitHub)" <gi...@apache.org>.
jermaine151 commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-1878598255
> Thanks @jermaine151, that seems to work indeed. It doesn't visually highlight a dagrun in the grid view (see Screenshot), but if you navigate to the Graph tab, then click on a Task it does lead you to the dagrun from the URL.
>
> 
>
> That last bit is important because in our case multiple runs are triggered on the same second, the gridview doesn't distinguish between those.
Are you adding the `&dag_run_id=` to the URL? That should land you on the right page, with the run selected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "hterik (via GitHub)" <gi...@apache.org>.
hterik commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-1784830804
Sorry for pinging but is there any way we can move this forward? I've already provided a fix in https://github.com/apache/airflow/pull/34887, waiting for review.
IMO not being able to permalink to specific dagruns is quite severe bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "QuintenBruynseraede (via GitHub)" <gi...@apache.org>.
QuintenBruynseraede commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-1873976696
Thanks @jermaine151, that seems to work indeed. It doesn't visually highligt a dagrun in the grid view (see Screenshot), but if you navigate to the Graph tab, then click on a Task it does lead you to the dagrun from the URL.
That last bit is important because in our case multiple runs are triggered on the same second, the gridview doesn't distinguish between those.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-1744840318
This date is the start date for the X dag runs, when you choose 2023-01-01 as start date, and 25 as a number of runs, the UI will show you the next 25 runs since 2023-01-01. To select a dag run, you need to click on the red/green bar in the grid view.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-1901867882
cc: @bbovenzi @pierrejeambrun - would be cool to take a look and fix that one as it seems to be an important feature for those who complain about loosing the old grid view (see #36884) and it seems a bit of an architectural decision on the grid view behaviour.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #34723: Links to specific dagruns don't work when older than 25 runs
URL: https://github.com/apache/airflow/issues/34723
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-1744378524
I cannot reproduce with main:
Could you provide some screenshots for your problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "hterik (via GitHub)" <gi...@apache.org>.
hterik commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-1744877344
This was not the case in Airflow 2.5, i could click on links referring to any dagrun, no matter how old, and it would correctly select it in the grid-view.
The selection of date and number of runs shouldn't have to be provided manually if run_id is part of the url.
To clarify, i am browsing the dagrun list attached below and clicking an item in the picture 1 here, then it results in the picture 2 (same as above) where no run is selected:
1.
2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-4897203401
Fixed in https://github.com/apache/airflow/pull/65388
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "AlexandreGCastor (via GitHub)" <gi...@apache.org>.
AlexandreGCastor commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-1750149249
Hello,
This was Ok in airflow 2.6.2. This possibility to get logs on the history (more than 25 run ago) is very important for us. It's way too hard to hack the url each time for us. We would probably rollback waiting for this issue.
Alexandre Gué,
Lead DevOps CastorDoc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "jermaine151 (via GitHub)" <gi...@apache.org>.
jermaine151 commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-1873545771
I found a workaround to this problem in v2.7.3. If you add `&base_date=<same as execution date>` to the URL, it will locate runs older than the latest 25. Here's an example link that is working for me:
`<base_url>/dags/<dag_id>/grid?execution_date=2023-12-14+19%3A10%3A24.218280%2B00%3A00&base_date=2023-12-14+19%3A10%3A24.218280%2B00%3A00&tab=graph&dag_run_id=<run_id>`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "jens-scheffler-bosch (via GitHub)" <gi...@apache.org>.
jens-scheffler-bosch commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-1745571114
I did a test with an example DAG, triggered this manually >25 times. As the GRID view shows 25 items per default all is fine if you click on one of the recent 25 items.
If you click on an item >25 runs, GRID view will not adjust the selection of the 25runs to include in the column section your run. In my setup it shows the grid view, but clicking on the "Details" tab it shows the DAG overview, not details of the (previously) selected DAG runs. So in total it is hard to navigate from a DAG search result to the GRID view t check for details. (Especially because in GRID view the search options are limited compared to the DAG search screen)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "hterik (via GitHub)" <gi...@apache.org>.
hterik commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-1744403887
Sure, hope this is enough
The url was given by clicking a run in the **List dag runs** page.
Something interesting to observe is that after clicking the url, it redirects 2 times, first adding the `dag_run_id` parameter (not `run_id`), which then quickly gets removed again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Links to specific dagruns don't work when older than 25 runs [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #34723:
URL: https://github.com/apache/airflow/issues/34723#issuecomment-2718739667
Unlikely Anything will be done about it - Airflow 3 is behind the corner with completely new UI,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org