You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ephraimbuddy (via GitHub)" <gi...@apache.org> on 2025/09/24 23:43:53 UTC
[PR] UI: Fix Grid for cleared runs when tasks were removed [airflow]
ephraimbuddy opened a new pull request, #56085:
URL: https://github.com/apache/airflow/pull/56085
Ensure removed/historical tasks from selected runs are visible in Grid even if they no longer exist in the current DAG version.
We now:
- Include synthetic leaf nodes for task_ids present in TIs but missing from the serialized DAG in both grid/structure and grid/ti_summaries.
- Aggregate TI states for these synthetic nodes
Add tests covering structure and TI summaries for removed tasks.
<img width="469" height="568" alt="Screenshot 2025-09-25 at 00 40 03" src="https://github.com/user-attachments/assets/a105f4da-d240-44ec-bad8-58adf622b2a8" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Fix Grid for cleared runs when tasks were removed [airflow]
Posted by "ephraimbuddy (via GitHub)" <gi...@apache.org>.
ephraimbuddy commented on code in PR #56085:
URL: https://github.com/apache/airflow/pull/56085#discussion_r2378064453
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py:
##########
@@ -158,7 +159,7 @@ def get_dag_structure(
task_group_sort = get_task_group_children_getter()
if not run_ids:
nodes = [task_group_to_dict_grid(x) for x in task_group_sort(latest_dag.task_group)]
- return nodes
+ return [GridNodeResponse(**n) for n in nodes]
Review Comment:
This kind of changes was to make mypy happy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Fix Grid for cleared runs when tasks were removed [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on code in PR #56085:
URL: https://github.com/apache/airflow/pull/56085#discussion_r2391753074
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py:
##########
@@ -182,7 +183,37 @@ def get_dag_structure(
nodes = [task_group_to_dict_grid(x) for x in task_group_sort(dag.task_group)]
_merge_node_dicts(merged_nodes, nodes)
- return merged_nodes
+ # Ensure historical tasks (e.g. removed) that exist in TIs for the selected runs are represented
+ def _collect_ids(nodes: list[dict[str, Any]]) -> set[str]:
+ ids: set[str] = set()
+ for n in nodes:
+ nid = n.get("id")
+ if nid:
+ ids.add(nid)
+ children = n.get("children")
+ if children:
+ ids |= _collect_ids(children) # recurse
+ return ids
+
+ existing_ids = _collect_ids(merged_nodes)
+ historical_task_ids = session.scalars(
Review Comment:
nah, i guess not
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Fix Grid for cleared runs when tasks were removed [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on code in PR #56085:
URL: https://github.com/apache/airflow/pull/56085#discussion_r2391649043
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py:
##########
@@ -158,7 +159,7 @@ def get_dag_structure(
task_group_sort = get_task_group_children_getter()
if not run_ids:
nodes = [task_group_to_dict_grid(x) for x in task_group_sort(latest_dag.task_group)]
- return nodes
+ return [GridNodeResponse(**n) for n in nodes]
Review Comment:
if it's just for mypy i think you can do typing.cast
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Fix Grid for cleared runs when tasks were removed [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on PR #56085:
URL: https://github.com/apache/airflow/pull/56085#issuecomment-3352770317
Just curious, how do we end up with tasks not in serdag now? In airlfow 3 seems should not really be possible?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Fix Grid for cleared runs when tasks were removed [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #56085:
URL: https://github.com/apache/airflow/pull/56085#issuecomment-3356648924
### Backport successfully created: v3-1-test
<table>
<tr>
<th>Status</th>
<th>Branch</th>
<th>Result</th>
</tr>
<tr>
<td>✅</td>
<td>v3-1-test</td>
<td><a href="https://github.com/apache/airflow/pull/56297"><img src="https://img.shields.io/badge/PR-56297-blue" alt="PR Link"></a></td>
</tr>
</table>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Fix Grid for cleared runs when tasks were removed [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on PR #56085:
URL: https://github.com/apache/airflow/pull/56085#issuecomment-3356214218
> Just curious, how do we end up with tasks not in serdag now? In airlfow 3 seems should not really be possible?
Synthetic nodes based on TI table for other selected runs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Fix Grid for cleared runs when tasks were removed [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi merged PR #56085:
URL: https://github.com/apache/airflow/pull/56085
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Fix Grid for cleared runs when tasks were removed [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on code in PR #56085:
URL: https://github.com/apache/airflow/pull/56085#discussion_r2391678516
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py:
##########
@@ -182,7 +183,37 @@ def get_dag_structure(
nodes = [task_group_to_dict_grid(x) for x in task_group_sort(dag.task_group)]
_merge_node_dicts(merged_nodes, nodes)
- return merged_nodes
+ # Ensure historical tasks (e.g. removed) that exist in TIs for the selected runs are represented
+ def _collect_ids(nodes: list[dict[str, Any]]) -> set[str]:
+ ids: set[str] = set()
+ for n in nodes:
+ nid = n.get("id")
+ if nid:
+ ids.add(nid)
+ children = n.get("children")
+ if children:
+ ids |= _collect_ids(children) # recurse
+ return ids
+
+ existing_ids = _collect_ids(merged_nodes)
+ historical_task_ids = session.scalars(
Review Comment:
do you also have to query TaskInstanceHistory here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Fix Grid for cleared runs when tasks were removed [airflow]
Posted by "ephraimbuddy (via GitHub)" <gi...@apache.org>.
ephraimbuddy commented on PR #56085:
URL: https://github.com/apache/airflow/pull/56085#issuecomment-3333021189
Here, the entire tasks were renamed and cleared to rerun. Now UI shows complete story of that in grid too
<img width="1355" height="828" alt="Screenshot 2025-09-25 at 10 07 16" src="https://github.com/user-attachments/assets/97066a88-f2f9-422d-84cd-a156cb77a207" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org