You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "bbovenzi (via GitHub)" <gi...@apache.org> on 2024/11/21 22:11:41 UTC
[PR] Add Dag Runs list to new UI [airflow]
bbovenzi opened a new pull request, #44269:
URL: https://github.com/apache/airflow/pull/44269
Use list dag run and get dag run to render a list of dag runs, initialize a dag run page, and add an Overview button to see failed runs.
Also:
- moved TaskCountChart to its own component
- use chakra's Status component
- remove 8 hours option in time range selector. 12 is good
Still a draft because I'm having issues with the Failed Run link passing search params correctly.
<img width="730" alt="Screenshot 2024-11-21 at 5 08 46 PM" src="https://github.com/user-attachments/assets/633cfee5-afb4-44b2-8544-ff5f224065b6">
<img width="837" alt="Screenshot 2024-11-21 at 5 08 41 PM" src="https://github.com/user-attachments/assets/89b64dce-27f3-40f2-aa78-550d15683997">
<img width="1334" alt="Screenshot 2024-11-21 at 5 08 35 PM" src="https://github.com/user-attachments/assets/aafe9400-cdb2-4ae0-bc09-877de2ca11c5">
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Dag Runs list to new UI [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on PR #44269:
URL: https://github.com/apache/airflow/pull/44269#issuecomment-2515478570
Fixed the params link issue by removing Chakra's tabs and making our Dag Tabs component just a list of NavLinks instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Dag Runs list to new UI [airflow]
Posted by "tirkarthi (via GitHub)" <gi...@apache.org>.
tirkarthi commented on PR #44269:
URL: https://github.com/apache/airflow/pull/44269#issuecomment-2493178752
Since `try_number` is present in the task_instance probably a filter to get `task_instance.try_number > 1` could be added to the API to fill retried tasks section. Something like below but the color palette needs a fix since up_for_retry has gold color which is not compatible with Chakra.
```patch
diff --git a/airflow/api_fastapi/core_api/openapi/v1-generated.yaml b/airflow/api_fastapi/core_api/openapi/v1-generated.yaml
index 2f74f22689..5238ad1f91 100644
--- a/airflow/api_fastapi/core_api/openapi/v1-generated.yaml
+++ b/airflow/api_fastapi/core_api/openapi/v1-generated.yaml
@@ -3894,6 +3894,22 @@ paths:
- type: number
- type: 'null'
title: Duration Lte
+ - name: try_number_gte
+ in: query
+ required: false
+ schema:
+ anyOf:
+ - type: number
+ - type: 'null'
+ title: Try Number Gte
+ - name: try_number_lte
+ in: query
+ required: false
+ schema:
+ anyOf:
+ - type: number
+ - type: 'null'
+ title: Try Number Lte
- name: state
in: query
required: false
diff --git a/airflow/api_fastapi/core_api/routes/public/task_instances.py b/airflow/api_fastapi/core_api/routes/public/task_instances.py
index 6d5b427abb..9278ed16cb 100644
--- a/airflow/api_fastapi/core_api/routes/public/task_instances.py
+++ b/airflow/api_fastapi/core_api/routes/public/task_instances.py
@@ -277,6 +277,7 @@ def get_task_instances(
end_date_range: Annotated[RangeFilter, Depends(datetime_range_filter_factory("end_date", TI))],
update_at_range: Annotated[RangeFilter, Depends(datetime_range_filter_factory("updated_at", TI))],
duration_range: Annotated[RangeFilter, Depends(float_range_filter_factory("duration", TI))],
+ try_number: Annotated[RangeFilter, Depends(float_range_filter_factory("try_number", TI))],
state: QueryTIStateFilter,
pool: QueryTIPoolFilter,
queue: QueryTIQueueFilter,
@@ -325,6 +326,7 @@ def get_task_instances(
end_date_range,
update_at_range,
duration_range,
+ try_number,
state,
pool,
queue,
diff --git a/airflow/ui/src/pages/DagsList/Dag/Overview/Overview.tsx b/airflow/ui/src/pages/DagsList/Dag/Overview/Overview.tsx
index 1a891cf8a7..d803b89eb3 100644
--- a/airflow/ui/src/pages/DagsList/Dag/Overview/Overview.tsx
+++ b/airflow/ui/src/pages/DagsList/Dag/Overview/Overview.tsx
@@ -49,6 +49,16 @@ export const Overview = () => {
state: ["failed"],
});
+ const { data: retriedTasks, isLoading: isLoadingRetriedTasks } =
+ useTaskInstanceServiceGetTaskInstances({
+ dagId: dagId ?? "",
+ dagRunId: "~",
+ logicalDateGte: startDate,
+ logicalDateLte: endDate,
+ state: ["failed"],
+ tryNumberGte: 1,
+ });
+
const { data: failedRuns, isLoading: isLoadingRuns } =
useDagRunServiceGetDagRuns({
dagId: dagId ?? "",
@@ -77,7 +87,7 @@ export const Overview = () => {
events={(failedTasks?.task_instances ?? []).map((ti) => ({
timestamp: ti.start_date ?? ti.logical_date,
}))}
- isLoading={isLoading}
+ isLoading={isLoadingRetriedTasks}
label="Failed Task"
route={`${location.pathname}/tasks`}
startDate={startDate}
@@ -97,6 +107,18 @@ export const Overview = () => {
}}
startDate={startDate}
/>
+ <TrendCountButton
+ colorPalette={stateColor.up_for_retry}
+ count={retriedTasks?.total_entries ?? 0}
+ endDate={endDate}
+ events={(retriedTasks?.task_instances ?? []).map((ti) => ({
+ timestamp: ti.start_date ?? ti.logical_date,
+ }))}
+ isLoading={isLoading}
+ label="Retried Task"
+ route={`${location.pathname}/tasks`}
+ startDate={startDate}
+ />
</HStack>
</Box>
);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Dag Runs list to new UI [airflow]
Posted by "tirkarthi (via GitHub)" <gi...@apache.org>.
tirkarthi commented on PR #44269:
URL: https://github.com/apache/airflow/pull/44269#issuecomment-2493907448
Sure @bbovenzi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Dag Runs list to new UI [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi merged PR #44269:
URL: https://github.com/apache/airflow/pull/44269
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Dag Runs list to new UI [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on PR #44269:
URL: https://github.com/apache/airflow/pull/44269#issuecomment-2493831162
> Since `try_number` is present in the task_instance probably a filter to get `task_instance.try_number > 1` could be added to the API to fill retried tasks section. Something like below but the color palette needs a fix since up_for_retry has gold color which is not compatible with Chakra.
>
> ```diff
> diff --git a/airflow/api_fastapi/core_api/openapi/v1-generated.yaml b/airflow/api_fastapi/core_api/openapi/v1-generated.yaml
> index 2f74f22689..5238ad1f91 100644
> --- a/airflow/api_fastapi/core_api/openapi/v1-generated.yaml
> +++ b/airflow/api_fastapi/core_api/openapi/v1-generated.yaml
> @@ -3894,6 +3894,22 @@ paths:
> - type: number
> - type: 'null'
> title: Duration Lte
> + - name: try_number_gte
> + in: query
> + required: false
> + schema:
> + anyOf:
> + - type: number
> + - type: 'null'
> + title: Try Number Gte
> + - name: try_number_lte
> + in: query
> + required: false
> + schema:
> + anyOf:
> + - type: number
> + - type: 'null'
> + title: Try Number Lte
> - name: state
> in: query
> required: false
> diff --git a/airflow/api_fastapi/core_api/routes/public/task_instances.py b/airflow/api_fastapi/core_api/routes/public/task_instances.py
> index 6d5b427abb..9278ed16cb 100644
> --- a/airflow/api_fastapi/core_api/routes/public/task_instances.py
> +++ b/airflow/api_fastapi/core_api/routes/public/task_instances.py
> @@ -277,6 +277,7 @@ def get_task_instances(
> end_date_range: Annotated[RangeFilter, Depends(datetime_range_filter_factory("end_date", TI))],
> update_at_range: Annotated[RangeFilter, Depends(datetime_range_filter_factory("updated_at", TI))],
> duration_range: Annotated[RangeFilter, Depends(float_range_filter_factory("duration", TI))],
> + try_number: Annotated[RangeFilter, Depends(float_range_filter_factory("try_number", TI))],
> state: QueryTIStateFilter,
> pool: QueryTIPoolFilter,
> queue: QueryTIQueueFilter,
> @@ -325,6 +326,7 @@ def get_task_instances(
> end_date_range,
> update_at_range,
> duration_range,
> + try_number,
> state,
> pool,
> queue,
> diff --git a/airflow/ui/src/pages/DagsList/Dag/Overview/Overview.tsx b/airflow/ui/src/pages/DagsList/Dag/Overview/Overview.tsx
> index 1a891cf8a7..d803b89eb3 100644
> --- a/airflow/ui/src/pages/DagsList/Dag/Overview/Overview.tsx
> +++ b/airflow/ui/src/pages/DagsList/Dag/Overview/Overview.tsx
> @@ -49,6 +49,16 @@ export const Overview = () => {
> state: ["failed"],
> });
>
> + const { data: retriedTasks, isLoading: isLoadingRetriedTasks } =
> + useTaskInstanceServiceGetTaskInstances({
> + dagId: dagId ?? "",
> + dagRunId: "~",
> + logicalDateGte: startDate,
> + logicalDateLte: endDate,
> + state: ["failed"],
> + tryNumberGte: 1,
> + });
> +
> const { data: failedRuns, isLoading: isLoadingRuns } =
> useDagRunServiceGetDagRuns({
> dagId: dagId ?? "",
> @@ -77,7 +87,7 @@ export const Overview = () => {
> events={(failedTasks?.task_instances ?? []).map((ti) => ({
> timestamp: ti.start_date ?? ti.logical_date,
> }))}
> - isLoading={isLoading}
> + isLoading={isLoadingRetriedTasks}
> label="Failed Task"
> route={`${location.pathname}/tasks`}
> startDate={startDate}
> @@ -97,6 +107,18 @@ export const Overview = () => {
> }}
> startDate={startDate}
> />
> + <TrendCountButton
> + colorPalette={stateColor.up_for_retry}
> + count={retriedTasks?.total_entries ?? 0}
> + endDate={endDate}
> + events={(retriedTasks?.task_instances ?? []).map((ti) => ({
> + timestamp: ti.start_date ?? ti.logical_date,
> + }))}
> + isLoading={isLoading}
> + label="Retried Task"
> + route={`${location.pathname}/tasks`}
> + startDate={startDate}
> + />
> </HStack>
> </Box>
> );
> ```
Nice! Do you want to make a pull request with that after this is merged?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org