You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "yuseok89 (via GitHub)" <gi...@apache.org> on 2026/07/26 13:13:01 UTC
[PR] [WIP] UI: Add a time-window scope to the Dags list run state filter [airflow]
yuseok89 opened a new pull request, #70470:
URL: https://github.com/apache/airflow/pull/70470
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
<!--
Thank you for contributing!
Please provide above a brief description of the changes made in this pull request.
Write a good git commit message following this guide: https://chris.beams.io/posts/git-commit/
Please make sure that your code changes are covered with tests.
And in case of new features or big changes remember to adjust the documentation.
For user-facing UI changes, please attach before/after screenshots (or a short
screen recording) so reviewers can assess the visual impact.
Feel free to ping (in general) for the review if you do not see reaction for a few days
(72 Hours is the minimum reaction time you can expect from volunteers) - we sometimes miss notifications.
In case of an existing issue, reference it using one of the following:
* closes: #ISSUE
* related: #ISSUE
-->
Follow-up to #69875, the run-state filter UX refactor that reviewers asked to split into its own PR.
The Dags list had two separate run-state filters ("Last run" and "Any run") side by side. This merges them into a single Run state control with a scope selector, and adds an optional time window so the run-state filters can target a recent period instead of an unbounded run history.
related: #67094
### What changed
- UI: The two run-state selects become one `Run state` select plus a scope: `Latest run` / `Last 24 hours` / `Last 7 days` / `Last 30 days` / `Any time`. Picking a state defaults the scope to Latest run, so today's behavior is unchanged. The scope maps onto the existing `last_dag_run_state` / `dag_run_state` params; a time window adds `dag_run_state_within` (hours).
- Backend: `_AnyDagRunStateFilter` gains an optional `dag_run_state_within_hours` bound, applied inside the existing correlated `EXISTS`.
### ⚠️ Performance validation still pending
Marking this [WIP] because the new time bound isn't perf-validated yet. The `run_after` bound isn't covered by the existing `dag_id_state` index, so before this merges I want to measure the windowed filter on a large `dag_run` table (millions of rows) and see whether an index change is needed. I'll post the numbers here and lift the WIP.
### Demo
Screen recording to follow.
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR, please
change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
-->
- [x] Yes (please specify the tool below)
- Opus 4.8
<!--
Generated-by: [Tool Name] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
---
* Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "yuseok89 (via GitHub)" <gi...@apache.org>.
yuseok89 commented on code in PR #70470:
URL: https://github.com/apache/airflow/pull/70470#discussion_r4138938759
##########
airflow-core/src/airflow/ui/src/components/FilterBar/filters/runStateParams.ts:
##########
@@ -0,0 +1,71 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import { SearchParamsKeys } from "src/constants/searchParams";
+
+import type { FilterValue } from "../types";
+
+// "latest" matches on the latest run only; numeric values are hours; "any" has no time bound.
+export type RunStateLookback = "168" | "24" | "720" | "any" | "latest";
+
+export const TIME_LOOKBACKS: ReadonlyArray<RunStateLookback> = ["24", "168", "720"];
+export const RUN_STATE_LOOKBACKS: ReadonlyArray<RunStateLookback> = ["latest", ...TIME_LOOKBACKS, "any"];
+
+export type RunStateValue = {
+ lookback: RunStateLookback;
+ state: string;
+};
+
+export const isRunStateValue = (value: FilterValue): value is RunStateValue =>
+ typeof value === "object" && value !== null && !Array.isArray(value) && "state" in value;
+
+// The unified Run state pill is backed by the existing independent URL params:
+// "latest" lookback → last_dag_run_state (match the latest run only)
+// a time lookback → dag_run_state (match any run) + a within-hours bound
+// "any" → dag_run_state with no time bound
+export const runStateToSearchParams = (value: FilterValue): Record<string, string | undefined> => {
+ const selection = isRunStateValue(value) ? value : undefined;
+ const isLatest = selection?.lookback === "latest";
+ const isTimeBounded = selection !== undefined && TIME_LOOKBACKS.includes(selection.lookback);
+
+ return {
+ [SearchParamsKeys.DAG_RUN_STATE]: selection !== undefined && !isLatest ? selection.state : undefined,
+ [SearchParamsKeys.DAG_RUN_STATE_WITHIN_HOURS]: isTimeBounded ? selection.lookback : undefined,
+ [SearchParamsKeys.LAST_DAG_RUN_STATE]: isLatest ? selection.state : undefined,
+ };
+};
+
+export const runStateFromSearchParams = (params: URLSearchParams): RunStateValue | undefined => {
+ const lastRunState = params.get(SearchParamsKeys.LAST_DAG_RUN_STATE);
+ const anyRunState = params.get(SearchParamsKeys.DAG_RUN_STATE);
+ const withinHours = params.get(SearchParamsKeys.DAG_RUN_STATE_WITHIN_HOURS);
+
+ // The latest-run param wins when both are present, matching how the list endpoint treats them.
+ if (lastRunState !== null && lastRunState !== "") {
+ return { lookback: "latest", state: lastRunState };
+ }
+ if (anyRunState === null || anyRunState === "") {
+ return undefined;
+ }
+
+ const lookback = (TIME_LOOKBACKS as ReadonlyArray<string>).includes(withinHours ?? "")
Review Comment:
Done, same approach as above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "yuseok89 (via GitHub)" <gi...@apache.org>.
yuseok89 commented on code in PR #70470:
URL: https://github.com/apache/airflow/pull/70470#discussion_r3744076984
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagsList.tsx:
##########
@@ -314,6 +316,7 @@ export const DagsList = () => {
dagDisplayNamePattern: Boolean(dagDisplayNamePattern) ? dagDisplayNamePattern : undefined,
dagRunsLimit,
dagRunState,
+ dagRunStateWithinHours: Boolean(dagRunState) && withinHours !== null ? Number(withinHours) : undefined,
Review Comment:
Thanks.
Added a guard so only a valid positive number is used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "yuseok89 (via GitHub)" <gi...@apache.org>.
yuseok89 commented on PR #70470:
URL: https://github.com/apache/airflow/pull/70470#issuecomment-5406777240
Rebased onto main.
#71731 changed the filter architecture quite a bit, so I reworked the run state filter as a single FilterBar pill. I also dropped the filter configs and i18n keys that became unused, let me know if that cleanup feels too aggressive.
I will update the PR description to match.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "yuseok89 (via GitHub)" <gi...@apache.org>.
yuseok89 commented on code in PR #70470:
URL: https://github.com/apache/airflow/pull/70470#discussion_r3744096623
##########
airflow-core/src/airflow/ui/public/i18n/locales/en/dags.json:
##########
@@ -24,6 +24,15 @@
"paused": "Paused"
},
"runIdPatternFilter": "Search Dag Runs",
+ "runScope": {
+ "anyTime": "Any time",
+ "in": "in",
+ "last7Days": "Last 7 days",
+ "last24Hours": "Last 24 hours",
+ "last30Days": "Last 30 days",
+ "latestRun": "Latest run"
+ },
+ "runState": "Run state",
Review Comment:
Thanks.
Moved the labels into the shared `common:timeRange` namespace.
Reused the existing `last24Hours` and added the missing `last7Days`/`last30Days`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "yuseok89 (via GitHub)" <gi...@apache.org>.
yuseok89 commented on code in PR #70470:
URL: https://github.com/apache/airflow/pull/70470#discussion_r3744222804
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagsFilters/RunStateLookbackSelect.tsx:
##########
@@ -0,0 +1,91 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import { HStack, Text, type Select as ChakraSelect } from "@chakra-ui/react";
+import { createListCollection } from "@chakra-ui/react/collection";
+import { useTranslation } from "react-i18next";
+import { FiClock, FiZap } from "react-icons/fi";
+
+import { Select } from "src/components/ui";
+
+// "latest" matches on the latest run only; numeric values are hours; "any" has no time bound.
+export type RunStateScope = "168" | "24" | "720" | "any" | "latest";
+
+type Props = {
+ readonly dataTestId?: string;
+ readonly onChange: (value: RunStateScope) => void;
+ readonly triggerProps?: ChakraSelect.TriggerProps;
+ readonly value: RunStateScope;
+};
+
+const SCOPE_OPTIONS: ReadonlyArray<{ labelKey: string; value: RunStateScope }> = [
+ { labelKey: "latestRun", value: "latest" },
+ { labelKey: "last24Hours", value: "24" },
+ { labelKey: "last7Days", value: "168" },
+ { labelKey: "last30Days", value: "720" },
Review Comment:
The time-range labels are now reused from the shared `common:timeRange` namespace.
Making the ranges configurable is a nice idea. I was thinking it might be better as a separate follow-up rather than part of this PR. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "yuseok89 (via GitHub)" <gi...@apache.org>.
yuseok89 commented on code in PR #70470:
URL: https://github.com/apache/airflow/pull/70470#discussion_r4138931709
##########
airflow-core/src/airflow/ui/src/components/FilterBar/filters/RunStateFilter.tsx:
##########
@@ -0,0 +1,234 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import { useRef, useState } from "react";
+
+import { Box, HStack, createListCollection } from "@chakra-ui/react";
+import { useTranslation } from "react-i18next";
+import { FiClock, FiZap } from "react-icons/fi";
+
+import { Select } from "src/system-components";
+
+import { FilterPill } from "../FilterPill";
+import type { FilterConfig, FilterPluginProps } from "../types";
+import { RUN_STATE_LOOKBACKS, isRunStateValue, type RunStateLookback } from "./runStateParams";
+
+const LOOKBACK_LABEL_KEYS: Record<RunStateLookback, string> = {
+ "24": "last24Hours",
+ "168": "last7Days",
+ "720": "last30Days",
+ any: "anyTime",
+ latest: "latestRun",
+};
+
+type SelectOption = {
+ label: string;
+ value: string;
+};
+
+type RunStateFilterConfig = {
+ options: Array<SelectOption>;
+};
+
+const collapsePill = () => {
+ setTimeout(() => {
+ const activeElement = document.activeElement as HTMLElement;
+
+ activeElement.blur();
+ }, 0);
+};
+
+/**
+ * One pill filtering Dags on a run state matched within a lookback: the latest run only, a
+ * recent time window, or the whole run history. Committed as a composite value that the
+ * config's projections spread over the underlying URL params.
+ */
+export const RunStateFilter = ({ filter, onChange, onRemove }: FilterPluginProps) => {
+ const { t: translate } = useTranslation(["common"]);
+ const config = filter.config as FilterConfig & RunStateFilterConfig;
+ const value = isRunStateValue(filter.value) ? filter.value : undefined;
+
+ // Holds a lookback picked before any state is chosen, when there is nothing to commit yet.
+ const [pendingLookback, setPendingLookback] = useState<RunStateLookback>(value?.lookback ?? "latest");
+
+ // Selecting an option closes the menu in the same tick as the value commits, before React has
+ // re-rendered. Recording the selection keeps the close from being mistaken for an abandoned
+ // filter, which a timing-based check gets wrong.
+ const hasJustSelected = useRef(false);
+
+ const handleStateChange = ({ value: selected }: { value: Array<string> }) => {
+ const [newState] = selected;
+
+ if (newState === undefined) {
+ return;
+ }
+ hasJustSelected.current = true;
+ onChange({ lookback: value?.lookback ?? pendingLookback, state: newState });
+ collapsePill();
+ };
+
+ const handleLookbackChange = ({ value: selected }: { value: Array<string> }) => {
+ const [newLookback] = selected as Array<RunStateLookback>;
Review Comment:
Done, thanks!
I used a `Set` for the guards instead of `.some(...)`.
Our `prefer-includes` lint rule autofixes `.some((lookback) => lookback === value)` into `.includes(value)`, which then fails type checking since `value` is a `string`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on code in PR #70470:
URL: https://github.com/apache/airflow/pull/70470#discussion_r3722892232
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagsList.tsx:
##########
@@ -314,6 +316,7 @@ export const DagsList = () => {
dagDisplayNamePattern: Boolean(dagDisplayNamePattern) ? dagDisplayNamePattern : undefined,
dagRunsLimit,
dagRunState,
+ dagRunStateWithinHours: Boolean(dagRunState) && withinHours !== null ? Number(withinHours) : undefined,
Review Comment:
Let's make sure that `Number(withinHours)` is returning a valid number.
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagsFilters/RunStateScopeSelect.tsx:
##########
@@ -0,0 +1,91 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import { HStack, Text, type Select as ChakraSelect } from "@chakra-ui/react";
+import { createListCollection } from "@chakra-ui/react/collection";
+import { useTranslation } from "react-i18next";
+import { FiClock, FiZap } from "react-icons/fi";
+
+import { Select } from "src/components/ui";
+
+// "latest" matches on the latest run only; numeric values are hours; "any" has no time bound.
+export type RunStateScope = "168" | "24" | "720" | "any" | "latest";
+
+type Props = {
+ readonly dataTestId?: string;
+ readonly onChange: (value: RunStateScope) => void;
+ readonly triggerProps?: ChakraSelect.TriggerProps;
+ readonly value: RunStateScope;
+};
+
+const SCOPE_OPTIONS: ReadonlyArray<{ labelKey: string; value: RunStateScope }> = [
+ { labelKey: "latestRun", value: "latest" },
+ { labelKey: "last24Hours", value: "24" },
+ { labelKey: "last7Days", value: "168" },
+ { labelKey: "last30Days", value: "720" },
+ { labelKey: "anyTime", value: "any" },
+];
+
+export const RunStateScopeSelect = ({ dataTestId, onChange, triggerProps, value }: Props) => {
Review Comment:
Nit. I think this variable name is confusing. Let's do `RunStateLookbackSelect`
##########
airflow-core/src/airflow/api_fastapi/common/parameters.py:
##########
@@ -1473,19 +1473,29 @@ def depends(cls, has_pending_actions: bool | None = Query(None)) -> _PendingActi
QueryPendingActionsFilter = Annotated[_PendingActionsFilter, Depends(_PendingActionsFilter.depends)]
+# A lookback this large is effectively unbounded (users omit the param for "any time"); capping it
+# also keeps utcnow() - timedelta(hours=...) from overflowing on an absurdly large value.
+_MAX_DAG_RUN_STATE_WINDOW_HOURS = 24 * 366 * 100 # ~100 years
+
+
class _AnyDagRunStateFilter(BaseParam[DagRunState | None]):
"""Filter Dags that have any DagRun in the given state, not only the latest one."""
+ def __init__(self, value: DagRunState | None = None, skip_none: bool = True) -> None:
+ super().__init__(value, skip_none)
+ self.within_hours: int | None = None
+
def to_orm(self, select: Select) -> Select:
if self.value is None and self.skip_none:
return select
- # EXISTS resolves each Dag via the (dag_id, state) index instead of scanning every run in the state.
- has_run_in_state = (
- sql_select(DagRun.dag_id)
- .where(DagRun.dag_id == DagModel.dag_id, DagRun.state == self.value)
- .exists()
- )
+ # EXISTS seeks the (dag_id, state) index per Dag rather than scanning the whole table; the
+ # optional run_after bound is not covered by that index, so it is filtered within each Dag's
+ # matching rows (still bounded per Dag, not a full scan).
+ conditions = [DagRun.dag_id == DagModel.dag_id, DagRun.state == self.value]
+ if self.within_hours is not None:
+ conditions.append(DagRun.run_after >= timezone.utcnow() - timedelta(hours=self.within_hours))
+ has_run_in_state = sql_select(DagRun.dag_id).where(*conditions).exists()
Review Comment:
Without new index and for time ranges, I think we should invert this subquery to look at the dag runs in the range first instead of checking the history of every each dag.
```
bound = timezone.utcnow() - timedelta(hours=self.within_hours)
dag_ids_in_window = (
sql_select(DagRun.dag_id)
.where(DagRun.run_after >= bound, DagRun.state == self.value)
.distinct()
)
```
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagsFilters/runStateFilter.ts:
##########
@@ -0,0 +1,51 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import type { RunStateScope } from "./RunStateScopeSelect";
+
+// The unified "Run state" control is backed by the existing independent URL params:
+// - "latest" scope → last_dag_run_state (match the latest run only)
+// - a time scope → dag_run_state (match any run) + a within-hours bound
+// ("any" is any-run with no time bound, so it carries no window param)
+// An unset key means "clear this param".
+export type RunStateSelection = {
+ readonly dagRunState?: string;
+ readonly dagRunStateWithinHours?: string;
+ readonly lastDagRunState?: string;
+};
+
+export const runStateSelectionFor = (state: string | undefined, scope: RunStateScope): RunStateSelection => {
+ if (state === undefined) {
+ return {};
+ }
+ if (scope === "latest") {
+ return { lastDagRunState: state };
+ }
+ if (scope === "any") {
+ return { dagRunState: state };
+ }
+
+ return { dagRunState: state, dagRunStateWithinHours: scope };
+};
+
+export const runScopeFor = (
+ lastRunState: string | null,
+ anyRunState: string | null,
+ anyRunStateWindow: string | null,
+): RunStateScope =>
+ lastRunState !== null || anyRunState === null ? "latest" : ((anyRunStateWindow ?? "any") as RunStateScope);
Review Comment:
Let's make sure we validate the run state window is a correct number here too.
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagsFilters/DagsFilters.tsx:
##########
@@ -67,14 +70,20 @@ export const DagsFilters = () => {
const showPaused = searchParams.get(PAUSED_PARAM);
const showFavorites = searchParams.get(FAVORITE_PARAM);
const needsReview = searchParams.get(NEEDS_REVIEW_PARAM);
- const state = searchParams.get(LAST_DAG_RUN_STATE_PARAM);
- const activeRunState = searchParams.get(DAG_RUN_STATE_PARAM);
+ const lastRunState = searchParams.get(LAST_DAG_RUN_STATE_PARAM);
+ const anyRunState = searchParams.get(DAG_RUN_STATE_PARAM);
+ const anyRunStateWindow = searchParams.get(DAG_RUN_STATE_WITHIN_HOURS_PARAM);
Review Comment:
Let's call everything a "window", a "period", or a "lookback" but let's not mix our variable names or "scope".
##########
airflow-core/src/airflow/ui/public/i18n/locales/en/dags.json:
##########
@@ -24,6 +24,15 @@
"paused": "Paused"
},
"runIdPatternFilter": "Search Dag Runs",
+ "runScope": {
+ "anyTime": "Any time",
+ "in": "in",
+ "last7Days": "Last 7 days",
+ "last24Hours": "Last 24 hours",
+ "last30Days": "Last 30 days",
+ "latestRun": "Latest run"
+ },
+ "runState": "Run state",
Review Comment:
We have a lot of these translation keys already. The dashboard page has a time range selector.
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagsFilters/RunStateScopeSelect.tsx:
##########
@@ -0,0 +1,91 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import { HStack, Text, type Select as ChakraSelect } from "@chakra-ui/react";
+import { createListCollection } from "@chakra-ui/react/collection";
+import { useTranslation } from "react-i18next";
+import { FiClock, FiZap } from "react-icons/fi";
+
+import { Select } from "src/components/ui";
+
+// "latest" matches on the latest run only; numeric values are hours; "any" has no time bound.
+export type RunStateScope = "168" | "24" | "720" | "any" | "latest";
+
+type Props = {
+ readonly dataTestId?: string;
+ readonly onChange: (value: RunStateScope) => void;
+ readonly triggerProps?: ChakraSelect.TriggerProps;
+ readonly value: RunStateScope;
+};
+
+const SCOPE_OPTIONS: ReadonlyArray<{ labelKey: string; value: RunStateScope }> = [
+ { labelKey: "latestRun", value: "latest" },
+ { labelKey: "last24Hours", value: "24" },
+ { labelKey: "last7Days", value: "168" },
+ { labelKey: "last30Days", value: "720" },
Review Comment:
We should try to reuse the time ranges that the dashboard and overview page already use. I could see a world where we move it into the airflow config so deployment managers can decide based on how often their runs fire. Some teams do every minute, others every day.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "yuseok89 (via GitHub)" <gi...@apache.org>.
yuseok89 commented on code in PR #70470:
URL: https://github.com/apache/airflow/pull/70470#discussion_r3744120592
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagsFilters/DagsFilters.tsx:
##########
@@ -67,14 +70,20 @@ export const DagsFilters = () => {
const showPaused = searchParams.get(PAUSED_PARAM);
const showFavorites = searchParams.get(FAVORITE_PARAM);
const needsReview = searchParams.get(NEEDS_REVIEW_PARAM);
- const state = searchParams.get(LAST_DAG_RUN_STATE_PARAM);
- const activeRunState = searchParams.get(DAG_RUN_STATE_PARAM);
+ const lastRunState = searchParams.get(LAST_DAG_RUN_STATE_PARAM);
+ const anyRunState = searchParams.get(DAG_RUN_STATE_PARAM);
+ const anyRunStateWindow = searchParams.get(DAG_RUN_STATE_WITHIN_HOURS_PARAM);
Review Comment:
Thanks.
Everything is on "lookback" now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "yuseok89 (via GitHub)" <gi...@apache.org>.
yuseok89 commented on code in PR #70470:
URL: https://github.com/apache/airflow/pull/70470#discussion_r3744342019
##########
airflow-core/src/airflow/api_fastapi/common/parameters.py:
##########
@@ -1473,19 +1473,29 @@ def depends(cls, has_pending_actions: bool | None = Query(None)) -> _PendingActi
QueryPendingActionsFilter = Annotated[_PendingActionsFilter, Depends(_PendingActionsFilter.depends)]
+# A lookback this large is effectively unbounded (users omit the param for "any time"); capping it
+# also keeps utcnow() - timedelta(hours=...) from overflowing on an absurdly large value.
+_MAX_DAG_RUN_STATE_WINDOW_HOURS = 24 * 366 * 100 # ~100 years
+
+
class _AnyDagRunStateFilter(BaseParam[DagRunState | None]):
"""Filter Dags that have any DagRun in the given state, not only the latest one."""
+ def __init__(self, value: DagRunState | None = None, skip_none: bool = True) -> None:
+ super().__init__(value, skip_none)
+ self.within_hours: int | None = None
+
def to_orm(self, select: Select) -> Select:
if self.value is None and self.skip_none:
return select
- # EXISTS resolves each Dag via the (dag_id, state) index instead of scanning every run in the state.
- has_run_in_state = (
- sql_select(DagRun.dag_id)
- .where(DagRun.dag_id == DagModel.dag_id, DagRun.state == self.value)
- .exists()
- )
+ # EXISTS seeks the (dag_id, state) index per Dag rather than scanning the whole table; the
+ # optional run_after bound is not covered by that index, so it is filtered within each Dag's
+ # matching rows (still bounded per Dag, not a full scan).
+ conditions = [DagRun.dag_id == DagModel.dag_id, DagRun.state == self.value]
+ if self.within_hours is not None:
+ conditions.append(DagRun.run_after >= timezone.utcnow() - timedelta(hours=self.within_hours))
+ has_run_in_state = sql_select(DagRun.dag_id).where(*conditions).exists()
Review Comment:
Thanks.
I did a quick benchmark of the inverted approach on Postgres (10M `dag_run` rows, warm cache). Rough numbers:
| within_hours | current | inverted (run_after-first) |
|---|---:|---:|
| none | < 50 ms | < 50 ms |
| 7 days | ~1 s | ~1 s |
| 30 days | < 100 ms | ~4 s |
| 90 days | < 50 ms | ~7 s |
| 365 days | < 50 ms | ~5 s |
At least in this setup, the inverted version scans the whole `run_after` slice, so it seemed to get worse as the window widens.
The one thing that held up across every window (tens of ms) was a covering index `(dag_id, state, run_after)`.
I know you wanted to avoid a new index.
Just flagging this in case it changes the trade-off.
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on PR #70470:
URL: https://github.com/apache/airflow/pull/70470#issuecomment-5375215908
Great. Let's rebase after https://github.com/apache/airflow/pull/71731 was merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on code in PR #70470:
URL: https://github.com/apache/airflow/pull/70470#discussion_r4136901837
##########
airflow-core/src/airflow/ui/src/components/FilterBar/filters/runStateParams.ts:
##########
@@ -0,0 +1,71 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import { SearchParamsKeys } from "src/constants/searchParams";
+
+import type { FilterValue } from "../types";
+
+// "latest" matches on the latest run only; numeric values are hours; "any" has no time bound.
+export type RunStateLookback = "168" | "24" | "720" | "any" | "latest";
+
+export const TIME_LOOKBACKS: ReadonlyArray<RunStateLookback> = ["24", "168", "720"];
+export const RUN_STATE_LOOKBACKS: ReadonlyArray<RunStateLookback> = ["latest", ...TIME_LOOKBACKS, "any"];
+
+export type RunStateValue = {
+ lookback: RunStateLookback;
+ state: string;
+};
+
+export const isRunStateValue = (value: FilterValue): value is RunStateValue =>
+ typeof value === "object" && value !== null && !Array.isArray(value) && "state" in value;
+
+// The unified Run state pill is backed by the existing independent URL params:
+// "latest" lookback → last_dag_run_state (match the latest run only)
+// a time lookback → dag_run_state (match any run) + a within-hours bound
+// "any" → dag_run_state with no time bound
+export const runStateToSearchParams = (value: FilterValue): Record<string, string | undefined> => {
+ const selection = isRunStateValue(value) ? value : undefined;
+ const isLatest = selection?.lookback === "latest";
+ const isTimeBounded = selection !== undefined && TIME_LOOKBACKS.includes(selection.lookback);
+
+ return {
+ [SearchParamsKeys.DAG_RUN_STATE]: selection !== undefined && !isLatest ? selection.state : undefined,
+ [SearchParamsKeys.DAG_RUN_STATE_WITHIN_HOURS]: isTimeBounded ? selection.lookback : undefined,
+ [SearchParamsKeys.LAST_DAG_RUN_STATE]: isLatest ? selection.state : undefined,
+ };
+};
+
+export const runStateFromSearchParams = (params: URLSearchParams): RunStateValue | undefined => {
+ const lastRunState = params.get(SearchParamsKeys.LAST_DAG_RUN_STATE);
+ const anyRunState = params.get(SearchParamsKeys.DAG_RUN_STATE);
+ const withinHours = params.get(SearchParamsKeys.DAG_RUN_STATE_WITHIN_HOURS);
+
+ // The latest-run param wins when both are present, matching how the list endpoint treats them.
+ if (lastRunState !== null && lastRunState !== "") {
+ return { lookback: "latest", state: lastRunState };
+ }
+ if (anyRunState === null || anyRunState === "") {
+ return undefined;
+ }
+
+ const lookback = (TIME_LOOKBACKS as ReadonlyArray<string>).includes(withinHours ?? "")
Review Comment:
Same here
```
const isLookback = (value: string): value is RunStateLookback =>
TIME_LOOKBACKS.some((lookback) => lookback === value);
const lookback = withinHours !== null && isLookback(withinHours) ? withinHours : "any";
```
##########
airflow-core/src/airflow/ui/src/components/FilterBar/filters/RunStateFilter.tsx:
##########
@@ -0,0 +1,234 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import { useRef, useState } from "react";
+
+import { Box, HStack, createListCollection } from "@chakra-ui/react";
+import { useTranslation } from "react-i18next";
+import { FiClock, FiZap } from "react-icons/fi";
+
+import { Select } from "src/system-components";
+
+import { FilterPill } from "../FilterPill";
+import type { FilterConfig, FilterPluginProps } from "../types";
+import { RUN_STATE_LOOKBACKS, isRunStateValue, type RunStateLookback } from "./runStateParams";
+
+const LOOKBACK_LABEL_KEYS: Record<RunStateLookback, string> = {
+ "24": "last24Hours",
+ "168": "last7Days",
+ "720": "last30Days",
+ any: "anyTime",
+ latest: "latestRun",
+};
+
+type SelectOption = {
+ label: string;
+ value: string;
+};
+
+type RunStateFilterConfig = {
+ options: Array<SelectOption>;
+};
+
+const collapsePill = () => {
+ setTimeout(() => {
+ const activeElement = document.activeElement as HTMLElement;
+
+ activeElement.blur();
+ }, 0);
+};
+
+/**
+ * One pill filtering Dags on a run state matched within a lookback: the latest run only, a
+ * recent time window, or the whole run history. Committed as a composite value that the
+ * config's projections spread over the underlying URL params.
+ */
+export const RunStateFilter = ({ filter, onChange, onRemove }: FilterPluginProps) => {
+ const { t: translate } = useTranslation(["common"]);
+ const config = filter.config as FilterConfig & RunStateFilterConfig;
+ const value = isRunStateValue(filter.value) ? filter.value : undefined;
+
+ // Holds a lookback picked before any state is chosen, when there is nothing to commit yet.
+ const [pendingLookback, setPendingLookback] = useState<RunStateLookback>(value?.lookback ?? "latest");
+
+ // Selecting an option closes the menu in the same tick as the value commits, before React has
+ // re-rendered. Recording the selection keeps the close from being mistaken for an abandoned
+ // filter, which a timing-based check gets wrong.
+ const hasJustSelected = useRef(false);
+
+ const handleStateChange = ({ value: selected }: { value: Array<string> }) => {
+ const [newState] = selected;
+
+ if (newState === undefined) {
+ return;
+ }
+ hasJustSelected.current = true;
+ onChange({ lookback: value?.lookback ?? pendingLookback, state: newState });
+ collapsePill();
+ };
+
+ const handleLookbackChange = ({ value: selected }: { value: Array<string> }) => {
+ const [newLookback] = selected as Array<RunStateLookback>;
Review Comment:
Let's try to avoid this typecasting with a "value in" check like:
```
export const isLookback = (value: string): value is RunStateLookback =>
RUN_STATE_LOOKBACKS.some((lookback) => lookback === value);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "yuseok89 (via GitHub)" <gi...@apache.org>.
yuseok89 commented on code in PR #70470:
URL: https://github.com/apache/airflow/pull/70470#discussion_r3744086604
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagsFilters/RunStateScopeSelect.tsx:
##########
@@ -0,0 +1,91 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import { HStack, Text, type Select as ChakraSelect } from "@chakra-ui/react";
+import { createListCollection } from "@chakra-ui/react/collection";
+import { useTranslation } from "react-i18next";
+import { FiClock, FiZap } from "react-icons/fi";
+
+import { Select } from "src/components/ui";
+
+// "latest" matches on the latest run only; numeric values are hours; "any" has no time bound.
+export type RunStateScope = "168" | "24" | "720" | "any" | "latest";
+
+type Props = {
+ readonly dataTestId?: string;
+ readonly onChange: (value: RunStateScope) => void;
+ readonly triggerProps?: ChakraSelect.TriggerProps;
+ readonly value: RunStateScope;
+};
+
+const SCOPE_OPTIONS: ReadonlyArray<{ labelKey: string; value: RunStateScope }> = [
+ { labelKey: "latestRun", value: "latest" },
+ { labelKey: "last24Hours", value: "24" },
+ { labelKey: "last7Days", value: "168" },
+ { labelKey: "last30Days", value: "720" },
+ { labelKey: "anyTime", value: "any" },
+];
+
+export const RunStateScopeSelect = ({ dataTestId, onChange, triggerProps, value }: Props) => {
Review Comment:
Thanks.
Renamed to `RunStateLookbackSelect`, and standardized on "lookback" everywhere.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "yuseok89 (via GitHub)" <gi...@apache.org>.
yuseok89 commented on code in PR #70470:
URL: https://github.com/apache/airflow/pull/70470#discussion_r3744196134
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagsFilters/runStateFilter.ts:
##########
@@ -0,0 +1,51 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import type { RunStateScope } from "./RunStateScopeSelect";
+
+// The unified "Run state" control is backed by the existing independent URL params:
+// - "latest" scope → last_dag_run_state (match the latest run only)
+// - a time scope → dag_run_state (match any run) + a within-hours bound
+// ("any" is any-run with no time bound, so it carries no window param)
+// An unset key means "clear this param".
+export type RunStateSelection = {
+ readonly dagRunState?: string;
+ readonly dagRunStateWithinHours?: string;
+ readonly lastDagRunState?: string;
+};
+
+export const runStateSelectionFor = (state: string | undefined, scope: RunStateScope): RunStateSelection => {
+ if (state === undefined) {
+ return {};
+ }
+ if (scope === "latest") {
+ return { lastDagRunState: state };
+ }
+ if (scope === "any") {
+ return { dagRunState: state };
+ }
+
+ return { dagRunState: state, dagRunStateWithinHours: scope };
+};
+
+export const runScopeFor = (
+ lastRunState: string | null,
+ anyRunState: string | null,
+ anyRunStateWindow: string | null,
+): RunStateScope =>
+ lastRunState !== null || anyRunState === null ? "latest" : ((anyRunStateWindow ?? "any") as RunStateScope);
Review Comment:
Thanks.
`runLookbackFor` now only accepts a known lookback value; anything else falls back to "any".
Added tests for the invalid cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi merged PR #70470:
URL: https://github.com/apache/airflow/pull/70470
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Add a time-window scope to the Dags list run state filter [airflow]
Posted by "yuseok89 (via GitHub)" <gi...@apache.org>.
yuseok89 commented on PR #70470:
URL: https://github.com/apache/airflow/pull/70470#issuecomment-5870107885
Hi @bbovenzi , I wanted to gently ping on this one. I've addressed the earlier feedback and CI is green. Would you have a moment to take another look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org