You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "RabahAmrouche05 (via GitHub)" <gi...@apache.org> on 2026/06/14 18:29:35 UTC
[PR] UI: Group and filter Dags by folder on the Dags page [airflow]
RabahAmrouche05 opened a new pull request, #68544:
URL: https://github.com/apache/airflow/pull/68544
## What
Adds a **folder navigation tree** to the Dags page. The tree is built from each
Dag's `relative_fileloc` directory, and selecting a folder filters the Dag list
to the Dags that live in it (and its subfolders).
## Why
The Dags page only offered a flat list. In deployments with many Dags spread
across subfolders of the dags folder (e.g. `team_a/etl/…`, `team_b/ml/…`) there
was no way to browse or filter by folder.
## How
- **Backend** — a server-side `relative_fileloc_prefix` filter on `GET /ui/dags`
(strict folder match, so `team_a` never catches `team_alpha`) and a new
`GET /ui/dags/folders` endpoint returning the distinct folders of all readable
Dags. Filtering is done server-side so the result stays correct across
pagination, sorting and the other filters.
- **Frontend** — a collapsible folder tree on the Dags page that drives the
filter via a `dag_folder` search param, with the tree hierarchy reconstructed
client-side from the folder paths.
Dags located at the bundle root contribute no folder and appear under "All Dags".
Tests: backend (folders endpoint + filter, incl. the `team_a` / `team_alpha`
edge case) and frontend (`buildFolderTree` unit tests + `DagFolderTree`
component tests).
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Opus 4.8)
Generated-by: Claude Code (Opus 4.8) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #68544:
URL: https://github.com/apache/airflow/pull/68544#issuecomment-5952563156
Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on PR #68544:
URL: https://github.com/apache/airflow/pull/68544#issuecomment-5022949688
Needs to fix the conflicts before we can merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #68544:
URL: https://github.com/apache/airflow/pull/68544#discussion_r3529917150
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dags.py:
##########
@@ -253,6 +258,36 @@ def get_dags(
)
+@dags_router.get(
+ "/folders",
+ dependencies=[Depends(requires_access_dag(method="GET"))],
+ operation_id="get_dag_folders",
+)
+def get_dag_folders(
+ readable_dags_filter: ReadableDagsFilterDep,
+ session: SessionDep,
+) -> DagFolderCollectionResponse:
+ """
+ Get the distinct folders the readable Dags live in.
Review Comment:
lets go like this for now. IF this proves to be a problem we can add a limit later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "RabahAmrouche05 (via GitHub)" <gi...@apache.org>.
RabahAmrouche05 commented on code in PR #68544:
URL: https://github.com/apache/airflow/pull/68544#discussion_r3501908510
##########
airflow-core/tests/unit/api_fastapi/core_api/routes/ui/test_dags.py:
##########
@@ -418,3 +418,92 @@ def test_is_favorite_field_user_specific(self, test_client, session):
# Verify that DAG1 is not marked as favorite for the test user
dag1_data = next(dag for dag in body["dags"] if dag["dag_id"] == DAG1_ID)
assert dag1_data["is_favorite"] is False
+
+
+# Maps dag_id -> relative_fileloc. ``team_alpha`` exists to prove that a folder
+# name is never matched as a prefix of another (``team_a`` must not catch it),
+# and ``root_dag.py`` lives at the bundle root (no folder).
+FOLDER_DAGS = {
+ "folder_dag_a_etl_extract": "team_a/etl/extract.py",
+ "folder_dag_a_etl_load": "team_a/etl/load.py",
+ "folder_dag_a_report": "team_a/report.py",
+ "folder_dag_b_ml_train": "team_b/ml/train.py",
+ "folder_dag_alpha": "team_alpha/x.py",
+ "folder_dag_root": "root_dag.py",
+}
+
+
+class TestDagFolders(TestPublicDagEndpoint):
Review Comment:
Added `test_get_dag_folders_query_count_does_not_scale_with_dags`: it captures the query count, adds 50 more Dags across new folders, then asserts the count is unchanged, locking in that the endpoint stays at a finite number of queries and never scales with the number of Dags.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Group and filter Dags by folder on the Dags page [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on PR #68544:
URL: https://github.com/apache/airflow/pull/68544#issuecomment-4770382239
PR is marked for draft, are you planning additional work before being ready for review?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on code in PR #68544:
URL: https://github.com/apache/airflow/pull/68544#discussion_r3639704315
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/DagFolderTree.tsx:
##########
@@ -0,0 +1,320 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import { Box, Flex, Heading, Skeleton, Text, VStack } from "@chakra-ui/react";
+import { useState } from "react";
+import { useTranslation } from "react-i18next";
+import { FiBox, FiChevronDown, FiChevronRight, FiFolder } from "react-icons/fi";
+
+import type { DagFolderResponse } from "openapi/requests/types.gen";
+
+import { groupFoldersByBundle, type BundleFolders, type FolderNode } from "./buildFolderTree";
+
+export type FolderSelection = {
+ readonly bundleName: string | undefined;
+ readonly folder: string | undefined;
+};
+
+type Props = {
+ readonly folders: ReadonlyArray<DagFolderResponse>;
+ readonly isLoading?: boolean;
+ readonly onSelectFolder: (selection: FolderSelection) => void;
+ readonly selectedBundle: string | undefined;
+ readonly selectedFolder: string | undefined;
+};
+
+const folderKey = (bundleName: string | undefined, path: string) => `folder:${bundleName ?? ""}:${path}`;
+const bundleKey = (bundleName: string) => `bundle:${bundleName}`;
+
+// Ancestor folder paths of the selected folder, so the tree opens to reveal the selection.
+const ancestorPaths = (folder: string | undefined): Array<string> => {
+ if (folder === undefined || folder === "") {
+ return [];
+ }
+
+ const segments = folder.split("/");
+
+ return segments.map((_, index) => segments.slice(0, index + 1).join("/"));
+};
+
+const initialExpanded = (
+ bundleName: string | undefined,
+ selectedFolder: string | undefined,
+ isMultiBundle: boolean,
+): Array<string> => {
+ const keys = ancestorPaths(selectedFolder).map((path) => folderKey(bundleName, path));
+
+ if (isMultiBundle && bundleName !== undefined) {
+ keys.push(bundleKey(bundleName));
+ }
+
+ return keys;
+};
+
+type FolderRowProps = {
+ readonly bundleName: string | undefined;
+ readonly depthOffset: number;
+ readonly expanded: Set<string>;
+ readonly node: FolderNode;
+ readonly onSelectFolder: (selection: FolderSelection) => void;
+ readonly onToggle: (key: string) => void;
+ readonly selectedBundle: string | undefined;
+ readonly selectedFolder: string | undefined;
+};
+
+const FolderRow = ({
+ bundleName,
+ depthOffset,
+ expanded,
+ node,
+ onSelectFolder,
+ onToggle,
+ selectedBundle,
+ selectedFolder,
+}: FolderRowProps) => {
+ const hasChildren = node.children.length > 0;
+ const key = folderKey(bundleName, node.path);
+ const isExpanded = expanded.has(key);
+ const isSelected = selectedBundle === bundleName && selectedFolder === node.path;
+ const depth = node.path.split("/").length - 1 + depthOffset;
+
+ return (
+ <Box>
+ <Flex
+ _hover={{ bg: isSelected ? "blue.subtle" : "bg.muted" }}
+ alignItems="center"
+ bg={isSelected ? "blue.subtle" : undefined}
+ borderRadius="sm"
+ cursor="pointer"
+ gap={1}
+ onClick={() => onSelectFolder({ bundleName, folder: node.path })}
+ pl={`${depth * 16 + 4}px`}
+ py={1}
+ >
+ <Box
+ aria-label={isExpanded ? "Collapse" : "Expand"}
+ as="span"
+ color="fg.muted"
+ onClick={(event) => {
+ event.stopPropagation();
+ if (hasChildren) {
+ onToggle(key);
+ }
+ }}
+ visibility={hasChildren ? "visible" : "hidden"}
+ >
+ {isExpanded ? <FiChevronDown /> : <FiChevronRight />}
+ </Box>
+ <Box as="span" color="fg.muted">
+ <FiFolder />
+ </Box>
+ <Text fontWeight={isSelected ? "bold" : "normal"} truncate>
+ {node.name}
+ </Text>
+ </Flex>
+ {hasChildren && isExpanded ? (
+ <Box>
+ {node.children.map((child) => (
+ <FolderRow
+ bundleName={bundleName}
+ depthOffset={depthOffset}
+ expanded={expanded}
+ key={child.path}
+ node={child}
+ onSelectFolder={onSelectFolder}
+ onToggle={onToggle}
+ selectedBundle={selectedBundle}
+ selectedFolder={selectedFolder}
+ />
+ ))}
+ </Box>
+ ) : undefined}
+ </Box>
+ );
+};
+
+type BundleRowProps = {
+ readonly bundle: BundleFolders;
+ readonly expanded: Set<string>;
+ readonly onSelectFolder: (selection: FolderSelection) => void;
+ readonly onToggle: (key: string) => void;
+ readonly selectedBundle: string | undefined;
+ readonly selectedFolder: string | undefined;
+};
+
+const BundleRow = ({
+ bundle,
+ expanded,
+ onSelectFolder,
+ onToggle,
+ selectedBundle,
+ selectedFolder,
+}: BundleRowProps) => {
+ const key = bundleKey(bundle.bundleName);
+ const isExpanded = expanded.has(key);
+ const isSelected = selectedBundle === bundle.bundleName && selectedFolder === undefined;
+ const hasChildren = bundle.tree.length > 0;
+
+ return (
+ <Box>
+ <Flex
+ _hover={{ bg: isSelected ? "blue.subtle" : "bg.muted" }}
+ alignItems="center"
+ bg={isSelected ? "blue.subtle" : undefined}
+ borderRadius="sm"
+ cursor="pointer"
+ gap={1}
+ onClick={() => onSelectFolder({ bundleName: bundle.bundleName, folder: undefined })}
Review Comment:
We shouldn't be putting onClick handlers on flex and divs. They should be buttons. But fortunately, we don't have to do this ourselves. Chakra has a Tree View component that we can use!
https://www.chakra-ui.com/docs/components/tree-view
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #68544:
URL: https://github.com/apache/airflow/pull/68544#discussion_r3614945154
##########
airflow-core/src/airflow/api_fastapi/common/parameters.py:
##########
@@ -894,6 +894,43 @@ def depends(cls, owners: list[str] = Query(default_factory=list)) -> _OwnersFilt
return cls().set_value(owners)
+class _RelativeFilelocPrefixFilter(BaseParam[str | None]):
+ """
+ Filter Dags by the folder they live in, derived from ``relative_fileloc``.
+
+ The value is treated as a directory path relative to the bundle root (e.g.
+ ``team_a/etl``). It matches every Dag whose file lives directly in that folder
+ or in any of its subfolders, using an escaped ``LIKE 'team_a/etl/%'`` so a
+ folder name is never a substring/prefix of another (``team_a`` won't match
+ ``team_alpha``). Dags at the bundle root (no ``/`` in ``relative_fileloc``)
+ are not matched by any folder value and appear only when no folder is selected.
+ """
+
+ def to_orm(self, select: Select) -> Select:
+ if self.value is None and self.skip_none:
+ return select
+
+ if not self.value:
+ return select
+
+ directory = self.value.rstrip("/")
Review Comment:
I think we can keep it like this for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #68544:
URL: https://github.com/apache/airflow/pull/68544#discussion_r3655722747
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/DagFolderTree.tsx:
##########
@@ -0,0 +1,320 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import { Box, Flex, Heading, Skeleton, Text, VStack } from "@chakra-ui/react";
+import { useState } from "react";
+import { useTranslation } from "react-i18next";
+import { FiBox, FiChevronDown, FiChevronRight, FiFolder } from "react-icons/fi";
+
+import type { DagFolderResponse } from "openapi/requests/types.gen";
+
+import { groupFoldersByBundle, type BundleFolders, type FolderNode } from "./buildFolderTree";
+
+export type FolderSelection = {
+ readonly bundleName: string | undefined;
+ readonly folder: string | undefined;
+};
+
+type Props = {
+ readonly folders: ReadonlyArray<DagFolderResponse>;
+ readonly isLoading?: boolean;
+ readonly onSelectFolder: (selection: FolderSelection) => void;
+ readonly selectedBundle: string | undefined;
+ readonly selectedFolder: string | undefined;
+};
+
+const folderKey = (bundleName: string | undefined, path: string) => `folder:${bundleName ?? ""}:${path}`;
+const bundleKey = (bundleName: string) => `bundle:${bundleName}`;
+
+// Ancestor folder paths of the selected folder, so the tree opens to reveal the selection.
+const ancestorPaths = (folder: string | undefined): Array<string> => {
+ if (folder === undefined || folder === "") {
+ return [];
+ }
+
+ const segments = folder.split("/");
+
+ return segments.map((_, index) => segments.slice(0, index + 1).join("/"));
+};
+
+const initialExpanded = (
+ bundleName: string | undefined,
+ selectedFolder: string | undefined,
+ isMultiBundle: boolean,
+): Array<string> => {
+ const keys = ancestorPaths(selectedFolder).map((path) => folderKey(bundleName, path));
+
+ if (isMultiBundle && bundleName !== undefined) {
+ keys.push(bundleKey(bundleName));
+ }
+
+ return keys;
+};
+
+type FolderRowProps = {
+ readonly bundleName: string | undefined;
+ readonly depthOffset: number;
+ readonly expanded: Set<string>;
+ readonly node: FolderNode;
+ readonly onSelectFolder: (selection: FolderSelection) => void;
+ readonly onToggle: (key: string) => void;
+ readonly selectedBundle: string | undefined;
+ readonly selectedFolder: string | undefined;
+};
+
+const FolderRow = ({
+ bundleName,
+ depthOffset,
+ expanded,
+ node,
+ onSelectFolder,
+ onToggle,
+ selectedBundle,
+ selectedFolder,
+}: FolderRowProps) => {
+ const hasChildren = node.children.length > 0;
+ const key = folderKey(bundleName, node.path);
+ const isExpanded = expanded.has(key);
+ const isSelected = selectedBundle === bundleName && selectedFolder === node.path;
+ const depth = node.path.split("/").length - 1 + depthOffset;
+
+ return (
+ <Box>
+ <Flex
+ _hover={{ bg: isSelected ? "blue.subtle" : "bg.muted" }}
+ alignItems="center"
+ bg={isSelected ? "blue.subtle" : undefined}
+ borderRadius="sm"
+ cursor="pointer"
+ gap={1}
+ onClick={() => onSelectFolder({ bundleName, folder: node.path })}
+ pl={`${depth * 16 + 4}px`}
+ py={1}
+ >
+ <Box
+ aria-label={isExpanded ? "Collapse" : "Expand"}
+ as="span"
+ color="fg.muted"
+ onClick={(event) => {
+ event.stopPropagation();
+ if (hasChildren) {
+ onToggle(key);
+ }
+ }}
+ visibility={hasChildren ? "visible" : "hidden"}
+ >
+ {isExpanded ? <FiChevronDown /> : <FiChevronRight />}
+ </Box>
+ <Box as="span" color="fg.muted">
+ <FiFolder />
+ </Box>
+ <Text fontWeight={isSelected ? "bold" : "normal"} truncate>
+ {node.name}
+ </Text>
+ </Flex>
+ {hasChildren && isExpanded ? (
+ <Box>
+ {node.children.map((child) => (
+ <FolderRow
+ bundleName={bundleName}
+ depthOffset={depthOffset}
+ expanded={expanded}
+ key={child.path}
+ node={child}
+ onSelectFolder={onSelectFolder}
+ onToggle={onToggle}
+ selectedBundle={selectedBundle}
+ selectedFolder={selectedFolder}
+ />
+ ))}
+ </Box>
+ ) : undefined}
+ </Box>
+ );
+};
+
+type BundleRowProps = {
+ readonly bundle: BundleFolders;
+ readonly expanded: Set<string>;
+ readonly onSelectFolder: (selection: FolderSelection) => void;
+ readonly onToggle: (key: string) => void;
+ readonly selectedBundle: string | undefined;
+ readonly selectedFolder: string | undefined;
+};
+
+const BundleRow = ({
+ bundle,
+ expanded,
+ onSelectFolder,
+ onToggle,
+ selectedBundle,
+ selectedFolder,
+}: BundleRowProps) => {
+ const key = bundleKey(bundle.bundleName);
+ const isExpanded = expanded.has(key);
+ const isSelected = selectedBundle === bundle.bundleName && selectedFolder === undefined;
+ const hasChildren = bundle.tree.length > 0;
+
+ return (
+ <Box>
+ <Flex
+ _hover={{ bg: isSelected ? "blue.subtle" : "bg.muted" }}
+ alignItems="center"
+ bg={isSelected ? "blue.subtle" : undefined}
+ borderRadius="sm"
+ cursor="pointer"
+ gap={1}
+ onClick={() => onSelectFolder({ bundleName: bundle.bundleName, folder: undefined })}
Review Comment:
+1 for using chakra tree component
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun merged PR #68544:
URL: https://github.com/apache/airflow/pull/68544
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "RabahAmrouche05 (via GitHub)" <gi...@apache.org>.
RabahAmrouche05 commented on code in PR #68544:
URL: https://github.com/apache/airflow/pull/68544#discussion_r3501905570
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/DagFolderTree.tsx:
##########
@@ -0,0 +1,178 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import { Box, Flex, Heading, Skeleton, Text, VStack } from "@chakra-ui/react";
+import { useMemo, useState } from "react";
+import { useTranslation } from "react-i18next";
+import { FiChevronDown, FiChevronRight, FiFolder } from "react-icons/fi";
+
+import { buildFolderTree, type FolderNode } from "./buildFolderTree";
+
+type Props = {
+ readonly folders: ReadonlyArray<string>;
+ readonly isLoading?: boolean;
+ readonly onSelectFolder: (path: string | undefined) => void;
+ readonly selectedFolder: string | undefined;
+};
+
+// Every ancestor folder of the selected path, so the tree opens to reveal the selection.
+const ancestorPaths = (folder: string | undefined): Array<string> => {
+ if (folder === undefined || folder === "") {
+ return [];
+ }
+
+ const segments = folder.split("/");
+
+ return segments.map((_, index) => segments.slice(0, index + 1).join("/"));
+};
+
+type RowProps = {
+ readonly expanded: Set<string>;
+ readonly node: FolderNode;
+ readonly onSelectFolder: (path: string | undefined) => void;
+ readonly onToggle: (path: string) => void;
+ readonly selectedFolder: string | undefined;
+};
+
+const FolderRow = ({ expanded, node, onSelectFolder, onToggle, selectedFolder }: RowProps) => {
+ const hasChildren = node.children.length > 0;
+ const isExpanded = expanded.has(node.path);
+ const isSelected = selectedFolder === node.path;
+ const depth = node.path.split("/").length - 1;
+
+ return (
+ <Box>
+ <Flex
+ _hover={{ bg: isSelected ? "blue.subtle" : "bg.muted" }}
+ alignItems="center"
+ bg={isSelected ? "blue.subtle" : undefined}
+ borderRadius="sm"
+ cursor="pointer"
+ gap={1}
+ onClick={() => onSelectFolder(node.path)}
+ pl={`${depth * 16 + 4}px`}
+ py={1}
+ >
+ <Box
+ aria-label={isExpanded ? "Collapse" : "Expand"}
+ as="span"
+ color="fg.muted"
+ onClick={(event) => {
+ event.stopPropagation();
+ if (hasChildren) {
+ onToggle(node.path);
+ }
+ }}
+ visibility={hasChildren ? "visible" : "hidden"}
+ >
+ {isExpanded ? <FiChevronDown /> : <FiChevronRight />}
+ </Box>
+ <Box as="span" color="fg.muted">
+ <FiFolder />
+ </Box>
+ <Text fontWeight={isSelected ? "bold" : "normal"} truncate>
+ {node.name}
+ </Text>
+ </Flex>
+ {hasChildren && isExpanded ? (
+ <Box>
+ {node.children.map((child) => (
+ <FolderRow
+ expanded={expanded}
+ key={child.path}
+ node={child}
+ onSelectFolder={onSelectFolder}
+ onToggle={onToggle}
+ selectedFolder={selectedFolder}
+ />
+ ))}
+ </Box>
+ ) : undefined}
+ </Box>
+ );
+};
+
+export const DagFolderTree = ({ folders, isLoading = false, onSelectFolder, selectedFolder }: Props) => {
+ const { t: translate } = useTranslation("dags");
+ const tree = useMemo(() => buildFolderTree(folders), [folders]);
Review Comment:
Done ! removed the `useMemo`, relying on react-compiler as per the repo convention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "RabahAmrouche05 (via GitHub)" <gi...@apache.org>.
RabahAmrouche05 commented on code in PR #68544:
URL: https://github.com/apache/airflow/pull/68544#discussion_r3501908510
##########
airflow-core/tests/unit/api_fastapi/core_api/routes/ui/test_dags.py:
##########
@@ -418,3 +418,92 @@ def test_is_favorite_field_user_specific(self, test_client, session):
# Verify that DAG1 is not marked as favorite for the test user
dag1_data = next(dag for dag in body["dags"] if dag["dag_id"] == DAG1_ID)
assert dag1_data["is_favorite"] is False
+
+
+# Maps dag_id -> relative_fileloc. ``team_alpha`` exists to prove that a folder
+# name is never matched as a prefix of another (``team_a`` must not catch it),
+# and ``root_dag.py`` lives at the bundle root (no folder).
+FOLDER_DAGS = {
+ "folder_dag_a_etl_extract": "team_a/etl/extract.py",
+ "folder_dag_a_etl_load": "team_a/etl/load.py",
+ "folder_dag_a_report": "team_a/report.py",
+ "folder_dag_b_ml_train": "team_b/ml/train.py",
+ "folder_dag_alpha": "team_alpha/x.py",
+ "folder_dag_root": "root_dag.py",
+}
+
+
+class TestDagFolders(TestPublicDagEndpoint):
Review Comment:
Added `test_get_dag_folders_query_count_does_not_scale_with_dags`: it captures the query count, adds 50 more Dags across new folders, then asserts the count is unchanged — locking in that the endpoint stays at a finite number of queries and never scales with the number of Dags.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Group and filter Dags by folder on the Dags page [airflow]
Posted by "RabahAmrouche05 (via GitHub)" <gi...@apache.org>.
RabahAmrouche05 commented on PR #68544:
URL: https://github.com/apache/airflow/pull/68544#issuecomment-4731258413
Thanks for the review @jscheffl !
Here's what the feature looks like:
- **Screenshot** — the example DAG folder structure on disk (`team_a/etl/…`, `team_b/ml/…`).
<img width="418" height="188" alt="Capture d’écran 2026-06-17 à 15 49 30" src="https://github.com/user-attachments/assets/c5dd3cca-61b9-4f5a-9f31-d2e4a882b71b" />
- **Video** — navigating the new folder tree on the Dags page: expanding/collapsing folders, selecting a folder to filter the list (server-side, so it stays correct across pagination), and going back to "All Dags".
https://github.com/user-attachments/assets/fc08eb31-60f6-4575-851c-0d682ce37af2
I also **removed the newsfragment** as you suggested.
One more note: a `main` merge had left the generated OpenAPI TS client out of sync (the `GetDagFolders` types were lost during the merge resolution), which was failing the static checks. I've regenerated the client, so that should be green now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Group and filter Dags by folder on the Dags page [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on PR #68544:
URL: https://github.com/apache/airflow/pull/68544#issuecomment-4703134350
Can you please add a screenshow or small video recording of how it looks like?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on code in PR #68544:
URL: https://github.com/apache/airflow/pull/68544#discussion_r3546532871
##########
airflow-core/src/airflow/api_fastapi/common/parameters.py:
##########
@@ -894,6 +894,43 @@ def depends(cls, owners: list[str] = Query(default_factory=list)) -> _OwnersFilt
return cls().set_value(owners)
+class _RelativeFilelocPrefixFilter(BaseParam[str | None]):
+ """
+ Filter Dags by the folder they live in, derived from ``relative_fileloc``.
+
+ The value is treated as a directory path relative to the bundle root (e.g.
+ ``team_a/etl``). It matches every Dag whose file lives directly in that folder
+ or in any of its subfolders, using an escaped ``LIKE 'team_a/etl/%'`` so a
+ folder name is never a substring/prefix of another (``team_a`` won't match
+ ``team_alpha``). Dags at the bundle root (no ``/`` in ``relative_fileloc``)
+ are not matched by any folder value and appear only when no folder is selected.
+ """
+
+ def to_orm(self, select: Select) -> Select:
+ if self.value is None and self.skip_none:
+ return select
+
+ if not self.value:
+ return select
+
+ directory = self.value.rstrip("/")
Review Comment:
Are we sure we can confidently determine dag folders just by parsing a string by `/`? This feels very brittle.
I am less familiar with how we generate the relative_fileloc. @jedcunningham any thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "RabahAmrouche05 (via GitHub)" <gi...@apache.org>.
RabahAmrouche05 commented on PR #68544:
URL: https://github.com/apache/airflow/pull/68544#issuecomment-5430619873
Thanks @pierrejeambrun for the review! All three points are addressed:
- Expanding from a URL: the tree state was only computed on the first render, when the folders are still loading, so it never expanded. It now re-applies the auto expansion once the selection resolves to real nodes, while keeping whatever the user expanded manually. You were right about the test too, it was passing the folders synchronously, so the new test renders with no folders first and then with them.
- Long names: rows now take the full panel width and the label truncates with an ellipsis, so the highlight covers the whole row. The full name is still available as a tooltip.
- Added a button to show/hide the tree, next to the sort and display controls. The choice is remembered in localStorage, so you can get the full width table back.
Also rebased on main.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Group and filter Dags by folder on the Dags page [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #68544:
URL: https://github.com/apache/airflow/pull/68544#discussion_r3492446434
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/DagFolderTree.tsx:
##########
@@ -0,0 +1,178 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import { Box, Flex, Heading, Skeleton, Text, VStack } from "@chakra-ui/react";
+import { useMemo, useState } from "react";
+import { useTranslation } from "react-i18next";
+import { FiChevronDown, FiChevronRight, FiFolder } from "react-icons/fi";
+
+import { buildFolderTree, type FolderNode } from "./buildFolderTree";
+
+type Props = {
+ readonly folders: ReadonlyArray<string>;
+ readonly isLoading?: boolean;
+ readonly onSelectFolder: (path: string | undefined) => void;
+ readonly selectedFolder: string | undefined;
+};
+
+// Every ancestor folder of the selected path, so the tree opens to reveal the selection.
+const ancestorPaths = (folder: string | undefined): Array<string> => {
+ if (folder === undefined || folder === "") {
+ return [];
+ }
+
+ const segments = folder.split("/");
+
+ return segments.map((_, index) => segments.slice(0, index + 1).join("/"));
+};
+
+type RowProps = {
+ readonly expanded: Set<string>;
+ readonly node: FolderNode;
+ readonly onSelectFolder: (path: string | undefined) => void;
+ readonly onToggle: (path: string) => void;
+ readonly selectedFolder: string | undefined;
+};
+
+const FolderRow = ({ expanded, node, onSelectFolder, onToggle, selectedFolder }: RowProps) => {
+ const hasChildren = node.children.length > 0;
+ const isExpanded = expanded.has(node.path);
+ const isSelected = selectedFolder === node.path;
+ const depth = node.path.split("/").length - 1;
+
+ return (
+ <Box>
+ <Flex
+ _hover={{ bg: isSelected ? "blue.subtle" : "bg.muted" }}
+ alignItems="center"
+ bg={isSelected ? "blue.subtle" : undefined}
+ borderRadius="sm"
+ cursor="pointer"
+ gap={1}
+ onClick={() => onSelectFolder(node.path)}
+ pl={`${depth * 16 + 4}px`}
+ py={1}
+ >
+ <Box
+ aria-label={isExpanded ? "Collapse" : "Expand"}
+ as="span"
+ color="fg.muted"
+ onClick={(event) => {
+ event.stopPropagation();
+ if (hasChildren) {
+ onToggle(node.path);
+ }
+ }}
+ visibility={hasChildren ? "visible" : "hidden"}
+ >
+ {isExpanded ? <FiChevronDown /> : <FiChevronRight />}
+ </Box>
+ <Box as="span" color="fg.muted">
+ <FiFolder />
+ </Box>
+ <Text fontWeight={isSelected ? "bold" : "normal"} truncate>
+ {node.name}
+ </Text>
+ </Flex>
+ {hasChildren && isExpanded ? (
+ <Box>
+ {node.children.map((child) => (
+ <FolderRow
+ expanded={expanded}
+ key={child.path}
+ node={child}
+ onSelectFolder={onSelectFolder}
+ onToggle={onToggle}
+ selectedFolder={selectedFolder}
+ />
+ ))}
+ </Box>
+ ) : undefined}
+ </Box>
+ );
+};
+
+export const DagFolderTree = ({ folders, isLoading = false, onSelectFolder, selectedFolder }: Props) => {
+ const { t: translate } = useTranslation("dags");
+ const tree = useMemo(() => buildFolderTree(folders), [folders]);
Review Comment:
`useMemo` probably not needed thanks to `react-compiler`. Repo convention is 'no manual useMemo, useCallback'.
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dags.py:
##########
@@ -253,6 +258,36 @@ def get_dags(
)
+@dags_router.get(
+ "/folders",
+ dependencies=[Depends(requires_access_dag(method="GET"))],
+ operation_id="get_dag_folders",
+)
+def get_dag_folders(
+ readable_dags_filter: ReadableDagsFilterDep,
+ session: SessionDep,
+) -> DagFolderCollectionResponse:
+ """
+ Get the distinct folders the readable Dags live in.
Review Comment:
Since this is a 'list' endpoint but it isn't paginated (in comparison to other list endpoint), I'm curious how this could go out of hands if there are a high number of dags. Should we set a manual limit here just in case?
##########
airflow-core/tests/unit/api_fastapi/core_api/routes/ui/test_dags.py:
##########
@@ -418,3 +418,92 @@ def test_is_favorite_field_user_specific(self, test_client, session):
# Verify that DAG1 is not marked as favorite for the test user
dag1_data = next(dag for dag in body["dags"] if dag["dag_id"] == DAG1_ID)
assert dag1_data["is_favorite"] is False
+
+
+# Maps dag_id -> relative_fileloc. ``team_alpha`` exists to prove that a folder
+# name is never matched as a prefix of another (``team_a`` must not catch it),
+# and ``root_dag.py`` lives at the bundle root (no folder).
+FOLDER_DAGS = {
+ "folder_dag_a_etl_extract": "team_a/etl/extract.py",
+ "folder_dag_a_etl_load": "team_a/etl/load.py",
+ "folder_dag_a_report": "team_a/report.py",
+ "folder_dag_b_ml_train": "team_b/ml/train.py",
+ "folder_dag_alpha": "team_alpha/x.py",
+ "folder_dag_root": "root_dag.py",
+}
+
+
+class TestDagFolders(TestPublicDagEndpoint):
Review Comment:
For tests, add an `assert_queries_count` just to lock in that this endpoint should only make a finite number of request (a couple for auth, + 1 for the payload), and should never scale with the number of dags.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "RabahAmrouche05 (via GitHub)" <gi...@apache.org>.
RabahAmrouche05 commented on PR #68544:
URL: https://github.com/apache/airflow/pull/68544#issuecomment-5050873268
Thanks @pierrejeambrun and @jedcunningham, good catch on the multi-bundle case! I updated the PR.
The folders endpoint now returns the bundle with each folder, since a folder path is relative to its bundle and can exist in more than one. When there are several bundles the tree shows the bundle first and the folders under it, so two bundles with the same path don't get mixed together. With a single bundle it stays flat like before.
Selecting a folder now uses its bundle too (with the existing bundle_name filter), so team_a/etl in two bundles points to the right dags.
Also rebased on main to fix the conflicts. Added tests for both (same path in two bundles staying separate, and the bundle grouping in the tree).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #68544:
URL: https://github.com/apache/airflow/pull/68544#discussion_r3795499792
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagsList.tsx:
##########
@@ -379,24 +420,39 @@ export const DagsList = () => {
</HStack>
</HStack>
</VStack>
- <Box pb={8}>
- <DataTable
- cardDef={cardDef}
- columns={columns}
- data={data?.dags ?? []}
- displayMode={display}
- errorMessage={<ErrorAlert error={error} />}
- initialState={tableURLState}
- isLoading={isLoading}
- modelName="common:dag"
- onDisplayToggleChange={setDisplay}
- onStateChange={setTableURLState}
- showDisplayToggle
- showRowCountHeading={false}
- skeletonCount={display === "card" ? 5 : undefined}
- total={totalEntries}
- />
- </Box>
+ <Flex align="flex-start" gap={4} pb={8}>
+ {/* Only show the folder sidebar when there is something to navigate; deployments with all
+ Dags at the bundle root would otherwise get an empty panel. */}
+ {showFolderTree ? (
+ <Box flexShrink={0} maxWidth="280px" overflowY="auto" position="sticky" top={0}>
+ <DagFolderTree
+ folders={folders}
+ isLoading={foldersLoading}
+ onSelectFolder={handleFolderChange}
+ selectedBundle={selectedBundle}
+ selectedFolder={selectedFolder}
+ />
+ </Box>
Review Comment:
Can you add a controler there (button switching to visible on/off) so we can actually hide / display this Tree?
If we want to retrieve a full width table to see multiple columns, it's better if we can visually hide the Tree.
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagsList.tsx:
##########
@@ -261,6 +273,14 @@ export const DagsList = () => {
const owners = searchParams.getAll(OWNERS);
const teams = searchParams.getAll(TEAMS);
const timetableType = searchParams.getAll(TIMETABLE_TYPE).filter((value) => value !== "");
+ const selectedFolder = searchParams.get(DAG_FOLDER) ?? undefined;
+ const selectedBundle = searchParams.get(DAG_BUNDLE) ?? undefined;
+
+ const { folders, isLoading: foldersLoading } = useDagFolders();
+ // Keep the panel out of the way for flat deployments (all Dags at the bundle root). Still show it
+ // while loading, or when a folder/bundle is selected so the user can always navigate back to "All Dags".
+ const showFolderTree =
+ foldersLoading || folders.length > 0 || Boolean(selectedFolder) || Boolean(selectedBundle);
Review Comment:
When linking to a filtered tree such as:
`http://localhost:28080/dags?dag_folder=subfolder1%2Fnested1%2Fnested2&dag_bundle=dags-folder`
The Tree stay collapsed, can we fix it so the tree appears expanded to the relevant folder.
test masks it by passing folders synchronously
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/DagFolderTree.tsx:
##########
Review Comment:
For long names, the highlighting is truncated like this, can we fix it:
<img width="407" height="280" alt="Image" src="https://github.com/user-attachments/assets/ec22a64d-4ae0-4baa-9081-48f1ea0a6bab" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "RabahAmrouche05 (via GitHub)" <gi...@apache.org>.
RabahAmrouche05 commented on code in PR #68544:
URL: https://github.com/apache/airflow/pull/68544#discussion_r3501905570
##########
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/DagFolderTree.tsx:
##########
@@ -0,0 +1,178 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import { Box, Flex, Heading, Skeleton, Text, VStack } from "@chakra-ui/react";
+import { useMemo, useState } from "react";
+import { useTranslation } from "react-i18next";
+import { FiChevronDown, FiChevronRight, FiFolder } from "react-icons/fi";
+
+import { buildFolderTree, type FolderNode } from "./buildFolderTree";
+
+type Props = {
+ readonly folders: ReadonlyArray<string>;
+ readonly isLoading?: boolean;
+ readonly onSelectFolder: (path: string | undefined) => void;
+ readonly selectedFolder: string | undefined;
+};
+
+// Every ancestor folder of the selected path, so the tree opens to reveal the selection.
+const ancestorPaths = (folder: string | undefined): Array<string> => {
+ if (folder === undefined || folder === "") {
+ return [];
+ }
+
+ const segments = folder.split("/");
+
+ return segments.map((_, index) => segments.slice(0, index + 1).join("/"));
+};
+
+type RowProps = {
+ readonly expanded: Set<string>;
+ readonly node: FolderNode;
+ readonly onSelectFolder: (path: string | undefined) => void;
+ readonly onToggle: (path: string) => void;
+ readonly selectedFolder: string | undefined;
+};
+
+const FolderRow = ({ expanded, node, onSelectFolder, onToggle, selectedFolder }: RowProps) => {
+ const hasChildren = node.children.length > 0;
+ const isExpanded = expanded.has(node.path);
+ const isSelected = selectedFolder === node.path;
+ const depth = node.path.split("/").length - 1;
+
+ return (
+ <Box>
+ <Flex
+ _hover={{ bg: isSelected ? "blue.subtle" : "bg.muted" }}
+ alignItems="center"
+ bg={isSelected ? "blue.subtle" : undefined}
+ borderRadius="sm"
+ cursor="pointer"
+ gap={1}
+ onClick={() => onSelectFolder(node.path)}
+ pl={`${depth * 16 + 4}px`}
+ py={1}
+ >
+ <Box
+ aria-label={isExpanded ? "Collapse" : "Expand"}
+ as="span"
+ color="fg.muted"
+ onClick={(event) => {
+ event.stopPropagation();
+ if (hasChildren) {
+ onToggle(node.path);
+ }
+ }}
+ visibility={hasChildren ? "visible" : "hidden"}
+ >
+ {isExpanded ? <FiChevronDown /> : <FiChevronRight />}
+ </Box>
+ <Box as="span" color="fg.muted">
+ <FiFolder />
+ </Box>
+ <Text fontWeight={isSelected ? "bold" : "normal"} truncate>
+ {node.name}
+ </Text>
+ </Flex>
+ {hasChildren && isExpanded ? (
+ <Box>
+ {node.children.map((child) => (
+ <FolderRow
+ expanded={expanded}
+ key={child.path}
+ node={child}
+ onSelectFolder={onSelectFolder}
+ onToggle={onToggle}
+ selectedFolder={selectedFolder}
+ />
+ ))}
+ </Box>
+ ) : undefined}
+ </Box>
+ );
+};
+
+export const DagFolderTree = ({ folders, isLoading = false, onSelectFolder, selectedFolder }: Props) => {
+ const { t: translate } = useTranslation("dags");
+ const tree = useMemo(() => buildFolderTree(folders), [folders]);
Review Comment:
Done — removed the `useMemo`, relying on react-compiler as per the repo convention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Group and filter Dags by folder on the Dags page [airflow]
Posted by "RabahAmrouche05 (via GitHub)" <gi...@apache.org>.
RabahAmrouche05 commented on PR #68544:
URL: https://github.com/apache/airflow/pull/68544#issuecomment-4782708054
Hi @pierrejeambrun No further work planned, it's ready for review. I've addressed @jscheffl 's feedback (added a screenshot + a short video of the feature, and removed the newsfragment).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on PR #68544:
URL: https://github.com/apache/airflow/pull/68544#issuecomment-5898586414
Ok let's do one last rebase before merging?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "RabahAmrouche05 (via GitHub)" <gi...@apache.org>.
RabahAmrouche05 commented on PR #68544:
URL: https://github.com/apache/airflow/pull/68544#issuecomment-5160091694
Thanks @bbovenzi @pierrejeambrun, switched the tree to the chakra TreeView component. No more onClick on divs, and we get keyboard navigation and the aria attributes for free.
I also fixed two things I hit while testing with a real multi bundle setup:
- Alignment: folders with children were shifted to the right compared to their siblings, because only they render an expand chevron. Leaf folders now render a hidden placeholder (same icon, visibility hidden + aria-hidden) so both line up on the same column.
- Stale dags: the folders endpoint now filters them out (is_stale == false), to match the dag list which hides stale dags by default. Before that, a bundle that stopped being parsed kept its folders in the tree, and clicking one gave you an empty list.
Both are covered by tests.
Here is the setup I used, two bundles that both have a team_a/etl folder:
<img width="241" height="251" alt="Capture d’écran 2026-08-01 à 23 42 31" src="https://github.com/user-attachments/assets/b50c7d67-3745-4712-841b-dfed7069ba4e" />
And the tree in the UI, each bundle keeping its own folders:
https://github.com/user-attachments/assets/7ebab2ab-a28b-4e68-8925-da39945dc87d
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Group and filter Dags by folder on the Dags page [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #68544:
URL: https://github.com/apache/airflow/pull/68544#issuecomment-4702656145
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our [Contributors' Guide](https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:
- Pay attention to the quality of your code (ruff, mypy and type annotations). Our [prek-hooks]( https://github.com/apache/airflow/blob/main/contributing-docs/08_static_code_checks.rst#prerequisites-for-prek-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/airflow-core/docs/howto/custom-operator.rst) Consider adding an example Dag that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/dev/breeze/doc/README.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#coding-style-and-best-practices).
- Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UI: Group and filter Dags by folder on the Dags page [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #68544:
URL: https://github.com/apache/airflow/pull/68544#discussion_r3410286289
##########
airflow-core/newsfragments/68544.feature.rst:
##########
Review Comment:
Newsfragment is only added for important changes. I think this is not needed here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "RabahAmrouche05 (via GitHub)" <gi...@apache.org>.
RabahAmrouche05 commented on code in PR #68544:
URL: https://github.com/apache/airflow/pull/68544#discussion_r3501954282
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dags.py:
##########
@@ -253,6 +258,36 @@ def get_dags(
)
+@dags_router.get(
+ "/folders",
+ dependencies=[Depends(requires_access_dag(method="GET"))],
+ operation_id="get_dag_folders",
+)
+def get_dag_folders(
+ readable_dags_filter: ReadableDagsFilterDep,
+ session: SessionDep,
+) -> DagFolderCollectionResponse:
+ """
+ Get the distinct folders the readable Dags live in.
Review Comment:
The list returned here is the distinct folders, not one entry per dag, so it stays small even with a lot of dags. It's also a single query (the new test checks the count doesn't grow when you add more dags).
I left it without pagination on purpose: the tree needs all the folders to build the hierarchy, and a limit would just hide some folders from the tree, which feels worse than the cost.
If you still think we need a safety limit I'm fine adding one with a warning log, just let me know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Group and filter Dags by folder on the Dags page [airflow]
Posted by "RabahAmrouche05 (via GitHub)" <gi...@apache.org>.
RabahAmrouche05 commented on code in PR #68544:
URL: https://github.com/apache/airflow/pull/68544#discussion_r3553923036
##########
airflow-core/src/airflow/api_fastapi/common/parameters.py:
##########
@@ -894,6 +894,43 @@ def depends(cls, owners: list[str] = Query(default_factory=list)) -> _OwnersFilt
return cls().set_value(owners)
+class _RelativeFilelocPrefixFilter(BaseParam[str | None]):
+ """
+ Filter Dags by the folder they live in, derived from ``relative_fileloc``.
+
+ The value is treated as a directory path relative to the bundle root (e.g.
+ ``team_a/etl``). It matches every Dag whose file lives directly in that folder
+ or in any of its subfolders, using an escaped ``LIKE 'team_a/etl/%'`` so a
+ folder name is never a substring/prefix of another (``team_a`` won't match
+ ``team_alpha``). Dags at the bundle root (no ``/`` in ``relative_fileloc``)
+ are not matched by any folder value and appear only when no folder is selected.
+ """
+
+ def to_orm(self, select: Select) -> Select:
+ if self.value is None and self.skip_none:
+ return select
+
+ if not self.value:
+ return select
+
+ directory = self.value.rstrip("/")
Review Comment:
That's a good question. `relative_fileloc` comes from `str(Path(file_path).relative_to(bundle_path))` in the dag processor (importers/base.py), so on the platforms Airflow runs its components on (posix) the separator is always `/`. And on posix `/` can't appear inside a file or folder name, so splitting on it isn't ambiguous. I'm also using `PurePosixPath(...).parent` to derive the folder, not a raw split.
The only case where this wouldn't hold is a Windows-style `\` path, which isn't a supported setup for the scheduler/dag processor afaik.
If we want to be extra safe we could normalize `relative_fileloc` to posix at the source (`.as_posix()`), but that touches every consumer of the field, so it feels like a separate change rather than something for this PR. Happy to follow whatever you and @jedcunningham think is best here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org