You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Pei-Cheng-Yu (via GitHub)" <gi...@apache.org> on 2026/02/24 08:28:02 UTC
[PR] Bulkdelete dagrun backend [airflow]
Pei-Cheng-Yu opened a new pull request, #62399:
URL: https://github.com/apache/airflow/pull/62399
## why
related #52439
according disscussion in [PR](https://github.com/apache/airflow/pull/60718)
we need a bulkdelete endpoint for dag run
## how
I implemented this by referencing the existing bulk action implementations in:
- `task_instance.py`
- `connection.py`
- `variable.py`
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
<!--
Thank you for contributing!
Please provide above a brief description of the changes made in this pull request.
Write a good git commit message following this guide: http://chris.beams.io/posts/git-commit/
Please make sure that your code changes are covered with tests.
And in case of new features or big changes remember to adjust the documentation.
Feel free to ping (in general) for the review if you do not see reaction for a few days
(72 Hours is the minimum reaction time you can expect from volunteers) - we sometimes miss notifications.
In case of an existing issue, reference it using one of the following:
* closes: #ISSUE
* related: #ISSUE
-->
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR, please
change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
-->
- [x] Yes (please specify the tool below)
Generated-by: [OpenAI Codex / GPT-5 ] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
for ideation and partial code generation. I reviewed and adjusted the changes to align with existing implementations of corresponding bulk actions elsewhere, and verified they pass `prek`.
I make sure that codex only modify the right part of code, and any thing about endpoint that requiring codegen were regenerated with `prek` and then pass the local prek test
---
* Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment: `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Bulkdelete dagrun backend [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #62399:
URL: https://github.com/apache/airflow/pull/62399#issuecomment-4038906790
@Pei-Cheng-Yu This PR has been converted to **draft** because it does not yet meet our [Pull Request quality criteria](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-quality-criteria).
**Issues found:**
- :x: **Merge conflicts**: This PR has merge conflicts with the `main` branch. Your branch is 209 commits behind `main`. Please rebase your branch (`git fetch origin && git rebase origin/main`), resolve the conflicts, and push again. See [contributing quick start](https://github.com/apache/airflow/blob/main/contributing-docs/03a_contributors_quick_start_beginners.rst).
- :warning: **Unresolved review comments**: This PR has 7 unresolved review threads from maintainers. Please review and resolve all inline review comments before requesting another review. You can resolve a conversation by clicking 'Resolve conversation' on each thread after addressing the feedback. See [pull request guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst).
> **Note:** Your branch is **209 commits behind `main`**. Some check failures may be caused by changes in the base branch rather than by your PR. Please rebase your branch and push again to get up-to-date CI results.
**What to do next:**
- The comment informs you what you need to do.
- Fix each issue, then mark the PR as "Ready for review" in the GitHub UI - but only after making sure that all the issues are fixed.
- Maintainers will then proceed with a normal review.
Converting a PR to draft is **not** a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. If you have questions, feel free to ask on the [Airflow Slack](https://s.apache.org/airflow-slack).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Bulkdelete dagrun backend [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #62399:
URL: https://github.com/apache/airflow/pull/62399#issuecomment-4387605671
@Pei-Cheng-Yu This draft PR has been inactive for 13 days since the last triage comment and no response from the author. Closing to keep the queue clean.
You are welcome to reopen this PR when you resume work, or to open a new one addressing the issues previously raised. There is no rush — take your time.
---
_Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this [two-stage triage process](https://github.com/apache/airflow/blob/main/contributing-docs/25_maintainer_pr_triage.md#why-the-first-pass-is-automated) so that our maintainers' limited time is spent where it matters most: the conversation with you._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Bulkdelete dagrun backend [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #62399:
URL: https://github.com/apache/airflow/pull/62399#issuecomment-4296165323
@Pei-Cheng-Yu This PR has a few issues that need to be addressed before it can be reviewed — please see our [Pull Request quality criteria](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-quality-criteria).
**Issues found:**
- :x: **Merge conflicts**: This branch now has conflicts with `main` and cannot be merged as-is. Please rebase your branch onto the current `main` and resolve the conflicts: `git fetch upstream main && git rebase upstream/main`, resolve, force-push.
- :x: **Failing CI checks**: `Additional PROD image tests / WebKit UI e2e tests`. Please investigate and fix.
**What to do next:**
- Rebase the branch onto the current `main` and resolve the merge conflicts.
- Re-run static checks locally (`prek run --from-ref main --stage pre-commit`).
- Push and let CI re-run. Mark the PR as "Ready for review" when you're done.
There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates. If you have questions, feel free to ask on the [Airflow Slack](https://s.apache.org/airflow-slack).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Bulkdelete dagrun backend [airflow]
Posted by "Pei-Cheng-Yu (via GitHub)" <gi...@apache.org>.
Pei-Cheng-Yu commented on PR #62399:
URL: https://github.com/apache/airflow/pull/62399#issuecomment-4062752376
Thanks, I understand now.
I followed the earlier suggestion and interpreted it in a way using `BulkBody[DAGRunPatchBody]` that kept the implementation effectively single-DAG, just with `dag_id` available from the payload for auth.
After working through it, I realized that does not actually achieve the intended cross-DAG bulk delete behavior.
I should have stopped and clarified the direction when that became ambiguous. I just rework it to align with the `bulk_task_instances` pattern.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Bulkdelete dagrun backend [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed pull request #62399: Bulkdelete dagrun backend
URL: https://github.com/apache/airflow/pull/62399
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Bulkdelete dagrun backend [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #62399:
URL: https://github.com/apache/airflow/pull/62399#discussion_r2853207282
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/dag_run.py:
##########
@@ -152,6 +154,22 @@ def delete_dag_run(dag_id: str, dag_run_id: str, session: SessionDep):
session.delete(dag_run)
+@dag_run_router.patch(
+ "",
+ dependencies=[
+ Depends(requires_access_dag(method="DELETE", access_entity=DagAccessEntity.RUN)),
+ Depends(action_logging()),
+ ],
+)
+def bulk_dag_runs(
+ dag_id: str,
+ request: BulkBody[BulkDagRunBody],
Review Comment:
This isn't consistent with other implementation. This should be:
```suggestion
request: BulkBody[DAGRunPatchBody],
```
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/dag_run.py:
##########
@@ -152,6 +154,22 @@ def delete_dag_run(dag_id: str, dag_run_id: str, session: SessionDep):
session.delete(dag_run)
+@dag_run_router.patch(
+ "",
+ dependencies=[
+ Depends(requires_access_dag(method="DELETE", access_entity=DagAccessEntity.RUN)),
Review Comment:
Permission is wrong. That should be `requires_access_dag_run_bulk` and you need to impement that similarly to connections variables and pools
##########
airflow-core/src/airflow/api_fastapi/core_api/services/public/dag_run.py:
##########
@@ -89,3 +99,84 @@ async def wait(self) -> AsyncGenerator[str, None]:
await asyncio.sleep(self.interval)
yield self._serialize_response(dag_run := await self._get_dag_run())
yield "\n"
+
+
+class BulkDagRunService(BulkService[BulkDagRunBody]):
+ """Service for handling bulk operations on dag runs."""
+
+ def _categorize_dag_runs(
+ self, dag_run_keys: set[tuple[str, str]]
Review Comment:
Maybe extract the `tuple[str, str]` into a reusable type `DagRunKey` with a comment there it will be more readable for functions signatures
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/dag_run.py:
##########
@@ -152,6 +154,22 @@ def delete_dag_run(dag_id: str, dag_run_id: str, session: SessionDep):
session.delete(dag_run)
+@dag_run_router.patch(
+ "",
+ dependencies=[
+ Depends(requires_access_dag(method="DELETE", access_entity=DagAccessEntity.RUN)),
+ Depends(action_logging()),
+ ],
+)
+def bulk_dag_runs(
+ dag_id: str,
+ request: BulkBody[BulkDagRunBody],
+ session: SessionDep,
+) -> BulkResponse:
+ """Bulk delete dag runs."""
Review Comment:
Docstring not in line with the method name. Method name suggests 'bulk' operation. The docstring only mentions 'delete'.
Just state that other bulk operation are 'no implemented' and raise a 'not implemented error'.
##########
airflow-core/src/airflow/api_fastapi/core_api/services/public/dag_run.py:
##########
@@ -89,3 +99,84 @@ async def wait(self) -> AsyncGenerator[str, None]:
await asyncio.sleep(self.interval)
yield self._serialize_response(dag_run := await self._get_dag_run())
yield "\n"
+
+
+class BulkDagRunService(BulkService[BulkDagRunBody]):
+ """Service for handling bulk operations on dag runs."""
+
+ def _categorize_dag_runs(
Review Comment:
Why a private function? Other services categorize function isn't private.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Bulkdelete dagrun backend [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #62399:
URL: https://github.com/apache/airflow/pull/62399#discussion_r2907413731
##########
airflow-core/src/airflow/api_fastapi/core_api/security.py:
##########
@@ -179,6 +181,36 @@ def inner(
return inner
+def requires_access_dag_run_bulk() -> Callable[[str, BulkBody[DAGRunPatchBody], BaseUser], None]:
+ def inner(
+ dag_id: str,
+ request: BulkBody[DAGRunPatchBody],
+ user: GetUserDep,
+ ) -> None:
+ normalized_dag_id: str | None = dag_id if dag_id != "~" else None
+ team_name = DagModel.get_team_name(normalized_dag_id) if normalized_dag_id else None
+
+ requests: list[IsAuthorizedDagRequest] = []
+ for action in request.actions:
+ methods = _get_resource_methods_from_bulk_request(action)
+ for method in methods:
+ req: IsAuthorizedDagRequest = {
+ "method": method,
+ "access_entity": DagAccessEntity.RUN,
+ "details": DagDetails(id=normalized_dag_id, team_name=team_name),
Review Comment:
dag_id should actually come from the payload so we can delete cross dags.
##########
airflow-core/src/airflow/api_fastapi/core_api/datamodels/dag_run.py:
##########
@@ -179,3 +179,10 @@ class DAGRunsBatchBody(StrictBaseModel):
duration_lt: float | None = None
conf_contains: str | None = None
+
+
+class BulkDagRunBody(StrictBaseModel):
Review Comment:
This is unused, but should be used I think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Bulkdelete dagrun backend [airflow]
Posted by "Pei-Cheng-Yu (via GitHub)" <gi...@apache.org>.
Pei-Cheng-Yu commented on PR #62399:
URL: https://github.com/apache/airflow/pull/62399#issuecomment-4012151711
Thanks for the review!
## Update
I think that I just misread the router pattern and used the wrong bulk body type, I just change to the correct one mention in the suggestion
as for the `requires_access_dag_run_bulk` part, the implementation follows the same bulk auth pattern, but because this endpoint is DAG-scoped (/dags/{dag_id}/dagRuns), if i understand it correctly, the authorization is resolved from path dag_id instead of per-entity keys.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org