You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "guan404ming (via GitHub)" <gi...@apache.org> on 2026/02/13 04:04:11 UTC
[PR] Add deny_dag_run_types option to limit specific dag run types [airflow]
guan404ming opened a new pull request, #61833:
URL: https://github.com/apache/airflow/pull/61833
<!--
Thank you for contributing!
Please provide above a brief description of the changes made in this pull request.
Write a good git commit message following this guide: http://chris.beams.io/posts/git-commit/
Please make sure that your code changes are covered with tests.
And in case of new features or big changes remember to adjust the documentation.
Feel free to ping (in general) for the review if you do not see reaction for a few days
(72 Hours is the minimum reaction time you can expect from volunteers) - we sometimes miss notifications.
In case of an existing issue, reference it using one of the following:
* closes: #ISSUE
* related: #ISSUE
-->
## Related Issue
Closes #61710
## Why
Allow DAG authors to restrict which run types (manual, scheduled, backfill, asset-triggered) are permitted on a per-DAG basis.
## How
- Add deny_dag_run_types field to DAG definition, DagModel, serialization, and a DB migration
- Enforce the deny list in scheduler, trigger, backfill, and asset materialization API routes
- Disable the trigger button in the UI when manual runs are denied
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR, please
change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
-->
- [x] Yes (please specify the tool below)
<!--
Generated-by: [Tool Name] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
---
* Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment: `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add deny_dag_run_types option to limit specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #61833:
URL: https://github.com/apache/airflow/pull/61833#issuecomment-3907115910
> Test should verify broken dag message is raised for all these 3 cases
Thanks, just updated to make the tests covering more cased.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add deny_dag_run_types option to limit specific dag run types [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2812753804
##########
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dag_run.py:
##########
@@ -1541,8 +1541,20 @@ def _dags_for_trigger_tests(self, session=None):
)
import_errors_dag.has_import_errors = True
+ deny_manual_dag = DagModel(
+ dag_id="deny_manual",
+ bundle_name="testing",
+ fileloc="/tmp/dag_del_3.py",
Review Comment:
nit:
```suggestion
allowed_scheduled_dag = DagModel(
dag_id="allowed_scheduled",
bundle_name="testing",
fileloc="/tmp/dag_del_3.py",
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2813134371
##########
airflow-core/src/airflow/jobs/scheduler_job_runner.py:
##########
@@ -2060,6 +2082,16 @@ def _create_dag_runs_asset_triggered(
)
continue
+ if (
+ dag_model.allowed_run_types is not None
+ and DagRunType.ASSET_TRIGGERED.value not in dag_model.allowed_run_types
+ ):
+ self.log.warning(
+ "Dag does not allow asset-triggered runs; skipping",
Review Comment:
Make sense to me, just moved to the parse time check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2813131458
##########
airflow-core/src/airflow/models/backfill.py:
##########
@@ -100,6 +100,14 @@ class InvalidBackfillDate(AirflowException):
"""
+class DeniedDagRunType(AirflowException):
Review Comment:
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2815350150
##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/dag_runs.py:
##########
@@ -112,6 +112,15 @@ def trigger_dag_run(
},
)
+ if dm.allowed_run_types is not None and DagRunType.MANUAL.value not in dm.allowed_run_types:
+ raise HTTPException(
+ status.HTTP_400_BAD_REQUEST,
+ detail={
+ "reason": "denied_run_type",
+ "message": f"Dag with dag_id '{dag_id}' does not allow manual runs",
+ },
+ )
+
Review Comment:
This check has been removed. Parse-time validation `in _validate_allowed_run_types` already raises a ValueError if an asset-scheduled DAG doesn't include `ASSET_TRIGGERED` in `allowed_run_types`, so the runtime check was redundant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2815362696
##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/dag_runs.py:
##########
@@ -112,6 +112,15 @@ def trigger_dag_run(
},
)
+ if dm.allowed_run_types is not None and DagRunType.MANUAL.value not in dm.allowed_run_types:
+ raise HTTPException(
+ status.HTTP_400_BAD_REQUEST,
+ detail={
+ "reason": "denied_run_type",
+ "message": f"Dag with dag_id '{dag_id}' does not allow manual runs",
+ },
+ )
+
Review Comment:
Good catch. `TriggerDagRunOperator` creates `MANUAL` runs, so the check is consistent with the current behavior. But you're right that operator-triggered runs are semantically different from user-initiated manual runs. A dedicated run type for operator-triggered runs could address this, but that's a larger change. I just added a TODO comment for now and I think I could follow up one to update more dedicated run type here.
##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/dag_runs.py:
##########
@@ -112,6 +112,15 @@ def trigger_dag_run(
},
)
+ if dm.allowed_run_types is not None and DagRunType.MANUAL.value not in dm.allowed_run_types:
+ raise HTTPException(
+ status.HTTP_400_BAD_REQUEST,
+ detail={
+ "reason": "denied_run_type",
+ "message": f"Dag with dag_id '{dag_id}' does not allow manual runs",
+ },
+ )
+
Review Comment:
Good catch. Currently `TriggerDagRunOperator` creates `MANUAL` runs, so the check is consistent with the current behavior. But you're right that operator-triggered runs are semantically different from user-initiated manual runs. A dedicated run type for operator-triggered runs could address this, but that's a larger change. I just added a TODO comment for now and I think I could follow up one to update more dedicated run type here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2815859545
##########
task-sdk/src/airflow/sdk/definitions/dag.py:
##########
@@ -590,6 +610,29 @@ def _validate_tags(self, _, tags: Collection[str]):
if tags and any(len(tag) > TAG_MAX_LEN for tag in tags):
raise ValueError(f"tag cannot be longer than {TAG_MAX_LEN} characters")
+ @allowed_run_types.validator
+ def _validate_allowed_run_types(self, _, allowed_run_types):
+ if not allowed_run_types:
+ return
+ from airflow.utils.types import DagRunType
Review Comment:
We should avoid using imports from `airflow-core` in task sdk going forward. It is best to use `airflow.sdk.api.datamodels._generated` as @jason810496 mentioned.
> However the existing code in this file already imports DagRunType from airflow.utils.types
Regarding this, we plan to get rid of it in these ticket: https://github.com/apache/airflow/issues/54713
Switching should not be too hard, just do it here, the other refs can be handled in the related ticket
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add deny_dag_run_types option to limit specific dag run types [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #61833:
URL: https://github.com/apache/airflow/pull/61833#issuecomment-3903876183
Great work!
I'm having an issue with trying to run:
```
dag = DAG(
...
schedule="@daily",
deny_dag_run_types=DagRunType.MANUAL
)
```
I get
```
Traceback (most recent call last):
File "/usr/python/lib/python3.11/enum.py", line 714, in __call__
return cls.__new__(cls, value)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/python/lib/python3.11/enum.py", line 1137, in __new__
raise ve_exc
ValueError: 'm' is not a valid DagRunType
```
It's solved if I change `deny_dag_run_types=[DagRunType.MANUAL]`.
I think we should allow also a single value... It makes things easier and shouldn't be too hard to fix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add deny_dag_run_types option to limit specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #61833:
URL: https://github.com/apache/airflow/pull/61833#issuecomment-3908590196
Thanks for the suggestions and discussion
1. Agree that an allowlist (whitelist) is more reasonable from a security perspective. It may require a bit more effort from users, but I think the safety tradeoff is worth it.
2. Good point on the naming. Will rename to `allowed_run_types`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2813112511
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/assets.py:
##########
@@ -402,6 +403,13 @@ def materialize_asset(
f"More than one DAG materializes asset with ID: {asset_id}",
)
+ dm = session.scalar(select(DagModel).where(DagModel.dag_id == dag_id).limit(1))
Review Comment:
> I'm not 100% sure this is right for this route?
I updated with moving the check after `get_latest_version_of_dag` and use `dag.allowed_run_types` directly from the `SerializedDAG`, removing the extra `DagModel` query.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2817944063
##########
task-sdk/src/airflow/sdk/definitions/dag.py:
##########
@@ -590,6 +610,29 @@ def _validate_tags(self, _, tags: Collection[str]):
if tags and any(len(tag) > TAG_MAX_LEN for tag in tags):
raise ValueError(f"tag cannot be longer than {TAG_MAX_LEN} characters")
+ @allowed_run_types.validator
+ def _validate_allowed_run_types(self, _, allowed_run_types):
+ if not allowed_run_types:
+ return
+ from airflow.utils.types import DagRunType
Review Comment:
Just updated, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "roykoand (via GitHub)" <gi...@apache.org>.
roykoand commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2815865992
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/dag_run.py:
##########
@@ -459,6 +459,12 @@ def trigger_dag_run(
f"DAG with dag_id: '{dag_id}' has import errors and cannot be triggered",
)
+ if dm.allowed_run_types is not None and DagRunType.MANUAL.value not in dm.allowed_run_types:
Review Comment:
```suggestion
if dm.allowed_run_types is not None and DagRunType.MANUAL not in dm.allowed_run_types:
```
Wouldn't it work without a `value`? It seems like accessing enum value throught the pr is not consistent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add deny_dag_run_types option to limit specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2811225477
##########
task-sdk/src/airflow/sdk/definitions/dag.py:
##########
Review Comment:
Thanks for the great catch. Just updated!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "jason810496 (via GitHub)" <gi...@apache.org>.
jason810496 commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2813089003
##########
task-sdk/src/airflow/sdk/definitions/dag.py:
##########
@@ -590,6 +610,29 @@ def _validate_tags(self, _, tags: Collection[str]):
if tags and any(len(tag) > TAG_MAX_LEN for tag in tags):
raise ValueError(f"tag cannot be longer than {TAG_MAX_LEN} characters")
+ @allowed_run_types.validator
+ def _validate_allowed_run_types(self, _, allowed_run_types):
+ if not allowed_run_types:
+ return
+ from airflow.utils.types import DagRunType
Review Comment:
Maybe import from `airflow.sdk.api.datamodels._generated` instead of airflow-core?
##########
airflow-core/src/airflow/jobs/scheduler_job_runner.py:
##########
@@ -1810,6 +1810,18 @@ def _create_dagruns_for_partitioned_asset_dags(self, session: Session) -> set[st
self.log.error("Dag '%s' not found in serialized_dag table", apdr.target_dag_id)
continue
+ dag_model = session.scalar(select(DagModel).where(DagModel.dag_id == apdr.target_dag_id).limit(1))
+ if (
+ dag_model
+ and dag_model.allowed_run_types is not None
+ and DagRunType.ASSET_TRIGGERED.value not in dag_model.allowed_run_types
+ ):
+ self.log.warning(
+ "Dag does not allow asset-triggered runs; skipping",
+ dag_id=apdr.target_dag_id,
+ )
+ continue
Review Comment:
It seems we could modularize the check and the warning.
##########
airflow-core/src/airflow/models/backfill.py:
##########
@@ -100,6 +100,14 @@ class InvalidBackfillDate(AirflowException):
"""
+class DeniedDagRunType(AirflowException):
Review Comment:
Maybe `models.dagrun` or `models.dag` module is a better place for `DeniedDagRunType`?
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/assets.py:
##########
@@ -402,6 +403,13 @@ def materialize_asset(
f"More than one DAG materializes asset with ID: {asset_id}",
)
+ dm = session.scalar(select(DagModel).where(DagModel.dag_id == dag_id).limit(1))
+ if dm and dm.allowed_run_types is not None and DagRunType.MANUAL.value not in dm.allowed_run_types:
Review Comment:
Not sure whether we should treat β`materialize_asset` as β`DagRunType.ASSET_TRIGGERED` or β`DagRunType.MANUAL` π
##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/dag_runs.py:
##########
@@ -112,6 +112,15 @@ def trigger_dag_run(
},
)
+ if dm.allowed_run_types is not None and DagRunType.MANUAL.value not in dm.allowed_run_types:
+ raise HTTPException(
+ status.HTTP_400_BAD_REQUEST,
+ detail={
+ "reason": "denied_run_type",
+ "message": f"Dag with dag_id '{dag_id}' does not allow manual runs",
+ },
+ )
+
Review Comment:
Thereβs an edge case where the β`TriggerDagRunOperator` task will also call the β`trigger_dag_run` route, but we can also choose not to set β`manual` in β`allowed_run_types` in the same Dag.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #61833:
URL: https://github.com/apache/airflow/pull/61833#issuecomment-3934759467
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add deny_dag_run_types option to limit specific dag run types [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #61833:
URL: https://github.com/apache/airflow/pull/61833#issuecomment-3907665600
> I'll try to clarify - Let's say that using this parameter you allow only asset runs (`allowed_run_types = ['assets']`). At some future point you decide that you want to re-add scheduled runs (`allowed_run_types =['assets', 'scheduled']`).
>
> 1. Do we respect the `catchup` parameter in this case to complete past runs? (see followup)
> 2. If we do, are **all** past scheduled runs queued, or only those between setting the initial `allowed_run_types` and updating it? You might had set `catchup=False` before adding `allowed_run_types`, and if you now change both `catchup=True` and allow scheduled runs, there will be a lot of potentially unnecessary runs for the users.
>
> We could keep it simple for now and make the required assumptions, but it should be documented IMO.
Catachup is not the same as backfill.
Catchup works only after the latest existed dagrun. In your example changing to catchup=true will do bothing regardless of the new flag.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2813113907
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/assets.py:
##########
@@ -402,6 +403,13 @@ def materialize_asset(
f"More than one DAG materializes asset with ID: {asset_id}",
)
+ dm = session.scalar(select(DagModel).where(DagModel.dag_id == dag_id).limit(1))
Review Comment:
> I almost wonder if those should be a different type to "manual" here
Agree this is worth considering. For now it creates a DagRunType.MANUAL run so the check is consistent, but a dedicated materialization run type could make sense as a follow-up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2813113907
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/assets.py:
##########
@@ -402,6 +403,13 @@ def materialize_asset(
f"More than one DAG materializes asset with ID: {asset_id}",
)
+ dm = session.scalar(select(DagModel).where(DagModel.dag_id == dag_id).limit(1))
Review Comment:
Agree this is worth considering. For now it creates a DagRunType.MANUAL run so the check is consistent, but a dedicated materialization run type could make sense as a follow-up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2815353632
##########
airflow-core/src/airflow/jobs/scheduler_job_runner.py:
##########
@@ -1810,6 +1810,18 @@ def _create_dagruns_for_partitioned_asset_dags(self, session: Session) -> set[st
self.log.error("Dag '%s' not found in serialized_dag table", apdr.target_dag_id)
continue
+ dag_model = session.scalar(select(DagModel).where(DagModel.dag_id == apdr.target_dag_id).limit(1))
+ if (
+ dag_model
+ and dag_model.allowed_run_types is not None
+ and DagRunType.ASSET_TRIGGERED.value not in dag_model.allowed_run_types
+ ):
+ self.log.warning(
+ "Dag does not allow asset-triggered runs; skipping",
+ dag_id=apdr.target_dag_id,
+ )
+ continue
Review Comment:
This check has been removed. Parse-time validation `in _validate_allowed_run_types` already raises a ValueError if an asset-scheduled dag doesn't include `ASSET_TRIGGERED` in `allowed_run_types`, so the runtime check was redundant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2812848680
##########
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dag_run.py:
##########
@@ -1541,8 +1541,20 @@ def _dags_for_trigger_tests(self, session=None):
)
import_errors_dag.has_import_errors = True
+ deny_manual_dag = DagModel(
+ dag_id="deny_manual",
+ bundle_name="testing",
+ fileloc="/tmp/dag_del_3.py",
Review Comment:
Thanks just updated!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "jason810496 (via GitHub)" <gi...@apache.org>.
jason810496 commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2813097392
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/assets.py:
##########
@@ -402,6 +403,13 @@ def materialize_asset(
f"More than one DAG materializes asset with ID: {asset_id}",
)
+ dm = session.scalar(select(DagModel).where(DagModel.dag_id == dag_id).limit(1))
+ if dm and dm.allowed_run_types is not None and DagRunType.MANUAL.value not in dm.allowed_run_types:
Review Comment:
Same as Ash pointed out, resolved.
##########
airflow-core/src/airflow/models/backfill.py:
##########
@@ -100,6 +100,14 @@ class InvalidBackfillDate(AirflowException):
"""
+class DeniedDagRunType(AirflowException):
Review Comment:
Same as Ash pointed out, resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2815344531
##########
task-sdk/src/airflow/sdk/definitions/dag.py:
##########
@@ -590,6 +610,29 @@ def _validate_tags(self, _, tags: Collection[str]):
if tags and any(len(tag) > TAG_MAX_LEN for tag in tags):
raise ValueError(f"tag cannot be longer than {TAG_MAX_LEN} characters")
+ @allowed_run_types.validator
+ def _validate_allowed_run_types(self, _, allowed_run_types):
+ if not allowed_run_types:
+ return
+ from airflow.utils.types import DagRunType
Review Comment:
Good suggestion. However the existing code in this file already imports `DagRunType` from `airflow.utils.types` (line 1222), so this is consistent with the current pattern. Also, the two `DagRunType` enums have different member names (`BACKFILL_JOB` vs `BACKFILL`), so switching would need some care. Maybe better addressed as a separate cleanup across the file. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2821210011
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/assets.py:
##########
@@ -402,6 +403,13 @@ def materialize_asset(
f"More than one DAG materializes asset with ID: {asset_id}",
)
+ dm = session.scalar(select(DagModel).where(DagModel.dag_id == dag_id).limit(1))
Review Comment:
Shall we open a followup task or leave a todo with instructions in the code? Otherwise we wil lose track of it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "jason810496 (via GitHub)" <gi...@apache.org>.
jason810496 commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2828451399
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/backfills.py:
##########
@@ -252,6 +252,11 @@ def create_backfill(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Could not find dag {backfill_request.dag_id}",
)
+ except DagRunTypeNotAllowed as e:
+ raise HTTPException(
+ status_code=status.HTTP_400_BAD_REQUEST,
Review Comment:
We need to update the `create_openapi_http_exception_doc` for the route definition.
##########
task-sdk/src/airflow/sdk/definitions/dag.py:
##########
@@ -590,6 +610,29 @@ def _validate_tags(self, _, tags: Collection[str]):
if tags and any(len(tag) > TAG_MAX_LEN for tag in tags):
raise ValueError(f"tag cannot be longer than {TAG_MAX_LEN} characters")
+ @allowed_run_types.validator
+ def _validate_allowed_run_types(self, _, allowed_run_types):
+ if not allowed_run_types:
+ return
+ from airflow.sdk.api.datamodels._generated import DagRunType
+
+ if isinstance(self.timetable, AssetTriggeredTimetable):
+ if DagRunType.ASSET_TRIGGERED not in allowed_run_types:
+ raise ValueError(
+ "allowed_run_types must include ASSET_TRIGGERED when the DAG is scheduled by assets"
+ )
+ elif self.timetable.can_be_scheduled:
+ if DagRunType.SCHEDULED not in allowed_run_types:
+ raise ValueError(
+ "allowed_run_types must include SCHEDULED when the DAG has a schedule defined"
+ )
+ else:
+ if DagRunType.MANUAL not in allowed_run_types:
+ raise ValueError(
+ "allowed_run_types must include MANUAL when the DAG "
Review Comment:
```suggestion
"allowed_run_types must include ASSET_TRIGGERED when the Dag is scheduled by assets"
)
elif self.timetable.can_be_scheduled:
if DagRunType.SCHEDULED not in allowed_run_types:
raise ValueError(
"allowed_run_types must include SCHEDULED when the Dag has a schedule defined"
)
else:
if DagRunType.MANUAL not in allowed_run_types:
raise ValueError(
"allowed_run_types must include MANUAL when the Dag "
```
##########
airflow-core/src/airflow/ui/public/i18n/locales/en/components.json:
##########
@@ -130,6 +130,7 @@
"intervalStart": "Start",
"loading": "Loading Dag information...",
"loadingFailed": "Failed to load Dag information. Please try again.",
+ "manualRunDenied": "Manual runs are not allowed for this DAG",
Review Comment:
```suggestion
"manualRunDenied": "Manual runs are not allowed for this Dag",
```
##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/dag_runs.py:
##########
@@ -112,6 +112,17 @@ def trigger_dag_run(
},
)
+ # TODO: TriggerDagRunOperator also calls this route but creates MANUAL runs.
+ # Consider a dedicated run type for operator-triggered runs.
+ if dm.allowed_run_types is not None and DagRunType.MANUAL not in dm.allowed_run_types:
+ raise HTTPException(
+ status.HTTP_400_BAD_REQUEST,
Review Comment:
We need to update the description of `HTTP_400_BAD_REQUEST` about `denied_run_type`.
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/assets.py:
##########
@@ -403,6 +403,13 @@ def materialize_asset(
)
dag = get_latest_version_of_dag(dag_bag, dag_id, session)
+
+ if dag.allowed_run_types is not None and DagRunType.MANUAL not in dag.allowed_run_types:
+ raise HTTPException(
+ status.HTTP_400_BAD_REQUEST,
Review Comment:
Same here.
##########
airflow-core/src/airflow/migrations/versions/0104_3_2_0_add_allowed_run_types_to_dag.py:
##########
@@ -0,0 +1,50 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Add allowed_run_types to dag.
+
+Revision ID: e42d9fcd10d9
+Revises: f8c9d7e6b5a4
+Create Date: 2026-02-12 11:49:40.753440
+
+"""
+
+from __future__ import annotations
+
+import sqlalchemy as sa
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision = "e42d9fcd10d9"
+down_revision = "f8c9d7e6b5a4"
+branch_labels = None
+depends_on = None
+airflow_version = "3.2.0"
+
+
+def upgrade():
+ """Add allowed_run_types column to dag table."""
+ with op.batch_alter_table("dag", schema=None) as batch_op:
+ batch_op.add_column(sa.Column("allowed_run_types", sa.JSON(), nullable=True))
Review Comment:
I had the same thought as Ash during last review, maybe we could track this in a new issue about the ARRAY <-> JSON trade-off for list-type columns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming merged PR #61833:
URL: https://github.com/apache/airflow/pull/61833
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add deny_dag_run_types option to limit specific dag run types [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2808882174
##########
task-sdk/src/airflow/sdk/definitions/dag.py:
##########
Review Comment:
Missing
`:param deny_dag_run_types` in the DAG class docstring
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add deny_dag_run_types option to limit specific dag run types [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #61833:
URL: https://github.com/apache/airflow/pull/61833#issuecomment-3907524688
> It should be clarified and documented how it behaves when a Dag type, or the entire parameter, is removed (e.g., does it respect the catchup parameter?).
I think the logic is pretty simple. This parameter prevents creation of runs. There are no exceptions thus if you prevent schedule runs all the settings that specify which scheduled runs should be created are ignored.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2815350150
##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/dag_runs.py:
##########
@@ -112,6 +112,15 @@ def trigger_dag_run(
},
)
+ if dm.allowed_run_types is not None and DagRunType.MANUAL.value not in dm.allowed_run_types:
+ raise HTTPException(
+ status.HTTP_400_BAD_REQUEST,
+ detail={
+ "reason": "denied_run_type",
+ "message": f"Dag with dag_id '{dag_id}' does not allow manual runs",
+ },
+ )
+
Review Comment:
This check has been removed. Parse-time validation `in _validate_allowed_run_types` already raises a ValueError if an asset-scheduled DAG doesn't include `ASSET_TRIGGERED` in `allowed_run_types`, so the runtime check was redundant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #61833:
URL: https://github.com/apache/airflow/pull/61833#issuecomment-3928322896
Hi @jason810496, thanks for your great catch. I just updated with your comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2817871805
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/dag_run.py:
##########
@@ -459,6 +459,12 @@ def trigger_dag_run(
f"DAG with dag_id: '{dag_id}' has import errors and cannot be triggered",
)
+ if dm.allowed_run_types is not None and DagRunType.MANUAL.value not in dm.allowed_run_types:
Review Comment:
Make sense, thanks. Just updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "roykoand (via GitHub)" <gi...@apache.org>.
roykoand commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2815865992
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/dag_run.py:
##########
@@ -459,6 +459,12 @@ def trigger_dag_run(
f"DAG with dag_id: '{dag_id}' has import errors and cannot be triggered",
)
+ if dm.allowed_run_types is not None and DagRunType.MANUAL.value not in dm.allowed_run_types:
Review Comment:
```suggestion
if dm.allowed_run_types is not None and DagRunType.MANUAL not in dm.allowed_run_types:
```
Wouldn't it work without a `value`? It seems like accessing enum value throught the pr is not consistent?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #61833:
URL: https://github.com/apache/airflow/pull/61833#issuecomment-3927381143
Yes, I think I have resolved all the comments and we could refine it if there is any issue after merging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2845563492
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/assets.py:
##########
@@ -402,6 +403,13 @@ def materialize_asset(
f"More than one DAG materializes asset with ID: {asset_id}",
)
+ dm = session.scalar(select(DagModel).where(DagModel.dag_id == dag_id).limit(1))
Review Comment:
Late to the discussion. It's ok to be manual, but it might be better to have a separate type. now reviewing the PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2813130623
##########
airflow-core/src/airflow/models/backfill.py:
##########
@@ -502,6 +510,14 @@ def _create_backfill(
if not serdag:
raise DagNotFound(f"Could not find dag {dag_id}")
+ dag_model = session.scalar(select(DagModel).where(DagModel.dag_id == dag_id).limit(1))
Review Comment:
Yes, I found that we could reused the already-fetched `dag_model` to check `dag_model.timetable_summary` == "None" instead of a separate `select(func.count())` query. Combined into one if `dag_model:` block with both checks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2813112511
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/assets.py:
##########
@@ -402,6 +403,13 @@ def materialize_asset(
f"More than one DAG materializes asset with ID: {asset_id}",
)
+ dm = session.scalar(select(DagModel).where(DagModel.dag_id == dag_id).limit(1))
Review Comment:
I updated with moving the check after `get_latest_version_of_dag` and use `dag.allowed_run_types` directly from the `SerializedDAG`, removing the extra `DagModel` query.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2812911121
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/assets.py:
##########
@@ -402,6 +403,13 @@ def materialize_asset(
f"More than one DAG materializes asset with ID: {asset_id}",
)
+ dm = session.scalar(select(DagModel).where(DagModel.dag_id == dag_id).limit(1))
Review Comment:
I'm not 100% sure this is right for this route?
Ptal @Lee-W
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #61833:
URL: https://github.com/apache/airflow/pull/61833#issuecomment-3926636683
@guan404ming did you address all comments? are we ready to merge?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #61833:
URL: https://github.com/apache/airflow/pull/61833#issuecomment-3909273255
Thanks for all the comments. I just updated with @ashb review comments and will handle @jason810496's tomorrow!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2812915140
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/assets.py:
##########
@@ -402,6 +403,13 @@ def materialize_asset(
f"More than one DAG materializes asset with ID: {asset_id}",
)
+ dm = session.scalar(select(DagModel).where(DagModel.dag_id == dag_id).limit(1))
Review Comment:
I almost wonder if those should be a different type to "manual" here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2812945393
##########
airflow-core/src/airflow/models/backfill.py:
##########
@@ -100,6 +100,14 @@ class InvalidBackfillDate(AirflowException):
"""
+class DeniedDagRunType(AirflowException):
Review Comment:
This is nothing to do with backfills, so this should be elsewhere
##########
airflow-core/src/airflow/migrations/versions/0104_3_2_0_add_allowed_run_types_to_dag.py:
##########
@@ -0,0 +1,50 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Add allowed_run_types to dag.
+
+Revision ID: e42d9fcd10d9
+Revises: f8c9d7e6b5a4
+Create Date: 2026-02-12 11:49:40.753440
+
+"""
+
+from __future__ import annotations
+
+import sqlalchemy as sa
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision = "e42d9fcd10d9"
+down_revision = "f8c9d7e6b5a4"
+branch_labels = None
+depends_on = None
+airflow_version = "3.2.0"
+
+
+def upgrade():
+ """Add allowed_run_types column to dag table."""
+ with op.batch_alter_table("dag", schema=None) as batch_op:
+ batch_op.add_column(sa.Column("allowed_run_types", sa.JSON(), nullable=True))
Review Comment:
Json is overkill/allows more than we want here - maybe for postgres this would be better as the native ARRAY type?
##########
airflow-core/src/airflow/models/backfill.py:
##########
@@ -502,6 +510,14 @@ def _create_backfill(
if not serdag:
raise DagNotFound(f"Could not find dag {dag_id}")
+ dag_model = session.scalar(select(DagModel).where(DagModel.dag_id == dag_id).limit(1))
Review Comment:
Is there any way we could combine this and the next select on to one?
##########
airflow-core/src/airflow/dag_processing/collection.py:
##########
@@ -600,6 +601,14 @@ def update_dags(
dm.timetable_description = dag.timetable.description
dm.fail_fast = dag.fail_fast if dag.fail_fast is not None else False
+ allowed_types = dag.allowed_run_types
+ if allowed_types:
+ dm.allowed_run_types = sorted(
+ v.value if isinstance(v, enum.Enum) else v for v in allowed_types
+ )
Review Comment:
This looks a bit odd. What stop us doing
```suggestion
dm.allowed_run_types = sorted(allowed_types)
```
##########
airflow-core/src/airflow/jobs/scheduler_job_runner.py:
##########
@@ -2060,6 +2082,16 @@ def _create_dag_runs_asset_triggered(
)
continue
+ if (
+ dag_model.allowed_run_types is not None
+ and DagRunType.ASSET_TRIGGERED.value not in dag_model.allowed_run_types
+ ):
+ self.log.warning(
+ "Dag does not allow asset-triggered runs; skipping",
Review Comment:
This shouldn't be a skip here - i think this should be a parse time error when you try to reference a dag from an asset without allowing the type
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2813082915
##########
airflow-core/src/airflow/migrations/versions/0104_3_2_0_add_allowed_run_types_to_dag.py:
##########
@@ -0,0 +1,50 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Add allowed_run_types to dag.
+
+Revision ID: e42d9fcd10d9
+Revises: f8c9d7e6b5a4
+Create Date: 2026-02-12 11:49:40.753440
+
+"""
+
+from __future__ import annotations
+
+import sqlalchemy as sa
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision = "e42d9fcd10d9"
+down_revision = "f8c9d7e6b5a4"
+branch_labels = None
+depends_on = None
+airflow_version = "3.2.0"
+
+
+def upgrade():
+ """Add allowed_run_types column to dag table."""
+ with op.batch_alter_table("dag", schema=None) as batch_op:
+ batch_op.add_column(sa.Column("allowed_run_types", sa.JSON(), nullable=True))
Review Comment:
The codebase currently uses `sa.JSON()` for all list-type columns, e.g. `assignees` and `chosen_options` in the HITL migration. I found there is no native `ARRAY` usage anywhere in models or migrations. Kept it as JSON here to stay consistent, but I am also happy to open a follow-up issue to explore switching these to native ARRAY where supported if that's preferred.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add deny_dag_run_types option to limit specific dag run types [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 commented on PR #61833:
URL: https://github.com/apache/airflow/pull/61833#issuecomment-3907647289
> > It should be clarified and documented how it behaves when a Dag type, or the entire parameter, is removed (e.g., does it respect the catchup parameter?).
>
> I think the logic is pretty simple. This parameter prevents creation of runs. There are no exceptions thus if you prevent schedule runs all the settings that specify which scheduled runs should be created are ignored.
I'll try to clarify -
Let's say that using this parameter you allow only asset runs (`allowed_run_types = ['assets']`).
At some future point you decide that you want to re-add scheduled runs (`allowed_run_types =['assets', 'scheduled']`).
1. Do we respect the `catchup` parameter in this case to complete past runs? (see followup)
2. If we do, are **all** past scheduled runs are queued, or only those between setting the initial setting of `allowed_run_types` and updating it? You might had set `catchup=False` before adding `allowed_run_types`, and if you now change both `catchup=True` and allow scheduled runs, there will be a lot of potentially unnecessary runs for the users.
We could keep it simple for now and make the required assumptions, but it should be documented IMO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add allowed_run_types to whitelist specific dag run types [airflow]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61833:
URL: https://github.com/apache/airflow/pull/61833#discussion_r2822413381
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/assets.py:
##########
@@ -402,6 +403,13 @@ def materialize_asset(
f"More than one DAG materializes asset with ID: {asset_id}",
)
+ dm = session.scalar(select(DagModel).where(DagModel.dag_id == dag_id).limit(1))
Review Comment:
Just opened. Feel free to drop any options there #62125. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org