You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "joaopamaral (via GitHub)" <gi...@apache.org> on 2024/07/10 21:08:47 UTC
[PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
joaopamaral opened a new pull request, #40703:
URL: https://github.com/apache/airflow/pull/40703
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
Work in progress
https://github.com/apache/airflow/issues/37329
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 merged PR #40703:
URL: https://github.com/apache/airflow/pull/40703
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 commented on PR #40703:
URL: https://github.com/apache/airflow/pull/40703#issuecomment-2225866119
> Hey @shahar1, can you help me revert the changes in `apache/iceberg/__init__.py` file? I've tried to revert the permissions [here](https://github.com/apache/airflow/pull/40703/commits/b2be841fad1e86e9c4578785b6a73057282c9247) but it still showing a file permission difference. And git commands are not reverting the change.
There you go: https://github.com/joaopamaral/airflow/pull/1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "joaopamaral (via GitHub)" <gi...@apache.org>.
joaopamaral commented on code in PR #40703:
URL: https://github.com/apache/airflow/pull/40703#discussion_r1675773625
##########
airflow/security/permissions.py:
##########
@@ -64,19 +67,38 @@
DEPRECATED_ACTION_CAN_DAG_READ = "can_dag_read"
DEPRECATED_ACTION_CAN_DAG_EDIT = "can_dag_edit"
-DAG_ACTIONS = {ACTION_CAN_READ, ACTION_CAN_EDIT, ACTION_CAN_DELETE}
+
+class ResourceDetails(TypedDict):
+ """Details of a resource (actions and prefix)."""
+
+ actions: set[str]
+ prefix: str
+
+
+RESOURCE_DETAILS_MAP = {
+ RESOURCE_DAG: ResourceDetails(
+ actions={ACTION_CAN_READ, ACTION_CAN_EDIT, ACTION_CAN_DELETE}, prefix=RESOURCE_DAG_PREFIX
+ ),
+ RESOURCE_DAG_RUN: ResourceDetails(
+ actions={ACTION_CAN_READ, ACTION_CAN_CREATE, ACTION_CAN_DELETE, ACTION_CAN_ACCESS_MENU},
+ prefix=RESOURCE_DAG_RUN_PREFIX,
+ ),
+}
+PREFIX_RESOURCES_MAP = {
+ prefix: resource for resource, actions in RESOURCE_DETAILS_MAP.items() for prefix in actions["prefix"]
+}
-def resource_name_for_dag(root_dag_id: str) -> str:
Review Comment:
Do you think it's enough?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "joaopamaral (via GitHub)" <gi...@apache.org>.
joaopamaral commented on PR #40703:
URL: https://github.com/apache/airflow/pull/40703#issuecomment-2225671773
Hey @shahar1, can you help me revert the changes in `apache/iceberg/__init__.py` file? I've tried to revert the permissions [here](https://github.com/apache/airflow/pull/40703/commits/b2be841fad1e86e9c4578785b6a73057282c9247) but it still showing a file permission difference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #40703:
URL: https://github.com/apache/airflow/pull/40703#issuecomment-2312411448
> It looks like for Airflow 2.9.1 this version of fab provider is not backward compatible
@ekosteev - do you see some evidence of this ? The issue here: https://github.com/apache/airflow/issues/41540#issuecomment-2293802672 - made the earlier FAB providers not compatible for 2.10.0 -> which shoudl be fixed in https://github.com/apache/airflow/issues/41540, but I have not seen/ compatibility issues with earlier airflow versions (It passed all the compatibility tests at the very least)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "joaopamaral (via GitHub)" <gi...@apache.org>.
joaopamaral commented on PR #40703:
URL: https://github.com/apache/airflow/pull/40703#issuecomment-2231425344
There are some tests failing because there was an increase in the number of queries. I'll check it out this week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "joaopamaral (via GitHub)" <gi...@apache.org>.
joaopamaral commented on PR #40703:
URL: https://github.com/apache/airflow/pull/40703#issuecomment-2251398535
@shahar1, fyi I've included this change to skip the Dag Run permissions (from dag level) in admin role https://github.com/apache/airflow/pull/40703/commits/f182af9484c1e88cfb662c027da57a759bbea73e . I think it will fix the db test issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #40703:
URL: https://github.com/apache/airflow/pull/40703#issuecomment-2293813085
I think that one caused problem https://github.com/apache/airflow/issues/41540#issuecomment-2293802672 - it does not seem backwards-compatible with the previous fab provider.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "joaopamaral (via GitHub)" <gi...@apache.org>.
joaopamaral commented on code in PR #40703:
URL: https://github.com/apache/airflow/pull/40703#discussion_r1675797557
##########
airflow/providers/fab/auth_manager/security_manager/override.py:
##########
@@ -1102,81 +1103,103 @@ def sync_perm_for_dag(
:param dag_id: the ID of the DAG whose permissions should be updated
:param access_control: a dict where each key is a role name and
- each value is a set() of action names (e.g.,
+ each value is a set() of action names in case of DAGs resource (e.g.,
{'can_read'}
+ or the value can be a dict where each key is a resource name and
+ each value is a set() of action names (e.g., {'DAG Runs': {'can_read'}})
Review Comment:
Tried to make it more clear in https://github.com/apache/airflow/pull/40703/commits/c7ebe69d6fe41e37262ace566d70d489a895fc75 . Please let me know if it's ok now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "kosteev (via GitHub)" <gi...@apache.org>.
kosteev commented on PR #40703:
URL: https://github.com/apache/airflow/pull/40703#issuecomment-2309684608
It looks like this change implies dependencies between Airflow core and fab provider and new version of fab provider requires changes also in Airflow core, thus fab provider should have constraint apache-airflow>=2.9.X?
It looks like for Airflow 2.9.1 this version of fab provider is not backward compatible
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "joaopamaral (via GitHub)" <gi...@apache.org>.
joaopamaral commented on code in PR #40703:
URL: https://github.com/apache/airflow/pull/40703#discussion_r1675765214
##########
airflow/security/permissions.py:
##########
@@ -64,19 +67,38 @@
DEPRECATED_ACTION_CAN_DAG_READ = "can_dag_read"
DEPRECATED_ACTION_CAN_DAG_EDIT = "can_dag_edit"
-DAG_ACTIONS = {ACTION_CAN_READ, ACTION_CAN_EDIT, ACTION_CAN_DELETE}
+
+class ResourceDetails(TypedDict):
+ """Details of a resource (actions and prefix)."""
+
+ actions: set[str]
+ prefix: str
+
+
+RESOURCE_DETAILS_MAP = {
+ RESOURCE_DAG: ResourceDetails(
+ actions={ACTION_CAN_READ, ACTION_CAN_EDIT, ACTION_CAN_DELETE}, prefix=RESOURCE_DAG_PREFIX
+ ),
+ RESOURCE_DAG_RUN: ResourceDetails(
+ actions={ACTION_CAN_READ, ACTION_CAN_CREATE, ACTION_CAN_DELETE, ACTION_CAN_ACCESS_MENU},
+ prefix=RESOURCE_DAG_RUN_PREFIX,
+ ),
+}
+PREFIX_RESOURCES_MAP = {
+ prefix: resource for resource, actions in RESOURCE_DETAILS_MAP.items() for prefix in actions["prefix"]
+}
-def resource_name_for_dag(root_dag_id: str) -> str:
Review Comment:
Ok, I've readded the `DAG_ACTIONS` to keep the compatibility.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "joaopamaral (via GitHub)" <gi...@apache.org>.
joaopamaral commented on code in PR #40703:
URL: https://github.com/apache/airflow/pull/40703#discussion_r1675984631
##########
airflow/security/permissions.py:
##########
@@ -64,19 +67,38 @@
DEPRECATED_ACTION_CAN_DAG_READ = "can_dag_read"
DEPRECATED_ACTION_CAN_DAG_EDIT = "can_dag_edit"
-DAG_ACTIONS = {ACTION_CAN_READ, ACTION_CAN_EDIT, ACTION_CAN_DELETE}
+
+class ResourceDetails(TypedDict):
+ """Details of a resource (actions and prefix)."""
+
+ actions: set[str]
+ prefix: str
+
+
+RESOURCE_DETAILS_MAP = {
+ RESOURCE_DAG: ResourceDetails(
+ actions={ACTION_CAN_READ, ACTION_CAN_EDIT, ACTION_CAN_DELETE}, prefix=RESOURCE_DAG_PREFIX
+ ),
+ RESOURCE_DAG_RUN: ResourceDetails(
+ actions={ACTION_CAN_READ, ACTION_CAN_CREATE, ACTION_CAN_DELETE, ACTION_CAN_ACCESS_MENU},
+ prefix=RESOURCE_DAG_RUN_PREFIX,
+ ),
+}
+PREFIX_RESOURCES_MAP = {
+ prefix: resource for resource, actions in RESOURCE_DETAILS_MAP.items() for prefix in actions["prefix"]
+}
-def resource_name_for_dag(root_dag_id: str) -> str:
Review Comment:
Thanks for catching this case @shahar1 ! Indeed it was not compatible with running only the new FAB.
I've added these [changes](https://github.com/apache/airflow/pull/40703/commits/1e80c84479a7180433ef88b877b364b351718ff4) to keep the compatibility.
<img width="1695" alt="image" src="https://github.com/user-attachments/assets/a7d8329e-f4e6-4f3e-b16e-6354f948b5ad">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "joaopamaral (via GitHub)" <gi...@apache.org>.
joaopamaral closed pull request #40703: Feature: Allow set Dag Run resource into Dag Level permission
URL: https://github.com/apache/airflow/pull/40703
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "joaopamaral (via GitHub)" <gi...@apache.org>.
joaopamaral commented on code in PR #40703:
URL: https://github.com/apache/airflow/pull/40703#discussion_r1705704846
##########
tests/dag_processing/test_processor.py:
##########
@@ -1035,7 +1035,7 @@ def test_counter_for_last_num_of_db_queries(self):
with create_session() as session:
with assert_queries_count(
- expected_count=94,
+ expected_count=154,
Review Comment:
@shahar1 as I asked in this [comment](https://github.com/apache/airflow/pull/40703#issuecomment-2270038205), this change is required because we have now more permissions in dag level to sync:
https://github.com/apache/airflow/blob/27ba034302c1109f721b667326fb14963a768f1d/airflow/providers/fab/auth_manager/security_manager/override.py#L1120-L1123
##########
tests/dag_processing/test_processor.py:
##########
@@ -1035,7 +1035,7 @@ def test_counter_for_last_num_of_db_queries(self):
with create_session() as session:
with assert_queries_count(
- expected_count=94,
+ expected_count=154,
Review Comment:
@shahar1 as I said in this [comment](https://github.com/apache/airflow/pull/40703#issuecomment-2270038205), this change is required because we have now more permissions in dag level to sync:
https://github.com/apache/airflow/blob/27ba034302c1109f721b667326fb14963a768f1d/airflow/providers/fab/auth_manager/security_manager/override.py#L1120-L1123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 commented on code in PR #40703:
URL: https://github.com/apache/airflow/pull/40703#discussion_r1709972191
##########
tests/dag_processing/test_processor.py:
##########
@@ -1035,7 +1035,7 @@ def test_counter_for_last_num_of_db_queries(self):
with create_session() as session:
with assert_queries_count(
- expected_count=94,
+ expected_count=154,
Review Comment:
It makes sense to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "joaopamaral (via GitHub)" <gi...@apache.org>.
joaopamaral commented on code in PR #40703:
URL: https://github.com/apache/airflow/pull/40703#discussion_r1675773024
##########
airflow/providers/fab/auth_manager/security_manager/override.py:
##########
@@ -1093,7 +1094,7 @@ def is_dag_resource(self, resource_name: str) -> bool:
def sync_perm_for_dag(
self,
dag_id: str,
- access_control: dict[str, Collection[str]] | None = None,
Review Comment:
Thanks! Fixed in https://github.com/apache/airflow/pull/40703/commits/2e027f65df370234ad185230894db6a233196182
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 commented on code in PR #40703:
URL: https://github.com/apache/airflow/pull/40703#discussion_r1675711163
##########
airflow/providers/fab/auth_manager/security_manager/override.py:
##########
@@ -1102,81 +1103,103 @@ def sync_perm_for_dag(
:param dag_id: the ID of the DAG whose permissions should be updated
:param access_control: a dict where each key is a role name and
- each value is a set() of action names (e.g.,
+ each value is a set() of action names in case of DAGs resource (e.g.,
{'can_read'}
+ or the value can be a dict where each key is a resource name and
+ each value is a set() of action names (e.g., {'DAG Runs': {'can_read'}})
Review Comment:
Try to clarify the description, it's quite hard to read (same goes for similar docstrings)
##########
airflow/providers/fab/auth_manager/security_manager/override.py:
##########
@@ -1093,7 +1094,7 @@ def is_dag_resource(self, resource_name: str) -> bool:
def sync_perm_for_dag(
self,
dag_id: str,
- access_control: dict[str, Collection[str]] | None = None,
Review Comment:
Instead of removing the type hint - make it up to date (same goes for the other mentions of `access_control` in methods' signatures)
##########
airflow/models/dag.py:
##########
@@ -906,21 +908,33 @@ def _upgrade_outdated_dag_access_control(access_control=None):
"""
if access_control is None:
return None
- new_perm_mapping = {
+ new_dag_perm_mapping = {
permissions.DEPRECATED_ACTION_CAN_DAG_READ: permissions.ACTION_CAN_READ,
permissions.DEPRECATED_ACTION_CAN_DAG_EDIT: permissions.ACTION_CAN_EDIT,
}
+
+ def update_old_perm(permission: str):
+ new_perm = new_dag_perm_mapping.get(permission, permission)
+ if new_perm != permission:
+ warnings.warn(
+ f"The '{permission}' permission is deprecated. Please use '{new_perm}'.",
+ RemovedInAirflow3Warning,
+ stacklevel=3,
+ )
Review Comment:
Could you please add a test for this warning?
##########
airflow/security/permissions.py:
##########
@@ -64,19 +67,38 @@
DEPRECATED_ACTION_CAN_DAG_READ = "can_dag_read"
DEPRECATED_ACTION_CAN_DAG_EDIT = "can_dag_edit"
-DAG_ACTIONS = {ACTION_CAN_READ, ACTION_CAN_EDIT, ACTION_CAN_DELETE}
+
+class ResourceDetails(TypedDict):
+ """Details of a resource (actions and prefix)."""
+
+ actions: set[str]
+ prefix: str
+
+
+RESOURCE_DETAILS_MAP = {
+ RESOURCE_DAG: ResourceDetails(
+ actions={ACTION_CAN_READ, ACTION_CAN_EDIT, ACTION_CAN_DELETE}, prefix=RESOURCE_DAG_PREFIX
+ ),
+ RESOURCE_DAG_RUN: ResourceDetails(
+ actions={ACTION_CAN_READ, ACTION_CAN_CREATE, ACTION_CAN_DELETE, ACTION_CAN_ACCESS_MENU},
+ prefix=RESOURCE_DAG_RUN_PREFIX,
+ ),
+}
+PREFIX_RESOURCES_MAP = {
+ prefix: resource for resource, actions in RESOURCE_DETAILS_MAP.items() for prefix in actions["prefix"]
+}
-def resource_name_for_dag(root_dag_id: str) -> str:
Review Comment:
I'm quite concerned of backward compatbility issues - FAB provider in an outdated version will fail importing `DAG_ACTIONS` and `resource_name_for_dag` as that they don't exist anymore.
##########
airflow/providers/fab/auth_manager/fab_auth_manager.py:
##########
@@ -412,7 +413,33 @@ def _is_authorized_dag(
if details and details.id:
# Check whether the user has permissions to access a specific DAG
- resource_dag_name = self._resource_name_for_dag(details.id)
+ resource_dag_name = self._resource_name(details.id, RESOURCE_DAG)
+ return self._is_authorized(method=method, resource_type=resource_dag_name, user=user)
+
+ return False
+
+ def _is_authorized_dag_run(
+ self,
+ method: ResourceMethod,
+ details: DagDetails | None = None,
+ user: BaseUser | None = None,
+ ) -> bool:
+ """
+ Return whether the user is authorized to perform a given action on a DAG Run.
+
+ :param method: the method to perform
+ :param details: optional details about the DAG
+ :param user: the user to perform the action on. If not provided (or None), it uses the current user
Review Comment:
```suggestion
:param details: optional, details about the DAG
:param user: optional, the user to perform the action on. If not provided, it uses the current user
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 commented on code in PR #40703:
URL: https://github.com/apache/airflow/pull/40703#discussion_r1675797799
##########
airflow/security/permissions.py:
##########
@@ -64,19 +67,38 @@
DEPRECATED_ACTION_CAN_DAG_READ = "can_dag_read"
DEPRECATED_ACTION_CAN_DAG_EDIT = "can_dag_edit"
-DAG_ACTIONS = {ACTION_CAN_READ, ACTION_CAN_EDIT, ACTION_CAN_DELETE}
+
+class ResourceDetails(TypedDict):
+ """Details of a resource (actions and prefix)."""
+
+ actions: set[str]
+ prefix: str
+
+
+RESOURCE_DETAILS_MAP = {
+ RESOURCE_DAG: ResourceDetails(
+ actions={ACTION_CAN_READ, ACTION_CAN_EDIT, ACTION_CAN_DELETE}, prefix=RESOURCE_DAG_PREFIX
+ ),
+ RESOURCE_DAG_RUN: ResourceDetails(
+ actions={ACTION_CAN_READ, ACTION_CAN_CREATE, ACTION_CAN_DELETE, ACTION_CAN_ACCESS_MENU},
+ prefix=RESOURCE_DAG_RUN_PREFIX,
+ ),
+}
+PREFIX_RESOURCES_MAP = {
+ prefix: resource for resource, actions in RESOURCE_DETAILS_MAP.items() for prefix in actions["prefix"]
+}
-def resource_name_for_dag(root_dag_id: str) -> str:
Review Comment:
Looks better, the `resource` param should be optional though.
Try to test it locally by using the current Airflow version + the changes that you made only to the provider, and see if anything breaks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "joaopamaral (via GitHub)" <gi...@apache.org>.
joaopamaral commented on code in PR #40703:
URL: https://github.com/apache/airflow/pull/40703#discussion_r1675790023
##########
airflow/models/dag.py:
##########
@@ -906,21 +908,33 @@ def _upgrade_outdated_dag_access_control(access_control=None):
"""
if access_control is None:
return None
- new_perm_mapping = {
+ new_dag_perm_mapping = {
permissions.DEPRECATED_ACTION_CAN_DAG_READ: permissions.ACTION_CAN_READ,
permissions.DEPRECATED_ACTION_CAN_DAG_EDIT: permissions.ACTION_CAN_EDIT,
}
+
+ def update_old_perm(permission: str):
+ new_perm = new_dag_perm_mapping.get(permission, permission)
+ if new_perm != permission:
+ warnings.warn(
+ f"The '{permission}' permission is deprecated. Please use '{new_perm}'.",
+ RemovedInAirflow3Warning,
+ stacklevel=3,
+ )
Review Comment:
There is already a test, but I've added some assert to get the warning count and check if is a `permission is deprecated` warning (now there will be one for each permission).
https://github.com/apache/airflow/pull/40703/commits/76867255fbeac65d629ba32d05b1186cc925c71d
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "joaopamaral (via GitHub)" <gi...@apache.org>.
joaopamaral commented on code in PR #40703:
URL: https://github.com/apache/airflow/pull/40703#discussion_r1675984631
##########
airflow/security/permissions.py:
##########
@@ -64,19 +67,38 @@
DEPRECATED_ACTION_CAN_DAG_READ = "can_dag_read"
DEPRECATED_ACTION_CAN_DAG_EDIT = "can_dag_edit"
-DAG_ACTIONS = {ACTION_CAN_READ, ACTION_CAN_EDIT, ACTION_CAN_DELETE}
+
+class ResourceDetails(TypedDict):
+ """Details of a resource (actions and prefix)."""
+
+ actions: set[str]
+ prefix: str
+
+
+RESOURCE_DETAILS_MAP = {
+ RESOURCE_DAG: ResourceDetails(
+ actions={ACTION_CAN_READ, ACTION_CAN_EDIT, ACTION_CAN_DELETE}, prefix=RESOURCE_DAG_PREFIX
+ ),
+ RESOURCE_DAG_RUN: ResourceDetails(
+ actions={ACTION_CAN_READ, ACTION_CAN_CREATE, ACTION_CAN_DELETE, ACTION_CAN_ACCESS_MENU},
+ prefix=RESOURCE_DAG_RUN_PREFIX,
+ ),
+}
+PREFIX_RESOURCES_MAP = {
+ prefix: resource for resource, actions in RESOURCE_DETAILS_MAP.items() for prefix in actions["prefix"]
+}
-def resource_name_for_dag(root_dag_id: str) -> str:
Review Comment:
Thanks for catching this case @shahar1 ! Indeed it was not compatible with running only the new FAB.
I've added these [changes](https://github.com/apache/airflow/pull/40703/commits/1e80c84479a7180433ef88b877b364b351718ff4) to keep the compatibility. And `access_control` works except with DAG Run resource:
<img width="1695" alt="image" src="https://github.com/user-attachments/assets/a7d8329e-f4e6-4f3e-b16e-6354f948b5ad">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 commented on PR #40703:
URL: https://github.com/apache/airflow/pull/40703#issuecomment-2226086935
Static checks still fail - plesae take look at the logs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 commented on PR #40703:
URL: https://github.com/apache/airflow/pull/40703#issuecomment-2266134478
> @shahar1, fyi I've included this change to skip the Dag Run permissions (from dag level) in admin role [f182af9](https://github.com/apache/airflow/commit/f182af9484c1e88cfb662c027da57a759bbea73e) . I think it will fix the db test issue.
Apologies for the late response:
1. Could you please merge again from `main` so we could ensure all checks pass?
1. Only now I realized that there's no documentation nor newsfragment includede in this PR. I think that both are essential for other people being aware of this feature, and using it properly. Could you please update the docs. `docs/apache-airflow-providers-fab/auth-manager/access-control.rst` and create `newsframent/40703.feature.rst`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]
Posted by "joaopamaral (via GitHub)" <gi...@apache.org>.
joaopamaral commented on PR #40703:
URL: https://github.com/apache/airflow/pull/40703#issuecomment-2270038205
Hey @shahar1, I've included the docs and I've dug a little more about the query count test and basic the difference is during the `_sync_perm_for_dag`:
```
Recorded query locations:
dagbag.py:_sync_perm_for_dag:748 > override.py:sync_perm_for_dag:1111 > override.py:create_permission:1848 > override.py:get_permission:1819 > override.py:get_action:1688: 15
dagbag.py:_sync_perm_for_dag:748 > override.py:sync_perm_for_dag:1111 > override.py:create_permission:1848 > override.py:get_permission:1820 > override.py:get_resource:1749: 15
dagbag.py:_serialize_dag_capturing_errors:692 > dagbag.py:_sync_perm_for_dag:748 > override.py:sync_perm_for_dag:1111 > override.py:create_permission:1848 > override.py:get_permission:1823: 10
E AssertionError: The expected number of db queries is 94 with extra margin: 10. The current number is 154.
E Recorded query locations:
E dagbag.py:_sync_perm_for_dag:748 > override.py:sync_perm_for_dag:1123 > override.py:create_permission:1890 > override.py:get_permission:1861 > override.py:get_action:1730: 35
E dagbag.py:_sync_perm_for_dag:748 > override.py:sync_perm_for_dag:1123 > override.py:create_permission:1890 > override.py:get_permission:1862 > override.py:get_resource:1791: 35
E dagbag.py:_serialize_dag_capturing_errors:692 > dagbag.py:_sync_perm_for_dag:748 > override.py:sync_perm_for_dag:1123 > override.py:create_permission:1890 > override.py:get_permission:1865: 30
```
As we included more permissions at the DAG level, we expect an increase in this number:
https://github.com/apache/airflow/blob/27ba034302c1109f721b667326fb14963a768f1d/airflow/providers/fab/auth_manager/security_manager/override.py#L1120-L1123
Do you think we can increase the query count for this test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org