You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/06/17 07:16:33 UTC
[GitHub] [airflow] xuejunteo opened a new issue #9342: RBAC access rights filtering by dag tags
xuejunteo opened a new issue #9342:
URL: https://github.com/apache/airflow/issues/9342
**Description**
Is there a way to auto generate permissions based on tags?
e.g.
I want to automatically grant access to user based on his role to a specific list of tags
currently only the can dag read on <dag name> and can dag write on <dag name> is auto populated when there is a new dag
it would be good if i can do something like can dag read on tag <tag name> and can dag write on tag <tag name>
thanks in advance!
**Use case / motivation**
I have multiple teams using the same airflow server and access rights granting has been very tedious as it is currently done by adding individual dags manually to the access rights of each team
**Related Issues**
https://github.com/apache/airflow/pull/6489#issuecomment-643951458
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] malthe commented on issue #9342: RBAC access rights filtering by dag tags
Posted by GitBox <gi...@apache.org>.
malthe commented on issue #9342:
URL: https://github.com/apache/airflow/issues/9342#issuecomment-899491013
I agree that conflating tags (or other metadata that primarily supports the user interface) with security is a bad thing.
If the mechanism was pluggable somehow, then one strategy could be that the first directory level of the DAGs repository was mapped to a group or "container" – and RBAC roles could then in turn be assigned to those groups. It's a simple strategy that would also fit nicely in the situation where you have multiple remote repositories contributing to the DAGs repository as the first directory level.
The interface should of course be generic enough to support other strategies (including one based on tags).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #9342: RBAC access rights filtering by dag tags
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #9342:
URL: https://github.com/apache/airflow/issues/9342#issuecomment-899674097
> This is like a key feature. I can't believe we currently don't have a way to set permissions to a group of DAGs. Tags may not be the best, but I think it could work, like "You can only see DAGs tagged with data-science tag".
My personal opninon - and this is not a universal truth, but my persona view on this is, that this is not really whether we want to use tags or something else. I think main reason is because in essence multi-tenancy feature of Airflow does not have a chance to be working (yet!). By providing "access per-group" UI feature, we would give people a completely false impression, that you can isolate those DAG groups from each other. I personally believe it would be a very bad idea to tell people they can do it, while under the hood they would miss the isolation.
In short - you CAN't currently isolate groups of DAGs in single instance of Airflow. You simply CAN't. it's impossible by design, and we are aiming to iterate and improve the design in the future to make it possible, but it will take a while (and likely will only be fully available in Airflow 3). There are multiple reasons why this kind of RBAC control is a UI-only feature (and can't be easily made into "execution isolation" - while "Role" separation makes sense as UI-only feature (Admin can access different functionality than User via UI).
I personally think introducing groups of DAGs is only really useful (and secure) if we will be able to isolate DAG groups not only via "UI" capabilities but also via "DAG writing" capabilities. I think the reason why you want to introduce this in UI is tha there is something that you want to prevent people doing between the groups (viewing, running, etc.). I think the main reason to give UI access for group is to give it to the same people who can write the dags. For me pretty much all UI "Can see DAGs in this group" access should map 1-1 to "Can write DAGS in this group". It's a bit simplistic view, I can - of course - imagine other reasons for permission separation, but to be honest - giving people the way to "view and operate" DAGs via UI "per-group", and not doing the same for "write code in DAGs" gives a very bad false-sense of security. And I think we should only introduce it when we can do it "full-stack". Otherwise people will be tempted to use current airflow as multi-ten
ant, where it is in fact not multi-tenant (yet!).
Currently the only way to have multi-tenancy is to have separate airflow instances for each tenant. Full stop. By introducing DAG grouping without going full-stack we give people wrong impression that the story is different. It's not.
> Since this has not been added into 2.1, when are we expecting 3.0 to be launched? Can't we expect this to be merged sooner?
I think it will take a while. If you feel the need that you MUST separate the DAGs in the UI, I'd strongly encourage you to implement single instance od Airflow per "group". This is far more robust, already proven, It provides better isolation between groups - actually it's infinitely bigger - comparing to trying to run them in single instance which provides no isolation. Better yet, it provides also capability of isolating the workloads between the different groups (which is completely not possible if you run them in single instance and one group can submit million tasks influencing the other). It can be nicely integrated into any authentication mechanism you have in the way that only users belonging to certain group can have access to each instance. And it can be easily automated to create/drop new "instances" automatically for different groups (been there, done that with nice Terraform templates) - even if you need to manage many of those.
Yeah it uses more resources, but you can also reuse a lot of those resources if you put those instances in auto-scaling K8S clusters. And you can create separate schema for each "group" in the same database to utilise your DB server better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alete89 commented on issue #9342: RBAC access rights filtering by dag tags
Posted by GitBox <gi...@apache.org>.
alete89 commented on issue #9342:
URL: https://github.com/apache/airflow/issues/9342#issuecomment-849677666
This is like a key feature. I can't believe we currently don't have a way to set permissions to a group of DAGs. Tags may not be the best, but I think it could work, like "You can only see DAGs tagged with data-science tag".
Since this has not been added into 2.1, when are we expecting 3.0 to be launched? Can't we expect this to be merged sooner?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #9342: RBAC access rights filtering by dag tags
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #9342:
URL: https://github.com/apache/airflow/issues/9342#issuecomment-645197793
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] sudarshan2906 edited a comment on issue #9342: RBAC access rights filtering by dag tags
Posted by GitBox <gi...@apache.org>.
sudarshan2906 edited a comment on issue #9342:
URL: https://github.com/apache/airflow/issues/9342#issuecomment-668466202
Any update on this? I want to do something similar to this. To give access to a group of dags. Is there any other way to do this. Since this is not taken by anyone?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #9342: RBAC access rights filtering by dag tags
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #9342:
URL: https://github.com/apache/airflow/issues/9342#issuecomment-827879841
Temporarily added it to 2.1 Milestone to revisit this at a later time.
Assigning permissions to a group of DAGs is a good future -- I am not yet sure if "Tags" are the best way for it for the reason James already mentioned. Probably better would be some sort of "wildcard" on permissions -- not sure at the moment.
Also, we should just have a single way to add/edit permissions via UI & API -- not via DAGs themselves -- I know currently we allow it but it is "anti-pattern" in my opinion and is already causing some headaches.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] sudarshan2906 commented on issue #9342: RBAC access rights filtering by dag tags
Posted by GitBox <gi...@apache.org>.
sudarshan2906 commented on issue #9342:
URL: https://github.com/apache/airflow/issues/9342#issuecomment-668466202
Any update on this? I want to do something similar to this. To give access to a group of dags. Is there any other way to do this. Since this is not take by anyone?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #9342: RBAC access rights filtering by dag tags
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #9342:
URL: https://github.com/apache/airflow/issues/9342#issuecomment-988431647
We have now Multi-tenancy effort in progress which I am leading. And while the first two AIPs that are very draft (but will soon be updated) do not address this final granularity yet, they pave the way for the third AIP that is going to address also this use case. Since this is in on the roadmap and planned and part of the bigger multi-tenancy effort. @xuejunteo @sudarshan2906 @alete89 @malthe if you are interested in joining the effort, please join Airlfow Devlist and possibly #sig-multitenancy slack channel and take part in the discussions:
You can find last meeting mintues and even recording of the meeting where we discussed the plans for Multitenancy. https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-1%3A+Improve+Airflow+Security
Note that this is for a long haul - full implementation of the multitenancy (and even discussion on the AIPs) will take quite some time (several months/half a year at the least as this is a really big set of features to make it possible)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org