[I] Airflow fully supports multi-tenancy [airflow]

["TakawaAkirayo (via GitHub)" <gi...@apache.org>]

TakawaAkirayo opened a new issue, #39417:
URL: https://github.com/apache/airflow/issues/39417

   ### Description
   
   Does Airflow now fully support multi-tenancy now? 
   
   ### Use case/motivation
   
   We have a requirement to share one Airflow cluster among different teams within a single organization.
   
   Currently, we provision a dedicated Airflow cluster for each team to achieve multi-tenancy. However, teams within the same organization often develop independently but collaborate on the entire business data flow. For instance, Team A's pipelines may serve as the upstream for Team B, requiring Team B's pipelines to wait for Team A's pipelines to finish before starting. Achieving this coordination across separate Airflow clusters is challenging without external signals. Additionally, potential version conflicts may arise among Python dependencies used by different teams if they share one Airflow cluster.
   
   Furthermore, teams have varying requirements for computing resources such as CPU, GPU, and memory. Some teams need worker nodes with ample memory, while others delegate heavy tasks to remote systems and require minimal resources.
   
   Our multi-tenancy requirements are as follows:
   
   1. Isolation of:
   
   - Execution environments
   - Workers
   - DAG resources
   - etc...
   
   2. Visibility of all pipelines from different tenants
   3. Granular access control lists (ACLs) at the tenant level
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Airflow fully supports multi-tenancy [airflow]

["Taragolis (via GitHub)" <gi...@apache.org>]

Taragolis closed issue #39417: Airflow fully supports multi-tenancy
URL: https://github.com/apache/airflow/issues/39417


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Airflow fully supports multi-tenancy [airflow]

["boring-cyborg[bot] (via GitHub)" <gi...@apache.org>]

boring-cyborg[bot] commented on issue #39417:
URL: https://github.com/apache/airflow/issues/39417#issuecomment-2094773674

   Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org