You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "adamgorkaextbi (via GitHub)" <gi...@apache.org> on 2025/01/17 09:27:35 UTC

[I] DatabricksWorkflowOperator do not update ACL on workflow reset [airflow]

adamgorkaextbi opened a new issue, #45738:
URL: https://github.com/apache/airflow/issues/45738

   ### Apache Airflow Provider(s)
   
   databricks
   
   ### Versions of Apache Airflow Providers
   
   apache-airflow-providers-databricks==7.0.0
   
   ### Apache Airflow version
   
   2.10.4
   
   ### Operating System
   
   Linux
   
   ### Deployment
   
   Official Apache Airflow Helm Chart
   
   ### Deployment details
   
   official Helm Chart
   
   ### What happened
   
   DatabricksWorkflowOperator do not update ACL on workflow reset, this is done only during creation of workflow (Databricks API 2.0 implementation)
   one more call is needed to Databricks API 2.0
   
   ### What you think should happen instead
   
   DatabricksWorkflowOperator on  workflow reset should also update ACL in next api call
   
   ### How to reproduce
   
   try to modify ACL after creation of workflow
   
   ### Anything else
   
   Proposed solution:
   after this line:
   https://github.com/apache/airflow/blob/6cde25f1ac005e217b32f19a8bf1be75bf8e7af6/providers/src/airflow/providers/databricks/operators/databricks_workflow.py#L179
   add this code:
   ```
               if "access_control_list" in job_spec.keys():
                   access_control_list = {"access_control_list": job_spec["access_control_list"]}
                   self.log.info(
                       "Updating ACL of Databricks workflow job %s with spec %s",
                       self.job_name,
                       json.dumps(access_control_list, indent=2),
                   )
                   self._hook.update_job_permission(job_id, access_control_list)
   ```
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] DatabricksWorkflowOperator do not update ACL on workflow reset [airflow]

Posted by "hardeybisey (via GitHub)" <gi...@apache.org>.
hardeybisey commented on issue #45738:
URL: https://github.com/apache/airflow/issues/45738#issuecomment-2721163863

   > @hardeybisey , would you like to work on this? 
   
   Sure, I will be happy to pick this up. You can assign it to me. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] DatabricksWorkflowOperator do not update ACL on workflow reset [airflow]

Posted by "adamgorkaextbi (via GitHub)" <gi...@apache.org>.
adamgorkaextbi commented on issue #45738:
URL: https://github.com/apache/airflow/issues/45738#issuecomment-2739937333

   > [@adamgorkaextbi](https://github.com/adamgorkaextbi) , you haven't explained why it needs to update ACL on workflow reset. If you don't update ACL, does it fail? What's the consequence?
   
   When you update permissions in the input JSON, the operator does not apply the changes automatically. You need to update the ACL manually through the UI or by using CLI commands. This lead to bugs and unwanted behavior and manual work


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] DatabricksWorkflowOperator do not update ACL on workflow reset [airflow]

Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on issue #45738:
URL: https://github.com/apache/airflow/issues/45738#issuecomment-2708656641

   @adamgorkaextbi , you haven't explained why it needs to update ACL on workflow reset. If you don't update ACL, does it fail? What's the consequence?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] DatabricksWorkflowOperator do not update ACL on workflow reset [airflow]

Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on issue #45738:
URL: https://github.com/apache/airflow/issues/45738#issuecomment-2721150531

   @hardeybisey , would you like to work on this? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] DatabricksWorkflowOperator do not update ACL on workflow reset [airflow]

Posted by "pankajkoti (via GitHub)" <gi...@apache.org>.
pankajkoti closed issue #45738: DatabricksWorkflowOperator do not update ACL on workflow reset
URL: https://github.com/apache/airflow/issues/45738


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] DatabricksWorkflowOperator do not update ACL on workflow reset [airflow]

Posted by "arollet22 (via GitHub)" <gi...@apache.org>.
arollet22 commented on issue #45738:
URL: https://github.com/apache/airflow/issues/45738#issuecomment-2721104245

   Maybe I can jump in to answer that question. We are also experiencing the same issue. What happens generally is the Airflow dev forgets to set the correct ACL during the first run, faces a permission issue on the Databricks UI and doesn't understand why its updated ACL configuration doesn't fix the problem. (Usually Airflow workflows/jobs are created using Service Principals which do not grant any rights by default to the group / users).
   One other use case I can imagine, although not faced yet, is if you were to migrate a workflow ownership to another group.
   
   The consequence is not that the job doesn't run, that works well, but if one task fails then the dev isn't able to check the logs to correct the error.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org