[GitHub] [airflow] potiuk commented on issue #18113: callback functions not called when a dag run is marked success or failure

[GitBox <gi...@apache.org>]

potiuk commented on issue #18113:
URL: https://github.com/apache/airflow/issues/18113#issuecomment-919217375


   The problem with running the callbacks on UI or API action is that those have different "environment"/context than the regular callbacks.
   
   When the task fails or succeeds during the normal execution, the callbacks are run in workers or Kubernetes PODs or Local Executor processes. When you change the state via UI or API, the callback (if we naively implement it) will be run in webserver. This is not good (and is a security issue if we allow for that). No DAG-provided code should ever be run in the webserver.
   
   Implementing callback "properly" in this case would require to actually schedule another "task" execution in the worker an run it there. Which is expensive and rather complex operation to do. So I see that as a current "feature" rather than "bug".
   
   However things might change possibly with Airflow 2.1.1 and the Triggerer / Defferable Operator approach possibly. I believer (@andrewgodwin ?) it would be much easier to use the mechanisms we have added for that AIP for that purpose.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org