[GitHub] [airflow] Oduig commented on issue #29461: Timeout in DatabricksSubmitRunOperator causes callbacks to be invoked twice

["Oduig (via GitHub)" <gi...@apache.org>]

Oduig commented on issue #29461:
URL: https://github.com/apache/airflow/issues/29461#issuecomment-1427514402

   Imo it is fair that QoS at-least-once applies here, and that sometimes a scenario can occur where two processes call a callback simultaneously, or a notification delivery succeeded but appeared to the sender as a failure, and it would be delivered again.
   
   However, a callback that is always called twice does not fit at-least-once semantics. In addition, with a BashOperator timeout, the callback is only executed once, so it seems like a solvable problem. Finally, when there is a timeout, the running task is marked skipped rather than failed - only the DAG itself fails, in my eyes this means the task level shouldn't even attempt to call the callback because it itself didn't fail


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org