[GitHub] [airflow] potiuk commented on issue #19218: Orphaned Tasks are Retried Leading to Multiple Running Aws Batch Jobs

[GitBox <gi...@apache.org>]

potiuk commented on issue #19218:
URL: https://github.com/apache/airflow/issues/19218#issuecomment-963519286


   This is correct behaviour. When you redeploy airflow, it gets restarted and tasks are failed abruptly. So they did not fail, they simply did not complete - and since we do not know if the task completed or not we re-run it. This is expected.
   
   The right way of redeploying airflow is to drain all workers before restarting (you need to send SIGTERM to those workers - that puts them in offline mode - which means that they will not take any more tasks but they shutdown when all the tasks complete. Only then you should redeploy airflow.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org