[GitHub] [airflow] karakanb commented on issue #24466: Long running tasks being killed with CeleryKubernetesExecutor

[GitBox <gi...@apache.org>]

karakanb commented on issue #24466:
URL: https://github.com/apache/airflow/issues/24466#issuecomment-1192775644

   I am running two schedulers, that is correct. 
   
   If there's an issue with the HA setup of the scheduler then it needs to be fixed in the scheduler I believe, rather than introducing an additional controller.
   
   The weird thing that I cannot explain still is why is this happening only with the tasks that run on Celery, and not with those that are running on Kubernetes. In both cases I am using CeleryKubernetesExecutor, and just the `queue` value is different.
   
   Unfortunately, I had to revert things back to the `kubernetes` cluster instead of running them with Celery, but in the upcoming weeks I will try to enable it back and collect some more logs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org