Airflow Crashing after Azure AgentPools are re-sized

[Kevin Lam <ke...@fathomhealth.co>]

Hi,

We are running airflow on kubernetes, hosted on Azure, using
https://github.com/mumoshu/kube-airflow. We are also running an autoscaler:
https://github.com/wbuchwalter/Kubernetes-acs-engine-autoscaler.

Whenever the autoscaler increase the agentPool sizes to allocate a new node
in kubernetes, airflow will crash.

It looks like the worker pod crashes causing the webserver to also quit.

I don't have any logs handy at the moment, but I can try and get some.

Is anyone running a similar set-up or has anyone encountered something like
this? Appreciate any ideas.

Thanks for your help!