Re: [I] Leak in Kubernetes Executor Running Tasks Slot Count [airflow]

["c-thiel (via GitHub)" <gi...@apache.org>]

c-thiel commented on issue #35675:
URL: https://github.com/apache/airflow/issues/35675#issuecomment-1842586190

   We had the same problem and only noticed it from Airflow 2.7.3, but it was there before.
   Maybe this helps for debugging:
   
   On the left of the red line we had Airflow 2.6.X
   On the right side is the update to 2.7.3.
   
   Each line is one scheduler.
   
   <img width="1522" alt="image" src="https://github.com/apache/airflow/assets/12560027/c6cd22e4-cbfd-445f-b0dd-5f19bbae0982">
   
   Before the update, scheduler open slots continuously went down to even below zero sometimes, but only for some executors.
   After the update this is much more distributed between all the executors.
   
   **Our Workaround**
   What helped us is limit the parallelism via pools, and just set `AIRFLOW__CORE__PARALLELISM` to an arbitrary high number. Also regular deployment restarts helps.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org