You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/08/17 10:27:42 UTC

[GitHub] [airflow] turbaszek opened a new issue #10358: Add docs / best practises for KEDA + Airflow

turbaszek opened a new issue #10358:
URL: https://github.com/apache/airflow/issues/10358


   **Description**
   
   Airflow is using [KEDA](https://keda.sh) to achieve autoscaling behavior for `CeleryExecutor` in case of Kubernetes deployment. It is also incorporated in our Helm chart.
   
   We should add documentation in the form of best practices / watchouts, it should include:
   - adjusting `terminationGracePeriodSeconds` of worker pods to avoid termination of long-running tasks 
   - suggestions for metric setup (min, max, target)
   
   **Use case / motivation**
   
   Help people understand what possible problems may occur with autoscaling. 
   
   **Related Issues**
   
   N/!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] abdidarmawan007 commented on issue #10358: Add docs / best practises for KEDA + Airflow

Posted by GitBox <gi...@apache.org>.

abdidarmawan007 commented on issue #10358:
URL: https://github.com/apache/airflow/issues/10358#issuecomment-1085511770


   Autoscaling Airflow with KEDA, very minimal documentation  


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org