You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/04/21 13:53:46 UTC

[GitHub] [airflow] turbaszek commented on issue #8480: Celery autoscaling overrides normal worker_concurrency setting

turbaszek commented on issue #8480:
URL: https://github.com/apache/airflow/issues/8480#issuecomment-617194692


   Duplicate of https://github.com/apache/airflow/issues/8044
   
   @jsmodic is right, autoscaling is weird in Celery in 4.x and there are issues reporting this problem in Celery itself. If you want to perform production-grade autoscaling and you are using k8s cluster this may be interesting to you: https://www.astronomer.io/blog/the-keda-autoscaler/
   
   I tested it on master and it seems to be working fine (`-c n` spawns main celery process and `n` child processes).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org