[GitHub] [airflow] RonaldGalea commented on issue #34219: Deploy multiple (celery) workers with the Helm Chart

["RonaldGalea (via GitHub)" <gi...@apache.org>]

RonaldGalea commented on issue #34219:
URL: https://github.com/apache/airflow/issues/34219#issuecomment-1712484775

   > It would be great if we add this feature to the helm chart, but it won't be an easy task, because to implement it we will need to add a loop for each celery template and support two configuration sources: a new value of type list and the existing one for backward compatibility. We also need to update KEDA to do the same and update its queries to scale the different celery clusters based on tasks' queues.
   > 
   > We can also use the existing value as default configurations for the different clusters, and override only the config provided in the list.
   > 
   > cc: @jedcunningham wdyt?
   > 
   > @RonaldGalea Would you like to work on it?
   
   Haven't done templating before but I'd be happy to give it a try. I'll see what I can learn this weekend


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org