[GitHub] [airflow] kaxil opened a new pull request #15213: Add support for modifying celery worker deployment strategy

[GitBox <gi...@apache.org>]

kaxil opened a new pull request #15213:
URL: https://github.com/apache/airflow/pull/15213


   This commit modifies the worker template to allow passing a non-default deployment update strategy to worker deployments, in particular celery workers. The values have been set to allow 100% `maxSurge` and 50% `maxUnavailable` allowing new deployments of celery workers to launch a full set of replicas before the old set goes away. Allowing the new workers to pick up work as quickly as possible, rather than the current default which is 1 at a time.
   
   `maxSurge` is the number of pods that will be scheduled beyond the replica count during a rolling deploy. You can specify specific values or percentages. For example if you set the `Marburger to 100% and had 4 replicas, when a rolling deployment started it would launch 4 new pods and then scale down the old ones as the new ones came online.
   
   This is explained here: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#rolling-update-deployment
   
   <!--
   Thank you for contributing! Please make sure that your code changes
   are covered with tests. And in case of new features or big changes
   remember to adjust the documentation.
   
   Feel free to ping committers for the review!
   
   In case of existing issue, reference it using one of the following:
   
   closes: #ISSUE
   related: #ISSUE
   
   How to write a good git commit message:
   http://chris.beams.io/posts/git-commit/
   -->
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
   In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil merged pull request #15213: Add support for modifying celery worker deployment strategy

[GitBox <gi...@apache.org>]

kaxil merged pull request #15213:
URL: https://github.com/apache/airflow/pull/15213


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #15213: Add support for modifying celery worker deployment strategy

[GitBox <gi...@apache.org>]

github-actions[bot] commented on pull request #15213:
URL: https://github.com/apache/airflow/pull/15213#issuecomment-813655134


   The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest master or amend the last commit of the PR, and push it with --force-with-lease.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org