You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/02/28 23:27:02 UTC
[GitHub] [airflow] jedcunningham opened a new pull request #21880: Minor cleanup on Celery config
jedcunningham opened a new pull request #21880:
URL: https://github.com/apache/airflow/pull/21880
This rewords the description and changes the type of
``worker_enable_remote_control`` and sets a default for
``worker_prefetch_multiplier`` instead of relying on fallback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #21880: Minor cleanup on Celery config
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #21880:
URL: https://github.com/apache/airflow/pull/21880#discussion_r816334373
##########
File path: airflow/config_templates/default_celery.py
##########
@@ -39,16 +39,16 @@ def _broker_supports_visibility_timeout(url):
DEFAULT_CELERY_CONFIG = {
'accept_content': ['json'],
'event_serializer': 'json',
- 'worker_prefetch_multiplier': conf.getint('celery', 'worker_prefetch_multiplier', fallback=1),
+ 'worker_prefetch_multiplier': conf.getint('celery', 'worker_prefetch_multiplier'),
Review comment:
We still need a fallback because existing configs out there may not have this set.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pingzh commented on pull request #21880: Minor cleanup on Celery config
Posted by GitBox <gi...@apache.org>.
pingzh commented on pull request #21880:
URL: https://github.com/apache/airflow/pull/21880#issuecomment-1055897000
@jedcunningham good catch. thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a change in pull request #21880: Minor cleanup on Celery config
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on a change in pull request #21880:
URL: https://github.com/apache/airflow/pull/21880#discussion_r816335464
##########
File path: airflow/config_templates/default_celery.py
##########
@@ -39,16 +39,16 @@ def _broker_supports_visibility_timeout(url):
DEFAULT_CELERY_CONFIG = {
'accept_content': ['json'],
'event_serializer': 'json',
- 'worker_prefetch_multiplier': conf.getint('celery', 'worker_prefetch_multiplier', fallback=1),
+ 'worker_prefetch_multiplier': conf.getint('celery', 'worker_prefetch_multiplier'),
Review comment:
(e.g. https://github.com/apache/airflow/blob/4ad21f5f7c2d416cf813a860564bc2bf3e161d46/airflow/configuration.py#L446)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #21880: Minor cleanup on Celery config
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #21880:
URL: https://github.com/apache/airflow/pull/21880#issuecomment-1058697261
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a change in pull request #21880: Minor cleanup on Celery config
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on a change in pull request #21880:
URL: https://github.com/apache/airflow/pull/21880#discussion_r816335004
##########
File path: airflow/config_templates/default_celery.py
##########
@@ -39,16 +39,16 @@ def _broker_supports_visibility_timeout(url):
DEFAULT_CELERY_CONFIG = {
'accept_content': ['json'],
'event_serializer': 'json',
- 'worker_prefetch_multiplier': conf.getint('celery', 'worker_prefetch_multiplier', fallback=1),
+ 'worker_prefetch_multiplier': conf.getint('celery', 'worker_prefetch_multiplier'),
Review comment:
Airflow falls back to the defaults shipped with it, so even if they don't have it, when this code lands it'll still end up being 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham merged pull request #21880: Minor cleanup on Celery config
Posted by GitBox <gi...@apache.org>.
jedcunningham merged pull request #21880:
URL: https://github.com/apache/airflow/pull/21880
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #21880: Minor cleanup on Celery config
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #21880:
URL: https://github.com/apache/airflow/pull/21880#discussion_r816336369
##########
File path: airflow/config_templates/default_celery.py
##########
@@ -39,16 +39,16 @@ def _broker_supports_visibility_timeout(url):
DEFAULT_CELERY_CONFIG = {
'accept_content': ['json'],
'event_serializer': 'json',
- 'worker_prefetch_multiplier': conf.getint('celery', 'worker_prefetch_multiplier', fallback=1),
+ 'worker_prefetch_multiplier': conf.getint('celery', 'worker_prefetch_multiplier'),
Review comment:
Ah, you’re right. (There are one too many layers of changes for me to follow here 😓)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on pull request #21880: Minor cleanup on Celery config
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on pull request #21880:
URL: https://github.com/apache/airflow/pull/21880#issuecomment-1055817736
cc @pingzh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org