You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/06 19:23:39 UTC
[GitHub] [airflow] drago-f5a opened a new pull request #13518: Correct the logic for webserver choosing number of workers to spawn (…
drago-f5a opened a new pull request #13518:
URL: https://github.com/apache/airflow/pull/13518
…#13469)
A key consequence of this fix is that webserver will properly
exit when gunicorn master dies and stops responding to signals.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13518: Fix webserver exiting when gunicorn master crashes
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13518:
URL: https://github.com/apache/airflow/pull/13518#issuecomment-759622371
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest master or amend the last commit of the PR, and push it with --force-with-lease.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #13518: Fix webserver exiting when gunicorn master crashes
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #13518:
URL: https://github.com/apache/airflow/pull/13518#discussion_r560547575
##########
File path: airflow/cli/commands/webserver_command.py
##########
@@ -258,15 +258,15 @@ def _check_workers(self) -> None:
num_workers_running = self._get_num_workers_running()
if num_workers_running < self.num_workers_expected:
new_worker_count = min(
- num_workers_running - self.worker_refresh_batch_size, self.worker_refresh_batch_size
+ self.num_workers_expected - num_workers_running, self.worker_refresh_batch_size
)
- self.log.debug(
+ self.log.info(
Review comment:
That makes sense, can you create a new PR please to change the log level back to info. I revert that changed before merging the PR
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] drago-f5a commented on a change in pull request #13518: Fix webserver exiting when gunicorn master crashes
Posted by GitBox <gi...@apache.org>.
drago-f5a commented on a change in pull request #13518:
URL: https://github.com/apache/airflow/pull/13518#discussion_r560544581
##########
File path: airflow/cli/commands/webserver_command.py
##########
@@ -258,15 +258,15 @@ def _check_workers(self) -> None:
num_workers_running = self._get_num_workers_running()
if num_workers_running < self.num_workers_expected:
new_worker_count = min(
- num_workers_running - self.worker_refresh_batch_size, self.worker_refresh_batch_size
+ self.num_workers_expected - num_workers_running, self.worker_refresh_batch_size
)
- self.log.debug(
+ self.log.info(
Review comment:
@kaxil
This action (spawning new workers) is taken specifically to correct an issue that was detected and logged at the error level few lines of code above. Given the prior error level log, it seemed reasonable to log an attempt at corrective action at the info level.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #13518: Fix webserver exiting when gunicorn master crashes
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #13518:
URL: https://github.com/apache/airflow/pull/13518#discussion_r560535672
##########
File path: airflow/cli/commands/webserver_command.py
##########
@@ -258,15 +258,15 @@ def _check_workers(self) -> None:
num_workers_running = self._get_num_workers_running()
if num_workers_running < self.num_workers_expected:
new_worker_count = min(
- num_workers_running - self.worker_refresh_batch_size, self.worker_refresh_batch_size
+ self.num_workers_expected - num_workers_running, self.worker_refresh_batch_size
)
- self.log.debug(
+ self.log.info(
Review comment:
Any reason for changing the log level? @drago-f5a
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] drago-f5a commented on a change in pull request #13518: Fix webserver exiting when gunicorn master crashes
Posted by GitBox <gi...@apache.org>.
drago-f5a commented on a change in pull request #13518:
URL: https://github.com/apache/airflow/pull/13518#discussion_r560549041
##########
File path: airflow/cli/commands/webserver_command.py
##########
@@ -258,15 +258,15 @@ def _check_workers(self) -> None:
num_workers_running = self._get_num_workers_running()
if num_workers_running < self.num_workers_expected:
new_worker_count = min(
- num_workers_running - self.worker_refresh_batch_size, self.worker_refresh_batch_size
+ self.num_workers_expected - num_workers_running, self.worker_refresh_batch_size
)
- self.log.debug(
+ self.log.info(
Review comment:
@kaxil Sure, I'll do it and I'll tag you on it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil merged pull request #13518: Fix webserver exiting when gunicorn master crashes
Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #13518:
URL: https://github.com/apache/airflow/pull/13518
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #13518: Fix webserver exiting when gunicorn master crashes
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on pull request #13518:
URL: https://github.com/apache/airflow/pull/13518#issuecomment-763180256
Awesome work, congrats on your first merged pull request!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #13518: Fix webserver exiting when gunicorn master crashes
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #13518:
URL: https://github.com/apache/airflow/pull/13518#discussion_r560536153
##########
File path: airflow/cli/commands/webserver_command.py
##########
@@ -258,15 +258,15 @@ def _check_workers(self) -> None:
num_workers_running = self._get_num_workers_running()
if num_workers_running < self.num_workers_expected:
new_worker_count = min(
- num_workers_running - self.worker_refresh_batch_size, self.worker_refresh_batch_size
+ self.num_workers_expected - num_workers_running, self.worker_refresh_batch_size
)
- self.log.debug(
+ self.log.info(
Review comment:
```suggestion
self.log.debug(
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org