You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/06/25 00:26:45 UTC
[GitHub] [airflow] kaxil opened a new pull request #16644: Set process title for ``serve-logs`` and ``LocalExecutor``
kaxil opened a new pull request #16644:
URL: https://github.com/apache/airflow/pull/16644
Follow up of https://github.com/apache/airflow/pull/16623.
This PR/commits adds title to serve-logs command and multiprocessing
manager for LocalExecutor.
The serve-logs process is on celery worker when using CeleryExecutor but
for LocalExecutor, it is a separate process in Scheduler.
**Before**:
```
root 124 0.0 0.0 6676 4636 pts/1 Ss Jun23 0:00 \_ -bash
root 25299 25.3 2.6 988372 326344 pts/1 Sl+ 01:30 0:09 | \_ /usr/local/bin/python /usr/local/bin/airflow webserver
root 25510 3.6 0.4 121068 57152 pts/1 S+ 01:31 0:00 | \_ gunicorn: master [airflow-webserver]
root 25555 35.7 2.5 983584 316564 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver]
root 25556 35.7 2.5 983840 316684 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver]
root 25557 35.5 2.5 983840 316548 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver]
root 25558 37.2 2.5 984920 317700 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver]
root 128 0.0 0.0 6676 4552 pts/2 Ss Jun23 0:00 \_ -bash
root 25090 5.8 0.9 467508 118808 pts/2 S+ 01:30 0:03 \_ /usr/local/bin/python /usr/local/bin/airflow scheduler
root 25098 0.0 0.7 466080 97800 pts/2 S+ 01:30 0:00 \_ /usr/local/bin/python /usr/local/bin/airflow scheduler
root 25099 0.4 0.8 1391812 99788 pts/2 Sl+ 01:30 0:00 \_ /usr/local/bin/python /usr/local/bin/airflow scheduler
root 25107 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor
root 25109 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor
root 25114 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor
root 25117 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor
root 25120 0.0 0.8 466080 98552 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor
root 25125 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor: ['airflow', 'tasks', 'run', 'example_bash_operator', 'run_after_loop', '2021-06-24T01:31:30.507415+00:00', '--lo
root 26139 0.0 0.8 468988 102204 pts/2 S+ 01:31 0:00 | \_ airflow task supervisor: ['airflow', 'tasks', 'run', 'example_bash_operator', 'run_after_loop', '2021-06-24T01:31:30.507415+00:00', '--local'
root 25128 0.0 0.7 466080 98076 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor
root 25132 0.0 0.7 466080 98076 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor
root 25137 1.1 0.8 466592 100016 pts/2 S 01:30 0:00 \_ airflow scheduler -- DagFileProcessorManager
root@a7c8aa590704:/opt/airflow# ps aux
```
**After**:
```
root 124 0.0 0.0 6676 4636 pts/1 Ss Jun23 0:00 \_ -bash
root 25299 25.3 2.6 988372 326344 pts/1 Sl+ 01:30 0:09 | \_ /usr/local/bin/python /usr/local/bin/airflow webserver
root 25510 3.6 0.4 121068 57152 pts/1 S+ 01:31 0:00 | \_ gunicorn: master [airflow-webserver]
root 25555 35.7 2.5 983584 316564 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver]
root 25556 35.7 2.5 983840 316684 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver]
root 25557 35.5 2.5 983840 316548 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver]
root 25558 37.2 2.5 984920 317700 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver]
root 128 0.0 0.0 6676 4552 pts/2 Ss Jun23 0:00 \_ -bash
root 25090 5.8 0.9 467508 118808 pts/2 S+ 01:30 0:03 \_ /usr/local/bin/python /usr/local/bin/airflow scheduler
root 25098 0.0 0.7 466080 97800 pts/2 S+ 01:30 0:00 \_ airflow serve-logs
root 25099 0.4 0.8 1391812 99788 pts/2 Sl+ 01:30 0:00 \_ airflow executor -- LocalExecutor
root 25107 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor
root 25109 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor
root 25114 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor
root 25117 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor
root 25120 0.0 0.8 466080 98552 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor
root 25125 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor: ['airflow', 'tasks', 'run', 'example_bash_operator', 'run_after_loop', '2021-06-24T01:31:30.507415+00:00', '--lo
root 26139 0.0 0.8 468988 102204 pts/2 S+ 01:31 0:00 | \_ airflow task supervisor: ['airflow', 'tasks', 'run', 'example_bash_operator', 'run_after_loop', '2021-06-24T01:31:30.507415+00:00', '--local'
root 25128 0.0 0.7 466080 98076 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor
root 25132 0.0 0.7 466080 98076 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor
root 25137 1.1 0.8 466592 100016 pts/2 S 01:30 0:00 \_ airflow scheduler -- DagFileProcessorManager
root@a7c8aa590704:/opt/airflow# ps aux
```
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #16644: Set process title for ``serve-logs`` and ``LocalExecutor``
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #16644:
URL: https://github.com/apache/airflow/pull/16644#issuecomment-868137669
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil merged pull request #16644: Set process title for ``serve-logs`` and ``LocalExecutor``
Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #16644:
URL: https://github.com/apache/airflow/pull/16644
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org