You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/03/10 13:57:08 UTC

[GitHub] [airflow] pkqs90 commented on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution.

pkqs90 commented on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution.
URL: https://github.com/apache/airflow/pull/6627#issuecomment-597099606
 
 
   Hi guys, I met some logging issues when upgrading to 1.10.7. Subprocess loggings can't be found in airflow webserver nor local task logs anymore. After digging around, I find this PR is related. The old way of spawning a process has a daemon thread that handles the log streams.
   https://github.com/apache/airflow/blob/master/airflow/task/task_runner/base_task_runner.py#L134
   
   After I change the code to always create a subprocess without fork, the logs come right back. Can someone take a look? Thanks in advance.
   
   Jira ticket: https://issues.apache.org/jira/browse/AIRFLOW-7030

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services