[jira] [Work started] (AIRFLOW-5663) PythonVirtualenvOperator doesn't print logs until the operator is finished

["Wei Jiang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-5663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on AIRFLOW-5663 started by Wei Jiang.
------------------------------------------
> PythonVirtualenvOperator doesn't print logs until the operator is finished
> --------------------------------------------------------------------------
>
>                 Key: AIRFLOW-5663
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5663
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: operators
>    Affects Versions: 1.10.5
>            Reporter: Wei Jiang
>            Assignee: Wei Jiang
>            Priority: Major
>
> It seems the {{PythonVirtualenvOperator}} doesn't print out the log until the job is done. When we use logging in {{PythonOperator}}, it does print out logging in real time. It would be nice for the {{PythonVirtualenvOperator}} to have the same functionality as the {{PythonOperator}} so we can see the logging output as the PythonVirtualenvOperator makes progress.
> [https://github.com/apache/airflow/blob/master/airflow/operators/python_operator.py#L332]
> {code:python}
>     def _execute_in_subprocess(self, cmd):
>         try:
>             self.log.info("Executing cmd\n%s", cmd)
>             output = subprocess.check_output(cmd,
>                                              stderr=subprocess.STDOUT,
>                                              close_fds=True)
>             if output:
>                 self.log.info("Got output\n%s", output)
>         except subprocess.CalledProcessError as e:
>             self.log.info("Got error output\n%s", e.output)
>             raise
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)