You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Jarek Potiuk (Jira)" <ji...@apache.org> on 2019/12/17 22:22:00 UTC
[jira] [Resolved] (AIRFLOW-5663) PythonVirtualenvOperator doesn't
print logs until the operator is finished
[ https://issues.apache.org/jira/browse/AIRFLOW-5663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jarek Potiuk resolved AIRFLOW-5663.
-----------------------------------
Fix Version/s: 2.0.0
Resolution: Fixed
> PythonVirtualenvOperator doesn't print logs until the operator is finished
> --------------------------------------------------------------------------
>
> Key: AIRFLOW-5663
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5663
> Project: Apache Airflow
> Issue Type: Bug
> Components: operators
> Affects Versions: 1.10.5
> Reporter: Wei Jiang
> Assignee: Wei Jiang
> Priority: Major
> Fix For: 2.0.0
>
>
> It seems the {{PythonVirtualenvOperator}} doesn't print out the log until the job is done. When we use logging in {{PythonOperator}}, it does print out logging in real time. It would be nice for the {{PythonVirtualenvOperator}} to have the same functionality as the {{PythonOperator}} so we can see the logging output as the PythonVirtualenvOperator makes progress.
> [https://github.com/apache/airflow/blob/master/airflow/operators/python_operator.py#L332]
> {code:python}
> def _execute_in_subprocess(self, cmd):
> try:
> self.log.info("Executing cmd\n%s", cmd)
> output = subprocess.check_output(cmd,
> stderr=subprocess.STDOUT,
> close_fds=True)
> if output:
> self.log.info("Got output\n%s", output)
> except subprocess.CalledProcessError as e:
> self.log.info("Got error output\n%s", e.output)
> raise
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)