You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Chris Bandy (JIRA)" <ji...@apache.org> on 2018/02/14 21:31:00 UTC
[jira] [Created] (AIRFLOW-2108) BashOperator discards process
indentation
Chris Bandy created AIRFLOW-2108:
------------------------------------
Summary: BashOperator discards process indentation
Key: AIRFLOW-2108
URL: https://issues.apache.org/jira/browse/AIRFLOW-2108
Project: Apache Airflow
Issue Type: Bug
Components: operators
Affects Versions: 1.9.0
Reporter: Chris Bandy
When the BashOperator logs every line of output from the executing process, it strips leading whitespace which makes it difficult to interpret output that was formatted with indentation.
For example, I'm executing [PGLoader|http://pgloader.readthedocs.io/] through this operator. When it finishes, it prints a summary which appears in the logs like so:
{noformat}
[2018-02-14 07:31:44,524] {bash_operator.py:101} INFO - 2018-02-14T07:31:44.115000Z LOG report summary reset
[2018-02-14 07:31:44,564] {bash_operator.py:101} INFO - table name errors read imported bytes total time read write
[2018-02-14 07:31:44,564] {bash_operator.py:101} INFO - ---------------------- --------- --------- --------- --------- -------------- --------- ---------
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - fetch meta data 0 524 524 1.438s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Create Schemas 0 0 0 0.161s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Create SQL Types 0 19 19 20.413s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Create tables 0 310 310 3m2.316s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Set Table OIDs 0 155 155 0.458s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - ---------------------- --------- --------- --------- --------- -------------- --------- ---------
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - ---------------------- --------- --------- --------- --------- -------------- --------- ---------
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Index Build Completion 0 353 353 1m37.323s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Create Indexes 0 353 353 3m25.929s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Reset Sequences 0 0 0 2.677s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Primary Keys 0 147 147 1m21.091s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Create Foreign Keys 0 16 16 8.283s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Create Triggers 0 0 0 0.339s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Install Comments 0 0 0 0.000s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - ---------------------- --------- --------- --------- --------- -------------- --------- ---------
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Total import time ∞ 0 0 6m35.642s
{noformat}
Ideally, the leading whitespace would be retained, so the output looks like this:
{noformat}
[2018-02-14 07:31:44,524] {bash_operator.py:101} INFO - 2018-02-14T07:31:44.115000Z LOG report summary reset
[2018-02-14 07:31:44,564] {bash_operator.py:101} INFO - table name errors read imported bytes total time read write
[2018-02-14 07:31:44,564] {bash_operator.py:101} INFO - ---------------------- --------- --------- --------- --------- -------------- --------- ---------
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - fetch meta data 0 524 524 1.438s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Create Schemas 0 0 0 0.161s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Create SQL Types 0 19 19 20.413s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Create tables 0 310 310 3m2.316s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Set Table OIDs 0 155 155 0.458s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - ---------------------- --------- --------- --------- --------- -------------- --------- ---------
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - ---------------------- --------- --------- --------- --------- -------------- --------- ---------
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Index Build Completion 0 353 353 1m37.323s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Create Indexes 0 353 353 3m25.929s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Reset Sequences 0 0 0 2.677s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Primary Keys 0 147 147 1m21.091s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Create Foreign Keys 0 16 16 8.283s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Create Triggers 0 0 0 0.339s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Install Comments 0 0 0 0.000s
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - ---------------------- --------- --------- --------- --------- -------------- --------- ---------
[2018-02-14 07:31:44,567] {bash_operator.py:101} INFO - Total import time ∞ 0 0 6m35.642s
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)