[jira] [Updated] (AIRFLOW-3069) Decode output of S3 file transform operator

["Szymon Bilinski (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Szymon Bilinski updated AIRFLOW-3069:
-------------------------------------
    Priority: Minor  (was: Trivial)

> Decode output of S3 file transform operator
> -------------------------------------------
>
>                 Key: AIRFLOW-3069
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3069
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: aws
>    Affects Versions: 1.10.0
>            Reporter: Szymon Bilinski
>            Assignee: Szymon Bilinski
>            Priority: Minor
>
> h3. Current behaviour
> {{S3FileTransformOperator}} logs {{stdout}} of the underlying process as such:
> {code}
> [2018-09-15 23:17:13,850] {{s3_file_transform_operator.py:122}} INFO - Transform script stdout b'Copying /tmp/tmpd5rjo8g0 to /tmp/tmpd3vkhzte\nDone\n'
> {code}
> While {{stderr}} is omitted entirely, unless exit code is not {{0}} (in this case it's included in the exception message only).
> h3. Proposed behaviour
> 1. Both streams are logged, regardless of the underlying process outcome (i.e. success or failure).
> 2. Stream output is decoded before logging (e.g. {{\n}} is replaced with an actual new line). 
> 3. If {{transform_script}} fails, the exception message contains return code of the process.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)