[jira] [Assigned] (AIRFLOW-4336) GoogleCloudStorageDownloadOperator allow not to save file to log

["Kaxil Naik (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-4336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kaxil Naik reassigned AIRFLOW-4336:
-----------------------------------

    Assignee: Kaxil Naik

> GoogleCloudStorageDownloadOperator allow not to save file to log
> ----------------------------------------------------------------
>
>                 Key: AIRFLOW-4336
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-4336
>             Project: Apache Airflow
>          Issue Type: Wish
>          Components: gcp, operators
>            Reporter: jack
>            Assignee: Kaxil Naik
>            Priority: Major
>             Fix For: 1.10.4
>
>
> The operator has 
> {code:java}
> store_to_xcom_key {code}
> flag which allow the user to deiced if to store the file in Xcom or not but there is no flag to specify if the file should be saved to the task log.
> Currently the operator download the file & auto dump the file into the log.
> We use the operator in production. It dump enormousness amount of files of GBs which spam the log with millions of rows.
> I fail to see the point of this auto dump to the log. At the end if there is a place where something shouldn't be saved to the log it's here. The operator gives a copy of the file. There is no point in having it in the log as well.
>  
> There are two approaches to fix this:
>  # Add flag and allow the user to deiced.
>  # Make it default not to save to the log.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)