You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "jack (JIRA)" <ji...@apache.org> on 2019/01/17 13:07:00 UTC

[jira] [Commented] (AIRFLOW-3723) Add Gzip capability to existing mongo_to_S3 operator

    [ https://issues.apache.org/jira/browse/AIRFLOW-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16745026#comment-16745026 ] 

jack commented on AIRFLOW-3723:
-------------------------------

With Google Storage the ability was added to the hook https://issues.apache.org/jira/browse/AIRFLOW-2932

then it was possible to extand it to all operators using the hook.

It might be better to mimic this behavior for S3. Adding the ability to the hook and then the mongo_to_S3 can use it from the hook.

> Add Gzip capability to existing mongo_to_S3 operator
> ----------------------------------------------------
>
>                 Key: AIRFLOW-3723
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3723
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: contrib
>    Affects Versions: 1.10.1
>            Reporter: deepak saroha
>            Assignee: deepak saroha
>            Priority: Minor
>             Fix For: 2.0.0
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> A feature to add gzip capabilities while writing to S3 from Mongo is crucial as S3 has space base billing system. Gzipping it will greatly benefit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)