You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "João Boto (Jira)" <ji...@apache.org> on 2020/02/12 21:05:00 UTC
[jira] [Commented] (FLINK-11794) Allow compression of row format
files created by StreamingFileSink
[ https://issues.apache.org/jira/browse/FLINK-11794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17035707#comment-17035707 ]
João Boto commented on FLINK-11794:
-----------------------------------
this is resolved on this issue: https://github.com/apache/flink/commit/959cf62a219270ed936f5c1ce04bc38c1f456092
> Allow compression of row format files created by StreamingFileSink
> ------------------------------------------------------------------
>
> Key: FLINK-11794
> URL: https://issues.apache.org/jira/browse/FLINK-11794
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Affects Versions: 1.7.2
> Reporter: Elias Levy
> Priority: Major
>
> Currently, there is no mechanism to compress files created using a StreamingFileSink. This is highly desirable when output is a text based row format such as JSON.
> Possible alternatives are the introduction of a callback that gets passed the local file before it is uploaded to the DFS, so that it could be compressed; or a factory method could be used that returns an OutputStream, such as GZIPOutputStream, that compresses a passed in output stream that could be then used by the Encoder.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)