You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Aljoscha Krettek (JIRA)" <ji...@apache.org> on 2018/09/24 09:28:00 UTC
[jira] [Closed] (FLINK-8939) Provide better support for saving
streaming data to s3
[ https://issues.apache.org/jira/browse/FLINK-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aljoscha Krettek closed FLINK-8939.
-----------------------------------
Resolution: Duplicate
> Provide better support for saving streaming data to s3
> ------------------------------------------------------
>
> Key: FLINK-8939
> URL: https://issues.apache.org/jira/browse/FLINK-8939
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API, Streaming Connectors
> Reporter: chris snow
> Priority: Major
> Attachments: 18652BC0-DD67-42C3-9A33-12F7BC10F9F3.png
>
>
> Flink seems to struggle saving data to s3 due to the lack of a truncate method, and in my test this resulted in lots of files with a .valid-length suffix
> I’m using a bucketing sink:
> {code:java}
> return new BucketingSink<Tuple2<String, Object>>(path)
> .setWriter(writer)
> .setBucketer(new DateTimeBucketer<Tuple2<String, Object>>(formatString));{code}
>
> !18652BC0-DD67-42C3-9A33-12F7BC10F9F3.png!
> See also, the discussion in the comments here: https://issues.apache.org/jira/browse/FLINK-8543?filter=-2
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)