You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Aljoscha Krettek (JIRA)" <ji...@apache.org> on 2018/05/02 12:31:00 UTC
[jira] [Closed] (FLINK-9238) There are some questions about
BuckeingSink when sinking data to S3.
[ https://issues.apache.org/jira/browse/FLINK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aljoscha Krettek closed FLINK-9238.
-----------------------------------
Resolution: Duplicate
Fix Version/s: (was: 1.4.2)
Unfortunately, the current bucketing sink does not work with eventually-consistent file systems. See FLINK-6306
> There are some questions about BuckeingSink when sinking data to S3.
> ---------------------------------------------------------------------
>
> Key: FLINK-9238
> URL: https://issues.apache.org/jira/browse/FLINK-9238
> Project: Flink
> Issue Type: Bug
> Components: filesystem-connector
> Affects Versions: 1.4.2
> Reporter: liuzhixin
> Priority: Major
>
> When flink is sinking data to S3, there are some IOExceptions from fs. For some reasons I throw the exceptions, then the job will restart. Sometimes，fs.rename fails and there are pending part files which are from the last job and finished part files, but the count of records is not the same，slight difference. The "Exactly once" should not be the results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)