You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Jingsong Lee (Jira)" <ji...@apache.org> on 2020/05/17 16:50:00 UTC
[jira] [Closed] (FLINK-17593) Support arbitrary recovery mechanism
for PartFileWriter
[ https://issues.apache.org/jira/browse/FLINK-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jingsong Lee closed FLINK-17593.
--------------------------------
Resolution: Fixed
master: 339f5d84d0b7e9fe64534ea4e2adf7e35dee8398
> Support arbitrary recovery mechanism for PartFileWriter
> -------------------------------------------------------
>
> Key: FLINK-17593
> URL: https://issues.apache.org/jira/browse/FLINK-17593
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / FileSystem
> Reporter: Yun Gao
> Assignee: Guowei Ma
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
>
> Currently Bucket relies directly on _RecoverableOutputStream_ provided by FileSystem to achieve snapshotting and recovery the in-progress part file for all the PartFileWriter implementations. This would require that the PartFileWriter must be based on the OutputStream.
> To support the path-based PartFileWriter required by the Hive Sink, we will first need to abstract the snapshotting mechanism of the PartFileWriter and make RecoverableOutputStream to be one type of implementation, thus we could decouple PartFileWriter with the output streams.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)