You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "vinoyang (JIRA)" <ji...@apache.org> on 2019/02/27 01:39:00 UTC
[jira] [Commented] (FLINK-5931) Make Flink highly available even if
defaultFS is unavailable
[ https://issues.apache.org/jira/browse/FLINK-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778777#comment-16778777 ]
vinoyang commented on FLINK-5931:
---------------------------------
Hi [~StephanEwen] What do you think about whether we can introduce a backup storage location configuration item? We have also experienced HDFS failures in the online environment. If Flink allows configuration of a backed up file system, it will increase the availability of the entire stream computing cluster. However, this mechanism involves the problem of metadata synchronization for storage. Synchronization in a synchronous or asynchronous manner also needs to be considered. cc [~till.rohrmann]
> Make Flink highly available even if defaultFS is unavailable
> ------------------------------------------------------------
>
> Key: FLINK-5931
> URL: https://issues.apache.org/jira/browse/FLINK-5931
> Project: Flink
> Issue Type: New Feature
> Components: Distributed Coordination
> Reporter: Haohui Mai
> Assignee: Haohui Mai
> Priority: Major
>
> In order to use Flink in mission-critical environments, Flink must be available even if the {{defaultFS}} is unavailable.
> We have deployed HDFS in HA mode in our production environment. In our experience we have experienced performance degradations and downtime when the HDFS cluster is being expanded or under maintenances. Under this case it is desirable to deploy jobs through alternative filesystem (e.g., S3).
> This jira is to track the improvements to Flink to enable Flink to continue to operate even {{defaultFS}} is unavailable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)