[jira] [Created] (FLINK-7873) Introduce HybridStreamStateHandle for quick recovery from checkpoint.

["Sihua Zhou (JIRA)" <ji...@apache.org>]

Sihua Zhou created FLINK-7873:
---------------------------------

             Summary: Introduce HybridStreamStateHandle for quick recovery from checkpoint.
                 Key: FLINK-7873
                 URL: https://issues.apache.org/jira/browse/FLINK-7873
             Project: Flink
          Issue Type: New Feature
          Components: State Backends, Checkpointing
    Affects Versions: 1.3.2
            Reporter: Sihua Zhou
            Assignee: Sihua Zhou


Current recovery strategy will always read checkpoint data from remote FileStream (HDFS). This will cost a lot of network when the state is so big (e.g. 1T), this cost can be saved by reading the checkpoint data from local disk. So i introduce a HybridStreamStateHandler which try to create a local input stream first, if failed, it then create a remote input stream, it prototype looks like below:
{code:java}
class HybridStreamHandle {
   private FileStateHandle localHandle;
   private FileStateHandle remoteHandle;
   ......
   public FSDataInputStream openInputStream() throws IOException {
        FSDataInputStream inputStream = localHandle.openInputStream();
        return inputStream != null ? inputStream : remoteHandle.openInputStream();
    }
    .....
}
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)