You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Aljoscha Krettek (JIRA)" <ji...@apache.org> on 2016/04/08 13:43:25 UTC

[jira] [Created] (FLINK-3718) Add Option For Completely Async Backup in RocksDB State Backend

Aljoscha Krettek created FLINK-3718:
---------------------------------------

             Summary: Add Option For Completely Async Backup in RocksDB State Backend
                 Key: FLINK-3718
                 URL: https://issues.apache.org/jira/browse/FLINK-3718
             Project: Flink
          Issue Type: Bug
          Components: Streaming
            Reporter: Aljoscha Krettek
            Assignee: Aljoscha Krettek


Right now, the snapshotting for RocksDB has a synchronous part where a backup of the RocksDB database is drawn and an asynchronous part where this backup is written to HDFS.

We should add an option that uses the snapshot feature of RocksDB to get an iterator over all keys at a set point in time. The iterator can be used to store everything to HDFS. Normal operation can continue while we store the keys. This makes the snapshot completely asynchronous.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)