[jira] [Created] (FLINK-4067) Add version header to savepoints

["Ufuk Celebi (JIRA)" <ji...@apache.org>]

Ufuk Celebi created FLINK-4067:
----------------------------------

             Summary: Add version header to savepoints
                 Key: FLINK-4067
                 URL: https://issues.apache.org/jira/browse/FLINK-4067
             Project: Flink
          Issue Type: Improvement
    Affects Versions: 1.0.3
            Reporter: Ufuk Celebi
             Fix For: 1.1.0


Adding a header with version information to savepoints ensures that we can migrate savepoints between Flink versions in the future (for example when changing internal serialization formats between versions).

After talking with Till, we propose to add the following meta data:

- Magic number (int): identify data as savepoint
- Version (int): savepoint version (independent of Flink version)
- Data Offset (int): specifies at which point the actual savepoint data starts. With this, we can allow future Flink versions to add fields to the header without breaking stuff, e.g. Flink 1.1 could read savepoints of Flink 2.0.

For Flink 1.0 savepoint support, we have to try reading the savepoints without a header before failing if we don't find the magic number.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)