[jira] [Updated] (FLINK-11947) Support MapState key / value schema evolution for RocksDB

["Aljoscha Krettek (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-11947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aljoscha Krettek updated FLINK-11947:
-------------------------------------
    Priority: Critical  (was: Blocker)

> Support MapState key / value schema evolution for RocksDB
> ---------------------------------------------------------
>
>                 Key: FLINK-11947
>                 URL: https://issues.apache.org/jira/browse/FLINK-11947
>             Project: Flink
>          Issue Type: Improvement
>          Components: API / Type Serialization System, Runtime / State Backends
>            Reporter: Tzu-Li (Gordon) Tai
>            Priority: Critical
>
> Currently, we do not attempt to perform state schema evolution if the key or value's schema of a user {{MapState}} has changed when using {{RocksDB}}:
> https://github.com/apache/flink/blob/953a5ffcbdae4115f7d525f310723cf8770779df/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java#L542
> This was disallowed in the initial support for state schema evolution because the way we did state evolution in the RocksDB state backend was simply overwriting values.
> For {{MapState}} key evolution, only overwriting RocksDB values does not work, since RocksDB entries for {{MapState}} uses a composite key containing the map state key. This means that when evolving {{MapState}} in this case with an evolved key schema, we will have new entries.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)