[jira] [Comment Edited] (FLINK-11947) Support MapState key / value schema evolution for RocksDB

["Tzu-Li (Gordon) Tai (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-11947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16848693#comment-16848693 ] 

Tzu-Li (Gordon) Tai edited comment on FLINK-11947 at 5/27/19 8:01 AM:
----------------------------------------------------------------------

[~cresny]
Not necessarily. It was downgraded to be a non-blocker, only because due to the nature Flink's timely release (every 3 months), new features should never block a release.
Only issues that are potentially bugs / regressions / affect stability of development branches should really be marked as blockers.

For this feature, I still find it quite important, and would like it to be in 1.9.0.
However, I'm currently quite occupied with other tasks, so probably wouldn't be able to work on it myself until a few weeks later (feature freeze is near end of June).
If someone else is willing to take ownership of this ticket, I'll be happy to shepherd the effort.


was (Author: tzulitai):
[~cresny]
Not necessarily. It was downgraded to be a non-blocker, because due to the nature Flink's timely release (every 3 months), new features should never block a release.
Only issues that are potentially bugs / regressions / affect stability of development branches should really be marked as blockers.

For this feature, I still find it quite important, and would like it to be in 1.9.0.
However, I'm currently quite occupied with other tasks, so probably wouldn't be able to work on it myself until a few weeks later (feature freeze is near end of June).
If someone else is willing to take ownership of this ticket, I'll be happy to shepherd the effort.

> Support MapState key / value schema evolution for RocksDB
> ---------------------------------------------------------
>
>                 Key: FLINK-11947
>                 URL: https://issues.apache.org/jira/browse/FLINK-11947
>             Project: Flink
>          Issue Type: Improvement
>          Components: API / Type Serialization System, Runtime / State Backends
>            Reporter: Tzu-Li (Gordon) Tai
>            Priority: Critical
>
> Currently, we do not attempt to perform state schema evolution if the key or value's schema of a user {{MapState}} has changed when using {{RocksDB}}:
> https://github.com/apache/flink/blob/953a5ffcbdae4115f7d525f310723cf8770779df/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java#L542
> This was disallowed in the initial support for state schema evolution because the way we did state evolution in the RocksDB state backend was simply overwriting values.
> For {{MapState}} key evolution, only overwriting RocksDB values does not work, since RocksDB entries for {{MapState}} uses a composite key containing the map state key. This means that when evolving {{MapState}} in this case with an evolved key schema, we will have new entries.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)