[jira] [Comment Edited] (FLINK-6554) CompatibilityResult should contain a notCompatible() option

["Tzu-Li (Gordon) Tai (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-6554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16007862#comment-16007862 ] 

Tzu-Li (Gordon) Tai edited comment on FLINK-6554 at 5/12/17 9:31 AM:
---------------------------------------------------------------------

Discussed this with [~Zentol] offline.

An incompatibility of serializers should simply imply a state migration, hence there should be no need of an explicit {{notCompatible}} option.
In the case of version 1 directly to version 3, state migration would be performed as follows:
- deserialize old state with v1 serializer to objects
- write with serializer v3.

There is no need for the v2 serializer. The only case where the job would fail would be if the v1 serializer can no longer be found.

On the other hand, we do propose to add a {{CompatibilityResult.requiresMigration()}} method without a {{TypeDeserializer}} parameter to make it more explicit that the serializer can choose not to provide a convert deserializer for migration, if it cannot produce one (in which case we just try to use the old serializer).


was (Author: tzulitai):
Discussed this with [~Zentol] offline.

An incompatibility of serializers should simply imply a state migration, hence there should be no need of an explicit `notCompatible` option.
In the case of version 1 directly to version 3, state migration would be performed as follows:
- deserialize old state with v1 serializer to objects
- write with serializer v3.

There is no need for the v2 serializer. The only case where the job would fail would be if the v1 serializer can no longer be found.

On the other hand, we do propose to add a `CompatibilityResult.requiresMigration()` method without a `TypeDeserializer` parameter to make it more explicit that the serializer can choose not to provide a convert deserializer for migration, if it cannot produce one (in which case we just try to use the old serializer).

> CompatibilityResult should contain a notCompatible() option
> -----------------------------------------------------------
>
>                 Key: FLINK-6554
>                 URL: https://issues.apache.org/jira/browse/FLINK-6554
>             Project: Flink
>          Issue Type: Improvement
>          Components: Type Serialization System
>    Affects Versions: 1.3.0, 1.4.0
>            Reporter: Chesnay Schepler
>            Assignee: Tzu-Li (Gordon) Tai
>            Priority: Minor
>
> The {{CompatibilityResult}} allows a {{TypeSerializer}} to specify whether it is compatible based on the given {{TypeSerializerConfigSnapshot}}.
> As it stands the only options are {{compatible}} and {{requiresMigration}}. We should allow serializers to also notify the system of an incompatibility which should then fail the job.
> This would for example be required when a serializer provides an upgrade path version1 -> version2 -> version3, but not directly from version1 -> version3. Currently, the serializer would either have to contain logic to upgrade from every single previous version or simply throw an exception on it's own.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)