[jira] [Commented] (FLINK-6482) Add nested serializers into configuration snapshots of composite serializers

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015315#comment-16015315 ] 

ASF GitHub Bot commented on FLINK-6482:
---------------------------------------

GitHub user tzulitai opened a pull request:

    https://github.com/apache/flink/pull/3937

    [FLINK-6482] [core] Add nested serializers to config snapshots of composite serializers

    This commit adds also the nested serializers themselves to the configuration snapshots of composite serializers. This opens up the opportunity to use the previous nested serializer as the convert deserializer in the case that a nested serializer in the new serializer determines that state migration is required.
    
    Take for example a Tuple serializer. If any one of the nested field serializers is not compatible, then the whole Tuple serializer will need to be migrated. Prior to this PR, this means that all serializers would need to be able to be either compatible or provide a convert deserializer. With this change, this relaxes this by opening up the opportunity to just use the previous serializer of fields as the convert deserializer.
    
    Only the composite serializers are affected by this change.
    
    This commit also consolidates all TypeSerializer-related serialization
    proxies into a single utility class. A few renamings are included, namely:
    1. `ClassNotFoundDummyTypeSerializer` --> `UnloadableDummyTypeSerializer`
    2. `TypeSerializerUtil` --> `TypeSerializerSerializationUtil` (because that's all its used for)
    3. `TypeSerializerSerializationProxy` --> functionality moved to a static method of `TypeSerializerSerializationUtil`
    4. `StateMigrationUtil` --> moved to flink-core and renamed to `CompatibilityUtil`.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/tzulitai/flink FLINK-6482

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/3937.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3937
    
----
commit a189999f799fe943f2297293a2f74707b9c4188d
Author: Tzu-Li (Gordon) Tai <tz...@apache.org>
Date:   2017-05-18T06:51:33Z

    [FLINK-6482] [core] Add nested serializers to config snapshots of composite serializers
    
    This commit adds also the nested serializers themselves to the
    configuration snapshots of composite serializers. This opens up the
    oppurtunity to use the previous nested serializer as the convert
    deserializer in the case that a nested serializer in the new serializer
    determines that state migration is required.
    
    This commit also consolidate all TypeSerializer-related serialization
    proxies into a single utility class.

----


> Add nested serializers into configuration snapshots of composite serializers
> ----------------------------------------------------------------------------
>
>                 Key: FLINK-6482
>                 URL: https://issues.apache.org/jira/browse/FLINK-6482
>             Project: Flink
>          Issue Type: Improvement
>          Components: State Backends, Checkpointing, Type Serialization System
>            Reporter: Tzu-Li (Gordon) Tai
>            Assignee: Tzu-Li (Gordon) Tai
>            Priority: Critical
>             Fix For: 1.3.0
>
>
> Currently, the composite serializers' configuration snapshots only wrap the config snapshots of nested serializers.
> We should also consider adding serialization of the nested serializers into the config snapshot, so that in the case where only some nested serializer cannot be loaded (class missing / implementation changed), we can also provide a path for serializer upgrades.
> This applies for all composite serializers that have nested serializers.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)