[jira] [Comment Edited] (FLINK-6763) Inefficient PojoSerializerConfigSnapshot serialization format

["Tzu-Li (Gordon) Tai (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16325059#comment-16325059 ] 

Tzu-Li (Gordon) Tai edited comment on FLINK-6763 at 1/13/18 9:20 AM:
---------------------------------------------------------------------

It seems like we forgot completely about adding this to 1.4.0.

Since the previous conclusion was that we do not want to change serialization formats across minor releases, we can't include this for 1.4.1.
We should make sure we include this change in 1.5.0 (as soon as possible), as serialization formats will affect us a long way ahead.

Marking this as a blocker for 1.5.0.


was (Author: tzulitai):
It seems like we forgot completely about adding this to 1.4.0.

Since the previous conclusion was that we do not want to change serialization formats across minor releases, we can't include this for 1.4.1.
We should make sure we include this change in 1.5.0 (as soon as possible), as serialization formats will affect us a long way ahead.

> Inefficient PojoSerializerConfigSnapshot serialization format
> -------------------------------------------------------------
>
>                 Key: FLINK-6763
>                 URL: https://issues.apache.org/jira/browse/FLINK-6763
>             Project: Flink
>          Issue Type: Improvement
>          Components: State Backends, Checkpointing, Type Serialization System
>    Affects Versions: 1.3.0, 1.4.0
>            Reporter: Till Rohrmann
>            Assignee: Tzu-Li (Gordon) Tai
>
> The {{PojoSerializerConfigSnapshot}} stores for each serializer the beginning offset and ending offset in the serialization stream. This information is also written if the serializer serialization is supposed to be ignored. The beginning and ending offsets are stored as a sequence of integers at the beginning of the serialization stream. We store this information to skip broken serializers.
> I think we don't need both offsets. Instead I would suggest to write the length of the serialized serializer first into the serialization stream and then the serialized serializer. This can be done in {{TypeSerializerSerializationUtil.writeSerializer}}. When reading the serializer via {{TypeSerializerSerializationUtil.tryReadSerializer}}, we can try to deserialize the serializer. If this operation fails, then we can skip the number of serialized serializer because we know how long it was.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)