[jira] [Commented] (BEAM-626) AvroCoder not deserializing correctly in Kryo

["Luke Cwik (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15617003#comment-15617003 ] 

Luke Cwik commented on BEAM-626:
--------------------------------

This will only solve the short term problem that AvroCoder is not serializable via Kryo.

Users who write their own objects that rely on readResolve will still have the same problem that your facing with AvroCoder. They will need to do additional work to get their objects to work with the Spark runner.

We'll need an official schema / serialization story for many of the objects used such as Coder/DoFn/... to be part of the Beam model for portability reasons but until then it seems worthwhile to fix this.

> AvroCoder not deserializing correctly in Kryo
> ---------------------------------------------
>
>                 Key: BEAM-626
>                 URL: https://issues.apache.org/jira/browse/BEAM-626
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-core
>            Reporter: Aviem Zur
>            Assignee: Aviem Zur
>            Priority: Minor
>
> Unlike with Java serialization, when deserializing AvroCoder using Kryo, the resulting AvroCoder is missing all of its transient fields.
> The reason it works with Java serialization is because of the usage of writeReplace and readResolve, which Kryo does not adhere to.
> In ProtoCoder for example there are also unserializable members, the way it is solved there is lazy initializing these members via their getters, so they are initialized in the deserialized object on first call to the member.
> It seems AvroCoder is the only class in Beam to use writeReplace convention.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)