[jira] [Commented] (FLINK-4149) Fix Serialization of NFA in AbstractKeyedCEPPatternOperator

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362717#comment-15362717 ] 

ASF GitHub Bot commented on FLINK-4149:
---------------------------------------

GitHub user aljoscha opened a pull request:

    https://github.com/apache/flink/pull/2202

    [FLINK-4149] Fix Serialization of NFA in AbstractKeyedCEPPatternOperator

    NFA is Serializable and has readObject()/writeObject() methods. In
    AbstractKeyedCEPPatternOperator a KryoSerializer was used as the
    TypeSerializer for the ValueState that holds NFA instances. Kryo does
    not call readObject()/writeObject() therefore the state of the NFA was
    invalid after deserialization.
    
    This change adds a new TypeSerializer for NFA that uses
    Java Serialization. In the long run it will be better to get rid of the
    readObject()/writeObject() methods and instead efficiently serialize
    using a specialized TypeSerializer.
    
    This also adds a test that verifies that checkpoint/restore work as expected.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/aljoscha/flink cep/fix-serialization

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/2202.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2202
    
----
commit ba1deb8cae7223f3ca9def1b09b311e7d7889a8a
Author: Aljoscha Krettek <al...@gmail.com>
Date:   2016-07-05T15:56:49Z

    Replace StreamEvent by StreamRecord in CEP Tests

commit 19aaa005e0389b512a563cf28e82d8ad07c4a194
Author: Aljoscha Krettek <al...@gmail.com>
Date:   2016-07-05T15:58:45Z

    [FLINK-4149] Fix Serialization of NFA in AbstractKeyedCEPPatternOperator
    
    NFA is Serializable and has readObject()/writeObject() methods. In
    AbstractKeyedCEPPatternOperator a KryoSerializer was used as the
    TypeSerializer for the ValueState that holds NFA instances. Kryo does
    not call readObject()/writeObject() therefore the state of the NFA was
    invalid after deserialization.
    
    This change adds a new TypeSerializer for NFA that uses
    Java Serialization. In the long run it will be better to get rid of the
    readObject()/writeObject() methods and instead efficiently serialize
    using a specialized TypeSerializer.

----


> Fix Serialization of NFA in AbstractKeyedCEPPatternOperator
> -----------------------------------------------------------
>
>                 Key: FLINK-4149
>                 URL: https://issues.apache.org/jira/browse/FLINK-4149
>             Project: Flink
>          Issue Type: Bug
>          Components: CEP
>    Affects Versions: 1.0.0, 1.1.0, 1.0.1, 1.0.2, 1.0.3
>            Reporter: Aljoscha Krettek
>            Assignee: Aljoscha Krettek
>            Priority: Blocker
>             Fix For: 1.1.0
>
>
> A job that uses CEP fails upon restore with a {{NullPointerException}} in {{NFA.process()}}. The reason seems to be that field {{computationStates}} is {{null}}. This field is transient and read in a custom {{readObject()}} method.
> In {{AbstractKeyedCEPPatternOperator}} this snipped is used to construct a {{StateDescriptor}} for an {{NFA}} state:
> {code}
> new ValueStateDescriptor<NFA<IN>>(
>     NFA_OPERATOR_STATE_NAME,
>     new KryoSerializer<NFA<IN>>((Class<NFA<IN>>) (Class<?>) NFA.class, getExecutionConfig()),
>     null)
> {code}
> It seems Kryo does not invoke {{readObject}}/{{writeObject}}. We probably need a custom {{TypeSerializer}} for {{NFA}} to solve the problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)