[jira] [Commented] (FLINK-5420) Make CEP operators rescalable

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15865674#comment-15865674 ] 

ASF GitHub Bot commented on FLINK-5420:
---------------------------------------

GitHub user kl0u opened a pull request:

    https://github.com/apache/flink/pull/3307

    [FLINK-5420] Make CEP operators rescalable

    Transforms the CEP operators into ProcessFunctions that use only managed keyed state. Rescalability now comes out-of-the-box. In addition, for the keyed operator, the list of seen keys is replaced by timers set by the incoming elements that will fire at the next watermark.
    
    More information about how rescalability is achieved can be found here:
    https://issues.apache.org/jira/browse/FLINK-5420

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kl0u/flink cep-ref

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/3307.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3307
    
----
commit 230ee80355f7219d56d1010d8850dbe39866c7e8
Author: kl0u <kk...@gmail.com>
Date:   2017-01-27T14:30:32Z

    [FLINK-5420] Make CEP operators rescalable
    
    Transforms the CEP operators into ProcessFunctions
    that use only managed keyed state. Rescalability
    now comes out-of-the-box. In addition, for the
    keyed operator, the list of seen keys is replaced
    by timers set by the incoming elements that will
    fire at the next watermark.

----


> Make CEP operators rescalable
> -----------------------------
>
>                 Key: FLINK-5420
>                 URL: https://issues.apache.org/jira/browse/FLINK-5420
>             Project: Flink
>          Issue Type: Bug
>          Components: CEP
>    Affects Versions: 1.2.0
>            Reporter: Kostas Kloudas
>            Assignee: Kostas Kloudas
>
> This issue targets making the operators in the CEP library re-scalable. After this is implemented, the user will be able to take a savepoint and restart his job with a different parallelism.
> The way to do it is to transform the CEP operators into the newly introduced {{ProcessFunction}} and use only managed keyed state to store their state. With this transformation, rescalability will come out-of-the-box. In addition, for the keyed operator and for event time, we will not have to keep the already seen keys in a list, but we can replace them with timers set for each incoming element (in the {{ProcessFunction#processElement()}}) and made to fire at the next watermark (their timestamp will be the that of the element itself). These timers will be set to fire at the next watermark and when they fire, they will register another timer for the next watermark (in the {{ProcessFunction#onTimer()}} they will re-register themselves with a timestamp equal to {{currentWatermark() + 1}}). This will preserve the previous behavior of the operators.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)