[jira] [Created] (FLINK-7043) Optimize state handling in NFA

["Kostas Kloudas (JIRA)" <ji...@apache.org>]

Kostas Kloudas created FLINK-7043:
-------------------------------------

             Summary: Optimize state handling in NFA
                 Key: FLINK-7043
                 URL: https://issues.apache.org/jira/browse/FLINK-7043
             Project: Flink
          Issue Type: Improvement
          Components: CEP
    Affects Versions: 1.3.1
            Reporter: Kostas Kloudas
            Assignee: Dawid Wysakowicz
             Fix For: 1.4.0


The checkpointed state of the NFA is consisted of: 
1) The States/ windowTime/ handleTimeout flag (also contains the IterativeConditions)
2) The sharedBuffer
3) The computationStates

From these 3:
1) is static, as it is the description of the NFA which is static throughout the lifetime of the NFA, and
2) and 3) that are dynamic, as at any point they depend on the already consumed input.

This issue proposes to leverage the fact the 1 is static and store it on a separate keyed valueState than the NFA. This way, in the {{AbstractKeyedCEPPatternOperator.getNFA()}} which is called at each watermark and for each active key, we will just serialize/deserialize the dynamic state, while the static one we will serialize it only upon checkpointing, and deserialize it upon restoring.

We need to checkpoint also the static state because we want in the future to be able to add Patterns at runtime to the same operator, e.g. through a coStream.

So given that now we must have a way to match the dynamic with the static state of the NFA, we will have to introduce a name (or id) for a given Pattern/NFA. This will be stored with both the 
static and the dynamic state, and upon restoring we will be able to match the two based on that name. This name will be user-defined and can be specified through a `name()` method in the Pattern class (like the begin()).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)