[GitHub] flink pull request #5401: [FLINK-8548] [examples] Add state machine example

[StephanEwen <gi...@git.apache.org>]

GitHub user StephanEwen opened a pull request:

    https://github.com/apache/flink/pull/5401

    [FLINK-8548] [examples] Add state machine example

    Example: Running a state machine for pattern detection
    ======================================================
    
    This example illustrates a minimal roll-your-own event pattern detection scenario,
    using a simple state machine that is evaluated over the stream.
    
    While this example is much simpler and more manual than what the CEP library supports,
    it illustrates the use of event processing and state management for a medium
    complex scenario.
    
    **Scenario Description**
    
    Events in streams are expected to occur in certain patterns. Any deviation from
    these patterns indicates an anomaly that the streaming system should recognize and that
    should trigger an alert.
    
    You can, for example, think of events as being generated by network devices and services,
    such as firewalls login-, and registration with an authentication service, etc. A deviation
    from expected the expected pattern might indicate an intrusion detection.
    
    The event patterns are tracked per interacting party (here simplified per source IP address)
    and are validated by a state machine. The state machine's states define what possible
    events may occur next, and what new states these events will result in.
    
    The following diagram depicts the state machine used in this example.
    
    ```
               +--<a>--> W --<b>--> Y --<e>---+
               |                    ^         |
       INITIAL-+                    |         |
               |                    |         +--> (Z) -----<g>---> TERM
               +--<c>--> X --<b>----+         |
                         |                    |
                         +--------<d>---------+
    ```
    
    **Example Program**
    
    The main class of this example program is `org.apache.flink.streaming.examples.statemachine.StateMachineExample`.
    The core logic is in the `flatMap` function, which runs the state machines per IP address.
    
    The streaming data flow is as shown below, where the source stream may come from either
    an embedded data generator, or from a from a Kafka topic:
    
    ```
     [ stream partition 1] --> source --> partition -+---> flatMap(state machine) --> sink
                                                \/
                                                /\
     [ stream partition 2] --> source --> partition -+---> flatMap(state machine) --> sink
    ```
    
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/StephanEwen/incubator-flink state_machine_example

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/5401.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5401
    
----
commit db5cc94e493584765797e57075c2b1e402017058
Author: Stephan Ewen <se...@...>
Date:   2018-02-01T18:51:59Z

    [FLINK-8548] [examples] Add state machine example
    
    This adds an example of using a state machine for pattern validation.
    The example illustrates the use of state and the kafka connector.

----


---

[GitHub] flink pull request #5401: [FLINK-8548] [examples] Add state machine example

[asfgit <gi...@git.apache.org>]

Github user asfgit closed the pull request at:

    https://github.com/apache/flink/pull/5401


---