You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by kisimple <gi...@git.apache.org> on 2018/06/07 18:23:01 UTC
[GitHub] flink pull request #6136: FLINK-4303] [CEP] Add CEP examples
GitHub user kisimple opened a pull request:
https://github.com/apache/flink/pull/6136
FLINK-4303] [CEP] Add CEP examples
## What is the purpose of the change
Currently neither CEP Java nor CEP Scala contain a runnable example. This PR fixes the problem by adding a `flink-examples-cep` module. The change is based on #2937
## Brief change log
- Add a `flink-examples-cep` module
- Add a `TemperatureMonitoring.java` example
- Add a `TemperatureMonitoring.scala` example
## Verifying this change
- Build the project and run the example by the following commands:
`./bin/flink run -c org.apache.flink.cep.examples.java.monitoring.TemperatureMonitoring ./examples/cep/flink-examples-cep-with-dependencies.jar`
and
`./bin/flink run -c org.apache.flink.cep.examples.scala.monitoring.TemperatureMonitoring ./examples/cep/flink-examples-cep-with-dependencies.jar`
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kisimple/flink FLINK-4303
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/6136.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6136
----
commit f9f382780e0975c83f2a957132c1c09c769cf3a3
Author: Aleksandr Chermenin <al...@...>
Date: 2016-12-05T09:44:39Z
[FLINK-4303] Added examples for CEP library.
commit 7b96b9d5ffd221c0ff4169ac84c601eb34f95876
Author: Aleksandr Chermenin <al...@...>
Date: 2016-12-05T10:09:50Z
[FLINK-4303] Fixed code style.
commit f26e7f403b7d7a74cd9a0d129086468789aa2747
Author: Aleksandr Chermenin <al...@...>
Date: 2016-12-05T10:28:59Z
[FLINK-4303] Another small code style fix.
commit 3a214a38f4cda5a3e965ce4512b960ac5a7fe59f
Author: Aleksandr Chermenin <al...@...>
Date: 2016-12-05T11:24:14Z
[FLINK-4303] Fixed Scala code style.
commit 134492dc23a5eaa2173669ed213c3f933dcc4391
Author: Aleksandr Chermenin <al...@...>
Date: 2017-02-22T06:59:28Z
[FLINK-4303] Fixed hashCode methods for events.
commit ccf26763acf363134ef4bfa5127c36af226a3ed2
Author: blueszheng <ki...@...>
Date: 2018-06-07T17:35:18Z
[FLINK-4303] Add CEP examples
----
---
[GitHub] flink issue #6136: [FLINK-4303] [CEP] Add CEP examples
Posted by medcv <gi...@git.apache.org>.
Github user medcv commented on the issue:
https://github.com/apache/flink/pull/6136
+1
@kisimple Thank you for update!
---
[GitHub] flink issue #6136: FLINK-4303] [CEP] Add CEP examples
Posted by medcv <gi...@git.apache.org>.
Github user medcv commented on the issue:
https://github.com/apache/flink/pull/6136
@kisimple Recently I started to work on same example. I ran your code on my local and it did work perfectly. I could see some checkstyle errors that I think travis might pick them up.
I have a suggestion for the sake of example scenario.
For the `alerts` event you using `LocalTime` which indicate when `alerts` occurred, it would be useful also add `warnings` timestamp to `alerts` event in order to show when racks temperature passed the threshold, something like this on [line](https://github.com/kisimple/flink/blob/5cd3a374b84b2a7aaedb4c4184caded073e19295/flink-examples/flink-examples-cep/src/main/java/org/apache/flink/cep/examples/java/monitoring/TemperatureMonitoring.java#L133)
`out.collect(new TemperatureAlert(first.getRackID(), second.getDatetime()));
`
of course it needs to update the `TemperatureAlert` model also.
---
[GitHub] flink issue #6136: [FLINK-4303] [CEP] Add CEP examples
Posted by kisimple <gi...@git.apache.org>.
Github user kisimple commented on the issue:
https://github.com/apache/flink/pull/6136
cc @dawidwys
---
[GitHub] flink pull request #6136: FLINK-4303] [CEP] Add CEP examples
Posted by medcv <gi...@git.apache.org>.
Github user medcv commented on a diff in the pull request:
https://github.com/apache/flink/pull/6136#discussion_r193934789
--- Diff: flink-examples/flink-examples-cep/src/main/scala/org/apache/flink/cep/examples/scala/monitoring/TemperatureMonitoring.scala ---
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.cep.examples.scala.monitoring
+
+import org.apache.flink.cep.examples.scala.monitoring.events.{MonitoringEvent, TemperatureAlert, TemperatureEvent, TemperatureWarning}
+import org.apache.flink.cep.examples.scala.monitoring.sources.MonitoringEventSource
+import org.apache.flink.cep.scala.CEP
+import org.apache.flink.cep.scala.pattern.Pattern
+import org.apache.flink.streaming.api.TimeCharacteristic
+import org.apache.flink.streaming.api.functions.IngestionTimeExtractor
+import org.apache.flink.streaming.api.scala.{DataStream, StreamExecutionEnvironment, createTypeInformation}
+import org.apache.flink.streaming.api.windowing.time.Time
+
+/**
+ * CEP example monitoring program.
+ * This example program generates a stream of monitoring events which are analyzed using
+ * Flink's CEP library. The input event stream consists of temperature and power events
+ * from a set of racks. The goal is to detect when a rack is about to overheat.
+ * In order to do that, we create a CEP pattern which generates a TemperatureWarning
+ * whenever it sees two consecutive temperature events in a given time interval whose temperatures
+ * are higher than a given threshold value. A warning itself is not critical but if we see
+ * two warning for the same rack whose temperatures are rising, we want to generate an alert.
+ * This is achieved by defining another CEP pattern which analyzes the stream of generated
+ * temperature warnings.
+ */
+object TemperatureMonitoring {
+
+ private val TEMPERATURE_THRESHOLD = 100
+
+ def main(args: Array[String]) {
+ println("Executing temperature monitoring Scala example.")
+ val env = StreamExecutionEnvironment.getExecutionEnvironment
+
+ // Use ingestion time => TimeCharacteristic == EventTime + IngestionTimeExtractor
+ env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)
+
+ // Input stream of monitoring events
+ val inputEventStream = env.addSource(new MonitoringEventSource())
+ .assignTimestampsAndWatermarks(new IngestionTimeExtractor[MonitoringEvent])
+
+ // Warning pattern: Two consecutive temperature events whose temperature is higher
+ // than the given threshold appearing within a time interval of 10 seconds
+ val warningPattern = Pattern
+ .begin[MonitoringEvent]("first")
+ .subtype(classOf[TemperatureEvent])
+ .where(_.temperature > TEMPERATURE_THRESHOLD)
+ .next("second")
+ .subtype(classOf[TemperatureEvent])
+ .where(_.temperature > TEMPERATURE_THRESHOLD)
+ .within(Time.seconds(10))
+
+ // Create a pattern stream from our warning pattern
+ val tempPatternStream = CEP.pattern(inputEventStream.keyBy(_.rackID), warningPattern)
+
+ // Generate temperature warnings for each matched warning pattern
+ val warnings: DataStream[TemperatureWarning] = tempPatternStream.select( pattern => {
+ val first = pattern("first").head.asInstanceOf[TemperatureEvent]
+ val second = pattern("second").head.asInstanceOf[TemperatureEvent]
+ new TemperatureWarning(first.rackID, (first.temperature + second.temperature) / 2)
+ }
+ )
+
+ // Alert pattern: Two consecutive temperature warnings
+ // appearing within a time interval of 20 seconds
+ val alertPattern = Pattern
+ .begin[TemperatureWarning]("first")
+ .next("second")
+ .within(Time.seconds(20))
+
+ // Create a pattern stream from our alert pattern
+ val alertPatternStream = CEP.pattern(warnings.keyBy(_.rackID), alertPattern)
+
+ // Generate a temperature alert iff the second temperature warning's average temperature
--- End diff --
type `iff`
---
[GitHub] flink pull request #6136: [FLINK-4303] [CEP] Add CEP examples
Posted by medcv <gi...@git.apache.org>.
Github user medcv commented on a diff in the pull request:
https://github.com/apache/flink/pull/6136#discussion_r194052688
--- Diff: flink-examples/flink-examples-cep/src/main/java/org/apache/flink/cep/examples/java/monitoring/TemperatureMonitoring.java ---
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.cep.examples.java.monitoring;
+
+import org.apache.flink.cep.CEP;
+import org.apache.flink.cep.PatternFlatSelectFunction;
+import org.apache.flink.cep.PatternSelectFunction;
+import org.apache.flink.cep.PatternStream;
+import org.apache.flink.cep.examples.java.monitoring.events.MonitoringEvent;
+import org.apache.flink.cep.examples.java.monitoring.events.TemperatureAlert;
+import org.apache.flink.cep.examples.java.monitoring.events.TemperatureEvent;
+import org.apache.flink.cep.examples.java.monitoring.events.TemperatureWarning;
+import org.apache.flink.cep.examples.java.monitoring.sources.MonitoringEventSource;
+import org.apache.flink.cep.pattern.Pattern;
+import org.apache.flink.cep.pattern.conditions.SimpleCondition;
+import org.apache.flink.streaming.api.TimeCharacteristic;
+import org.apache.flink.streaming.api.datastream.DataStream;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+import org.apache.flink.streaming.api.functions.IngestionTimeExtractor;
+import org.apache.flink.streaming.api.windowing.time.Time;
+import org.apache.flink.util.Collector;
+
+import java.util.List;
+import java.util.Map;
+
+/**
+ * CEP example monitoring program.
+ * This example program generates a stream of monitoring events which are analyzed using
+ * Flink's CEP library. The input event stream consists of temperature and power events
+ * from a set of racks. The goal is to detect when a rack is about to overheat.
+ * In order to do that, we create a CEP pattern which generates a TemperatureWarning
+ * whenever it sees two consecutive temperature events in a given time interval whose temperatures
+ * are higher than a given threshold value. A warning itself is not critical but if we see
+ * two warning for the same rack whose temperatures are rising, we want to generate an alert.
+ * This is achieved by defining another CEP pattern which analyzes the stream of generated
+ * temperature warnings.
+ */
+public class TemperatureMonitoring {
+
+ private static final double TEMPERATURE_THRESHOLD = 100;
+
+ public static void main(String[] args) throws Exception {
+ System.out.println("Executing temperature monitoring Java example.");
+ StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
+
+ // Use ingestion time => TimeCharacteristic == EventTime + IngestionTimeExtractor
+ env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
+
+ // Input stream of monitoring events
+ DataStream<MonitoringEvent> inputEventStream = env.addSource(new MonitoringEventSource())
+ .assignTimestampsAndWatermarks(new IngestionTimeExtractor<>());
+
+ // Warning pattern: Two consecutive temperature events whose temperature is higher
+ // than the given threshold appearing within a time interval of 10 seconds
+ Pattern<MonitoringEvent, ?> warningPattern = Pattern
+ .<MonitoringEvent>begin("first")
+ .subtype(TemperatureEvent.class)
+ .where(new SimpleCondition<TemperatureEvent>() {
+ @Override
+ public boolean filter(TemperatureEvent event) throws Exception {
+ return event.getTemperature() > TEMPERATURE_THRESHOLD;
+ }
+ })
+ .next("second")
+ .subtype(TemperatureEvent.class)
+ .where(new SimpleCondition<TemperatureEvent>() {
+ @Override
+ public boolean filter(TemperatureEvent event) throws Exception {
+ return event.getTemperature() > TEMPERATURE_THRESHOLD;
+ }
+ })
+ .within(Time.seconds(10));
+
+ // Create a pattern stream from our warning pattern
+ PatternStream<MonitoringEvent> tempPatternStream = CEP.pattern(
+ inputEventStream.keyBy("rackID"),
+ warningPattern);
+
+ // Generate temperature warnings for each matched warning pattern
+ DataStream<TemperatureWarning> warnings = tempPatternStream.select(
+ new PatternSelectFunction<MonitoringEvent, TemperatureWarning>() {
+ @Override
+ public TemperatureWarning select(
+ Map<String, List<MonitoringEvent>> pattern) throws Exception {
+ TemperatureEvent first = (TemperatureEvent) pattern.get("first").get(0);
+ TemperatureEvent second = (TemperatureEvent) pattern.get("second").get(0);
+ return new TemperatureWarning(
+ first.getRackID(),
+ (first.getTemperature() + second.getTemperature()) / 2
+ );
+ }
+ }
+ );
+
+ // Alert pattern: Two consecutive temperature warnings
+ // appearing within a time interval of 20 seconds
+ Pattern<TemperatureWarning, ?> alertPattern = Pattern
+ .<TemperatureWarning>begin("first")
+ .next("second")
+ .within(Time.seconds(20));
+
+ // Create a pattern stream from our alert pattern
+ PatternStream<TemperatureWarning> alertPatternStream = CEP.pattern(
+ warnings.keyBy("rackID"),
+ alertPattern);
+
+ // Generate a temperature alert iff the second temperature warning's average temperature
--- End diff --
lol, didn't know lol
---
[GitHub] flink issue #6136: FLINK-4303] [CEP] Add CEP examples
Posted by kisimple <gi...@git.apache.org>.
Github user kisimple commented on the issue:
https://github.com/apache/flink/pull/6136
cc @kl0u
---
[GitHub] flink pull request #6136: FLINK-4303] [CEP] Add CEP examples
Posted by kisimple <gi...@git.apache.org>.
Github user kisimple commented on a diff in the pull request:
https://github.com/apache/flink/pull/6136#discussion_r193941724
--- Diff: flink-examples/flink-examples-cep/src/main/java/org/apache/flink/cep/examples/java/monitoring/TemperatureMonitoring.java ---
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.cep.examples.java.monitoring;
+
+import org.apache.flink.cep.CEP;
+import org.apache.flink.cep.PatternFlatSelectFunction;
+import org.apache.flink.cep.PatternSelectFunction;
+import org.apache.flink.cep.PatternStream;
+import org.apache.flink.cep.examples.java.monitoring.events.MonitoringEvent;
+import org.apache.flink.cep.examples.java.monitoring.events.TemperatureAlert;
+import org.apache.flink.cep.examples.java.monitoring.events.TemperatureEvent;
+import org.apache.flink.cep.examples.java.monitoring.events.TemperatureWarning;
+import org.apache.flink.cep.examples.java.monitoring.sources.MonitoringEventSource;
+import org.apache.flink.cep.pattern.Pattern;
+import org.apache.flink.cep.pattern.conditions.SimpleCondition;
+import org.apache.flink.streaming.api.TimeCharacteristic;
+import org.apache.flink.streaming.api.datastream.DataStream;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+import org.apache.flink.streaming.api.functions.IngestionTimeExtractor;
+import org.apache.flink.streaming.api.windowing.time.Time;
+import org.apache.flink.util.Collector;
+
+import java.util.List;
+import java.util.Map;
+
+/**
+ * CEP example monitoring program.
+ * This example program generates a stream of monitoring events which are analyzed using
+ * Flink's CEP library. The input event stream consists of temperature and power events
+ * from a set of racks. The goal is to detect when a rack is about to overheat.
+ * In order to do that, we create a CEP pattern which generates a TemperatureWarning
+ * whenever it sees two consecutive temperature events in a given time interval whose temperatures
+ * are higher than a given threshold value. A warning itself is not critical but if we see
+ * two warning for the same rack whose temperatures are rising, we want to generate an alert.
+ * This is achieved by defining another CEP pattern which analyzes the stream of generated
+ * temperature warnings.
+ */
+public class TemperatureMonitoring {
+
+ private static final double TEMPERATURE_THRESHOLD = 100;
+
+ public static void main(String[] args) throws Exception {
+ System.out.println("Executing temperature monitoring Java example.");
+ StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
+
+ // Use ingestion time => TimeCharacteristic == EventTime + IngestionTimeExtractor
+ env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
+
+ // Input stream of monitoring events
+ DataStream<MonitoringEvent> inputEventStream = env.addSource(new MonitoringEventSource())
+ .assignTimestampsAndWatermarks(new IngestionTimeExtractor<>());
+
+ // Warning pattern: Two consecutive temperature events whose temperature is higher
+ // than the given threshold appearing within a time interval of 10 seconds
+ Pattern<MonitoringEvent, ?> warningPattern = Pattern
+ .<MonitoringEvent>begin("first")
+ .subtype(TemperatureEvent.class)
+ .where(new SimpleCondition<TemperatureEvent>() {
+ @Override
+ public boolean filter(TemperatureEvent event) throws Exception {
+ return event.getTemperature() > TEMPERATURE_THRESHOLD;
+ }
+ })
+ .next("second")
+ .subtype(TemperatureEvent.class)
+ .where(new SimpleCondition<TemperatureEvent>() {
+ @Override
+ public boolean filter(TemperatureEvent event) throws Exception {
+ return event.getTemperature() > TEMPERATURE_THRESHOLD;
+ }
+ })
+ .within(Time.seconds(10));
+
+ // Create a pattern stream from our warning pattern
+ PatternStream<MonitoringEvent> tempPatternStream = CEP.pattern(
+ inputEventStream.keyBy("rackID"),
+ warningPattern);
+
+ // Generate temperature warnings for each matched warning pattern
+ DataStream<TemperatureWarning> warnings = tempPatternStream.select(
+ new PatternSelectFunction<MonitoringEvent, TemperatureWarning>() {
+ @Override
+ public TemperatureWarning select(
+ Map<String, List<MonitoringEvent>> pattern) throws Exception {
+ TemperatureEvent first = (TemperatureEvent) pattern.get("first").get(0);
+ TemperatureEvent second = (TemperatureEvent) pattern.get("second").get(0);
+ return new TemperatureWarning(
+ first.getRackID(),
+ (first.getTemperature() + second.getTemperature()) / 2
+ );
+ }
+ }
+ );
+
+ // Alert pattern: Two consecutive temperature warnings
+ // appearing within a time interval of 20 seconds
+ Pattern<TemperatureWarning, ?> alertPattern = Pattern
+ .<TemperatureWarning>begin("first")
+ .next("second")
+ .within(Time.seconds(20));
+
+ // Create a pattern stream from our alert pattern
+ PatternStream<TemperatureWarning> alertPatternStream = CEP.pattern(
+ warnings.keyBy("rackID"),
+ alertPattern);
+
+ // Generate a temperature alert iff the second temperature warning's average temperature
--- End diff --
Not a typo, just an abbreviation for `if and only if` :)
---
[GitHub] flink issue #6136: [FLINK-4303] [CEP] Add CEP examples
Posted by kisimple <gi...@git.apache.org>.
Github user kisimple commented on the issue:
https://github.com/apache/flink/pull/6136
cc @twalthr
---
[GitHub] flink issue #6136: [FLINK-4303] [CEP] Add CEP examples
Posted by kisimple <gi...@git.apache.org>.
Github user kisimple commented on the issue:
https://github.com/apache/flink/pull/6136
@medcv Thanks for your review :) Updated as your suggestions.
---
[GitHub] flink pull request #6136: FLINK-4303] [CEP] Add CEP examples
Posted by medcv <gi...@git.apache.org>.
Github user medcv commented on a diff in the pull request:
https://github.com/apache/flink/pull/6136#discussion_r193934744
--- Diff: flink-examples/flink-examples-cep/src/main/java/org/apache/flink/cep/examples/java/monitoring/TemperatureMonitoring.java ---
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.cep.examples.java.monitoring;
+
+import org.apache.flink.cep.CEP;
+import org.apache.flink.cep.PatternFlatSelectFunction;
+import org.apache.flink.cep.PatternSelectFunction;
+import org.apache.flink.cep.PatternStream;
+import org.apache.flink.cep.examples.java.monitoring.events.MonitoringEvent;
+import org.apache.flink.cep.examples.java.monitoring.events.TemperatureAlert;
+import org.apache.flink.cep.examples.java.monitoring.events.TemperatureEvent;
+import org.apache.flink.cep.examples.java.monitoring.events.TemperatureWarning;
+import org.apache.flink.cep.examples.java.monitoring.sources.MonitoringEventSource;
+import org.apache.flink.cep.pattern.Pattern;
+import org.apache.flink.cep.pattern.conditions.SimpleCondition;
+import org.apache.flink.streaming.api.TimeCharacteristic;
+import org.apache.flink.streaming.api.datastream.DataStream;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+import org.apache.flink.streaming.api.functions.IngestionTimeExtractor;
+import org.apache.flink.streaming.api.windowing.time.Time;
+import org.apache.flink.util.Collector;
+
+import java.util.List;
+import java.util.Map;
+
+/**
+ * CEP example monitoring program.
+ * This example program generates a stream of monitoring events which are analyzed using
+ * Flink's CEP library. The input event stream consists of temperature and power events
+ * from a set of racks. The goal is to detect when a rack is about to overheat.
+ * In order to do that, we create a CEP pattern which generates a TemperatureWarning
+ * whenever it sees two consecutive temperature events in a given time interval whose temperatures
+ * are higher than a given threshold value. A warning itself is not critical but if we see
+ * two warning for the same rack whose temperatures are rising, we want to generate an alert.
+ * This is achieved by defining another CEP pattern which analyzes the stream of generated
+ * temperature warnings.
+ */
+public class TemperatureMonitoring {
+
+ private static final double TEMPERATURE_THRESHOLD = 100;
+
+ public static void main(String[] args) throws Exception {
+ System.out.println("Executing temperature monitoring Java example.");
+ StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
+
+ // Use ingestion time => TimeCharacteristic == EventTime + IngestionTimeExtractor
+ env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
+
+ // Input stream of monitoring events
+ DataStream<MonitoringEvent> inputEventStream = env.addSource(new MonitoringEventSource())
+ .assignTimestampsAndWatermarks(new IngestionTimeExtractor<>());
+
+ // Warning pattern: Two consecutive temperature events whose temperature is higher
+ // than the given threshold appearing within a time interval of 10 seconds
+ Pattern<MonitoringEvent, ?> warningPattern = Pattern
+ .<MonitoringEvent>begin("first")
+ .subtype(TemperatureEvent.class)
+ .where(new SimpleCondition<TemperatureEvent>() {
+ @Override
+ public boolean filter(TemperatureEvent event) throws Exception {
+ return event.getTemperature() > TEMPERATURE_THRESHOLD;
+ }
+ })
+ .next("second")
+ .subtype(TemperatureEvent.class)
+ .where(new SimpleCondition<TemperatureEvent>() {
+ @Override
+ public boolean filter(TemperatureEvent event) throws Exception {
+ return event.getTemperature() > TEMPERATURE_THRESHOLD;
+ }
+ })
+ .within(Time.seconds(10));
+
+ // Create a pattern stream from our warning pattern
+ PatternStream<MonitoringEvent> tempPatternStream = CEP.pattern(
+ inputEventStream.keyBy("rackID"),
+ warningPattern);
+
+ // Generate temperature warnings for each matched warning pattern
+ DataStream<TemperatureWarning> warnings = tempPatternStream.select(
+ new PatternSelectFunction<MonitoringEvent, TemperatureWarning>() {
+ @Override
+ public TemperatureWarning select(
+ Map<String, List<MonitoringEvent>> pattern) throws Exception {
+ TemperatureEvent first = (TemperatureEvent) pattern.get("first").get(0);
+ TemperatureEvent second = (TemperatureEvent) pattern.get("second").get(0);
+ return new TemperatureWarning(
+ first.getRackID(),
+ (first.getTemperature() + second.getTemperature()) / 2
+ );
+ }
+ }
+ );
+
+ // Alert pattern: Two consecutive temperature warnings
+ // appearing within a time interval of 20 seconds
+ Pattern<TemperatureWarning, ?> alertPattern = Pattern
+ .<TemperatureWarning>begin("first")
+ .next("second")
+ .within(Time.seconds(20));
+
+ // Create a pattern stream from our alert pattern
+ PatternStream<TemperatureWarning> alertPatternStream = CEP.pattern(
+ warnings.keyBy("rackID"),
+ alertPattern);
+
+ // Generate a temperature alert iff the second temperature warning's average temperature
--- End diff --
type `iff`
---