You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by rm...@apache.org on 2020/03/24 14:12:07 UTC
[flink-web] branch asf-site updated (92ef003 -> 5541234)
This is an automated email from the ASF dual-hosted git repository.
rmetzger pushed a change to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git.
from 92ef003 rebuild website
new d70cde9 Add Flink Patterns Blogpost Part 2
new 5541234 rebuild site
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
_posts/2020-02-26-demo-fraud-detection-2.md | 200 ++++++++++
content/blog/index.html | 36 +-
content/blog/page10/index.html | 40 +-
content/blog/page11/index.html | 25 ++
content/blog/page2/index.html | 36 +-
content/blog/page3/index.html | 36 +-
content/blog/page4/index.html | 38 +-
content/blog/page5/index.html | 42 +-
content/blog/page6/index.html | 42 +-
content/blog/page7/index.html | 40 +-
content/blog/page8/index.html | 40 +-
content/blog/page9/index.html | 40 +-
content/img/blog/patterns-blog-2/broadcast.png | Bin 0 -> 156450 bytes
content/img/blog/patterns-blog-2/forward.png | Bin 0 -> 112715 bytes
content/img/blog/patterns-blog-2/hash.png | Bin 0 -> 259743 bytes
content/img/blog/patterns-blog-2/job-graph.png | Bin 0 -> 426837 bytes
content/img/blog/patterns-blog-2/rebalance.png | Bin 0 -> 142683 bytes
content/img/blog/patterns-blog-2/rule-dsl.png | Bin 0 -> 166981 bytes
content/index.html | 8 +-
.../news/2020/02/26/demo-fraud-detection-2.html | 426 +++++++++++++++++++++
content/zh/index.html | 8 +-
img/blog/patterns-blog-2/broadcast.png | Bin 0 -> 156450 bytes
img/blog/patterns-blog-2/forward.png | Bin 0 -> 112715 bytes
img/blog/patterns-blog-2/hash.png | Bin 0 -> 259743 bytes
img/blog/patterns-blog-2/job-graph.png | Bin 0 -> 426837 bytes
img/blog/patterns-blog-2/rebalance.png | Bin 0 -> 142683 bytes
img/blog/patterns-blog-2/rule-dsl.png | Bin 0 -> 166981 bytes
27 files changed, 901 insertions(+), 156 deletions(-)
create mode 100644 _posts/2020-02-26-demo-fraud-detection-2.md
create mode 100644 content/img/blog/patterns-blog-2/broadcast.png
create mode 100644 content/img/blog/patterns-blog-2/forward.png
create mode 100644 content/img/blog/patterns-blog-2/hash.png
create mode 100644 content/img/blog/patterns-blog-2/job-graph.png
create mode 100644 content/img/blog/patterns-blog-2/rebalance.png
create mode 100644 content/img/blog/patterns-blog-2/rule-dsl.png
create mode 100644 content/news/2020/02/26/demo-fraud-detection-2.html
create mode 100644 img/blog/patterns-blog-2/broadcast.png
create mode 100644 img/blog/patterns-blog-2/forward.png
create mode 100644 img/blog/patterns-blog-2/hash.png
create mode 100644 img/blog/patterns-blog-2/job-graph.png
create mode 100644 img/blog/patterns-blog-2/rebalance.png
create mode 100644 img/blog/patterns-blog-2/rule-dsl.png
[flink-web] 01/02: Add Flink Patterns Blogpost Part 2
Posted by rm...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
rmetzger pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
commit d70cde95e7c6a13b3ae2bf22c3ba6d48f55321fc
Author: Alexander Fedulov <14...@users.noreply.github.com>
AuthorDate: Thu Feb 27 15:12:50 2020 +0100
Add Flink Patterns Blogpost Part 2
This closes #310
---
_posts/2020-02-26-demo-fraud-detection-2.md | 200 ++++++++++++++++++++++++++++
content/blog/index.html | 36 +++--
content/blog/page10/index.html | 40 +++---
content/blog/page11/index.html | 25 ++++
content/blog/page2/index.html | 36 +++--
content/blog/page3/index.html | 36 +++--
content/blog/page4/index.html | 38 +++---
content/blog/page5/index.html | 42 +++---
content/blog/page6/index.html | 42 +++---
content/blog/page7/index.html | 40 +++---
content/blog/page8/index.html | 40 +++---
content/blog/page9/index.html | 40 +++---
content/zh/index.html | 8 +-
img/blog/patterns-blog-2/broadcast.png | Bin 0 -> 156450 bytes
img/blog/patterns-blog-2/forward.png | Bin 0 -> 112715 bytes
img/blog/patterns-blog-2/hash.png | Bin 0 -> 259743 bytes
img/blog/patterns-blog-2/job-graph.png | Bin 0 -> 426837 bytes
img/blog/patterns-blog-2/rebalance.png | Bin 0 -> 142683 bytes
img/blog/patterns-blog-2/rule-dsl.png | Bin 0 -> 166981 bytes
19 files changed, 472 insertions(+), 151 deletions(-)
diff --git a/_posts/2020-02-26-demo-fraud-detection-2.md b/_posts/2020-02-26-demo-fraud-detection-2.md
new file mode 100644
index 0000000..c941b9f
--- /dev/null
+++ b/_posts/2020-02-26-demo-fraud-detection-2.md
@@ -0,0 +1,200 @@
+---
+layout: post
+title: "Advanced Flink Application Patterns Vol.2:
+Dynamic Updates of Application Logic"
+date: 2020-02-26T12:00:00.000Z
+authors:
+- alex:
+ name: "Alexander Fedulov"
+ twitter: "alex_fedulov"
+categories: news
+excerpt: In this series of blog posts you will learn about powerful Flink patterns for building streaming applications.
+---
+
+In the [first article](https://flink.apache.org/news/2020/01/15/demo-fraud-detection.html) of the series, we gave a high-level description of the objectives and required functionality of a Fraud Detection engine. We also described how to make data partitioning in Apache Flink customizable based on modifiable rules instead of using a hardcoded `KeysExtractor` implementation.
+
+We intentionally omitted details of how the applied rules are initialized and what possibilities exist for updating them at runtime. In this post, we will address exactly these details. You will learn how the approach to data partitioning described in [Part 1](https://flink.apache.org/news/2020/01/15/demo-fraud-detection.html) can be applied in combination with a dynamic configuration. These two patterns, when used together, can eliminate the need to recompile the code and redeploy your [...]
+
+## Rules Broadcasting
+
+Let's first have a look at the [previously-defined](https://flink.apache.org/news/2020/01/15/demo-fraud-detection.html#dynamic-data-partitioning) data-processing pipeline:
+
+```java
+DataStream<Alert> alerts =
+ transactions
+ .process(new DynamicKeyFunction())
+ .keyBy((keyed) -> keyed.getKey());
+ .process(new DynamicAlertFunction())
+```
+
+`DynamicKeyFunction` provides dynamic data partitioning while `DynamicAlertFunction` is responsible for executing the main logic of processing transactions and sending alert messages according to defined rules.
+
+Vol.1 of this series simplified the use case and assumed that the applied set of rules is pre-initialized and accessible via the `List<Rules>` within `DynamicKeyFunction`.
+
+```java
+public class DynamicKeyFunction
+ extends ProcessFunction<Transaction, Keyed<Transaction, String, Integer>> {
+
+ /* Simplified */
+ List<Rule> rules = /* Rules that are initialized somehow.*/;
+ ...
+}
+```
+
+Adding rules to this list is obviously possible directly inside the code of the Flink Job at the stage of its initialization (Create a `List` object; use it's `add` method). A major drawback of doing so is that it will require recompilation of the job with each rule modification. In a real Fraud Detection system, rules are expected to change on a frequent basis, making this approach unacceptable from the point of view of business and operational requirements. A different approach is needed.
+
+Next, let's take a look at a sample rule definition that we introduced in the previous post of the series:
+
+<center>
+<img src="{{ site.baseurl }}/img/blog/patterns-blog-2/rule-dsl.png" width="800px" alt="Figure 1: Rule definition"/>
+<br/>
+<i><small>Figure 1: Rule definition</small></i>
+</center>
+<br/>
+
+The previous post covered use of `groupingKeyNames` by `DynamicKeyFunction` to extract message keys. Parameters from the second part of this rule are used by `DynamicAlertFunction`: they define the actual logic of the performed operations and their parameters (such as the alert-triggering limit). This means that the same rule must be present in both `DynamicKeyFunction` and `DynamicAlertFunction`. To achieve this result, we will use the [broadcast data distribution mechanism](https://ci. [...]
+
+Figure 2 presents the final job graph of the system that we are building:
+
+<center>
+<img src="{{ site.baseurl }}/img/blog/patterns-blog-2/job-graph.png" width="800px" alt="Figure 2: Job Graph of the Fraud Detection Flink Job"/>
+<br/>
+<i><small>Figure 2: Job Graph of the Fraud Detection Flink Job</small></i>
+</center>
+<br/>
+
+The main blocks of the Transactions processing pipeline are:<br>
+
+* **Transaction Source** that consumes transaction messages from Kafka partitions in parallel. <br>
+
+* **Dynamic Key Function** that performs data enrichment with a dynamic key. The subsequent `keyBy` hashes this dynamic key and partitions the data accordingly among all parallel instances of the following operator.
+
+* **Dynamic Alert Function** that accumulates a data window and creates Alerts based on it.
+
+## Data Exchange inside Apache Flink
+
+The job graph above also indicates various data exchange patterns between the operators. In order to understand how the broadcast pattern works, let's take a short detour and discuss what methods of message propagation exist in Apache Flink's distributed runtime.
+
+* The __FORWARD__ connection after the Transaction Source means that all data consumed by one of the parallel instances of the Transaction Source operator is transferred to exactly one instance of the subsequent `DynamicKeyFunction` operator. It also indicates the same level of parallelism of the two connected operators (12 in the above case). This communication pattern is illustrated in Figure 3. Orange circles represent transactions, and dotted rectangles depict parallel instances of t [...]
+
+<center>
+<img src="{{ site.baseurl }}/img/blog/patterns-blog-2/forward.png" width="800px" alt="Figure 3: FORWARD message passing across operator instances"/>
+<br/>
+<i><small>Figure 3: FORWARD message passing across operator instances</small></i>
+</center>
+<br/>
+
+* The __HASH__ connection between `DynamicKeyFunction` and `DynamicAlertFunction` means that for each message a hash code is calculated and messages are evenly distributed among available parallel instances of the next operator. Such a connection needs to be explicitly "requested" from Flink by using `keyBy`.
+
+<center>
+<img src="{{ site.baseurl }}/img/blog/patterns-blog-2/hash.png" width="800px" alt="Figure 4: HASHED message passing across operator instances (via `keyBy`)"/>
+<br/>
+<i><small>Figure 4: HASHED message passing across operator instances (via `keyBy`)</small></i>
+</center>
+<br/>
+
+* A __REBALANCE__ distribution is either caused by an explicit call to `rebalance()` or by a change of parallelism (12 -> 1 in the case of the job graph from Figure 2). Calling `rebalance()` causes data to be repartitioned in a round-robin fashion and can help to mitigate data skew in certain scenarios.
+
+<center>
+<img src="{{ site.baseurl }}/img/blog/patterns-blog-2/rebalance.png" width="800px" alt="Figure 5: REBALANCE message passing across operator instances"/>
+<br/>
+<i><small>Figure 5: REBALANCE message passing across operator instances</small></i>
+</center>
+<br/>
+
+The Fraud Detection job graph in Figure 2 contains an additional data source: _Rules Source_. It also consumes from Kafka. Rules are "mixed into" the main processing data flow through the __BROADCAST__ channel. Unlike other methods of transmitting data between operators, such as `forward`, `hash` or `rebalance` that make each message available for processing in only one of the parallel instances of the receiving operator, `broadcast` makes each message available at the input of all of th [...]
+
+<center>
+ <img src="{{ site.baseurl }}/img/blog/patterns-blog-2/broadcast.png" width="800px" alt="Figure 6: BROADCAST message passing across operator instances"/>
+ <br/>
+ <i><small>Figure 6: BROADCAST message passing across operator instances</small></i>
+ </center>
+ <br/>
+
+<div class="alert alert-info" markdown="1">
+<span class="label label-info" style="display: inline-block"><span class="glyphicon glyphicon-info-sign" aria-hidden="true"></span> Note</span>
+There are actually a few more specialized data partitioning schemes in Flink which we did not mention here. If you want to find out more, please refer to Flink's documentation on __[stream partitioning](https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/#physical-partitioning)__.
+</div>
+
+## Broadcast State Pattern
+
+In order to make use of the Rules Source, we need to "connect" it to the main data stream:
+
+```java
+// Streams setup
+DataStream<Transaction> transactions = [...]
+DataStream<Rule> rulesUpdateStream = [...]
+
+BroadcastStream<Rule> rulesStream = rulesUpdateStream.broadcast(RULES_STATE_DESCRIPTOR);
+
+// Processing pipeline setup
+ DataStream<Alert> alerts =
+ transactions
+ .connect(rulesStream)
+ .process(new DynamicKeyFunction())
+ .keyBy((keyed) -> keyed.getKey())
+ .connect(rulesStream)
+ .process(new DynamicAlertFunction())
+```
+
+As you can see, the broadcast stream can be created from any regular stream by calling the `broadcast` method and specifying a state descriptor. Flink assumes that broadcasted data needs to be stored and retrieved while processing events of the main data flow and, therefore, always automatically creates a corresponding _broadcast state_ from this state descriptor. This is different from any other Apache Flink state type in which you need to initialize it in the `open()` method of the pr [...]
+
+```java
+public static final MapStateDescriptor<Integer, Rule> RULES_STATE_DESCRIPTOR =
+ new MapStateDescriptor<>("rules", Integer.class, Rule.class);
+```
+
+Connecting to `rulesStream` causes some changes in the signature of the processing functions. The previous article presented it in a slightly simplified way as a `ProcessFunction`. However, `DynamicKeyFunction` is actually a `BroadcastProcessFunction`.
+
+```java
+public abstract class BroadcastProcessFunction<IN1, IN2, OUT> {
+
+ public abstract void processElement(IN1 value,
+ ReadOnlyContext ctx,
+ Collector<OUT> out) throws Exception;
+
+ public abstract void processBroadcastElement(IN2 value,
+ Context ctx,
+ Collector<OUT> out) throws Exception;
+
+}
+```
+
+The difference is the addition of the `processBroadcastElement` method through which messages of the rules stream will arrive. The following new version of `DynamicKeyFunction` allows modifying the list of data-distribution keys at runtime through this stream:
+
+```java
+public class DynamicKeyFunction
+ extends BroadcastProcessFunction<Transaction, Rule, Keyed<Transaction, String, Integer>> {
+
+
+ @Override
+ public void processBroadcastElement(Rule rule,
+ Context ctx,
+ Collector<Keyed<Transaction, String, Integer>> out) {
+ BroadcastState<Integer, Rule> broadcastState = ctx.getBroadcastState(RULES_STATE_DESCRIPTOR);
+ broadcastState.put(rule.getRuleId(), rule);
+ }
+
+ @Override
+ public void processElement(Transaction event,
+ ReadOnlyContext ctx,
+ Collector<Keyed<Transaction, String, Integer>> out){
+ ReadOnlyBroadcastState<Integer, Rule> rulesState =
+ ctx.getBroadcastState(RULES_STATE_DESCRIPTOR);
+ for (Map.Entry<Integer, Rule> entry : rulesState.immutableEntries()) {
+ final Rule rule = entry.getValue();
+ out.collect(
+ new Keyed<>(
+ event, KeysExtractor.getKey(rule.getGroupingKeyNames(), event), rule.getRuleId()));
+ }
+ }
+}
+```
+
+In the above code, `processElement()` receives Transactions, and `processBroadcastElement()` receives Rule updates. When a new rule is created, it is distributed as depicted in Figure 6 and saved in all parallel instances of the operator using `processBroadcastState`. We use a Rule's ID as the key to store and reference individual rules. Instead of iterating over a hardcoded `List<Rules>`, we iterate over entries in the dynamically-updated broadcast state.
+
+`DynamicAlertFunction` follows the same logic with respect to storing the rules in the broadcast `MapState`. As described in [Part 1](https://flink.apache.org/news/2020/01/15/demo-fraud-detection.html), each message in the `processElement` input is intended to be processed by one specific rule and comes "pre-marked" with a corresponding ID by `DynamicKeyFunction`. All we need to do is retrieve the definition of the corresponding rule from `BroadcastState` by using the provided ID and pr [...]
+
+# Summary
+
+In this blog post, we continued our investigation of the use case of a Fraud Detection System built with Apache Flink. We looked into different ways in which data can be distributed between parallel operator instances and, most importantly, examined broadcast state. We demonstrated how dynamic partitioning — a pattern described in the [first part](https://flink.apache.org/news/2020/01/15/demo-fraud-detection.html) of the series — can be combined and enhanced by the functionality provided [...]
diff --git a/content/blog/index.html b/content/blog/index.html
index 173b5cf..390d053 100644
--- a/content/blog/index.html
+++ b/content/blog/index.html
@@ -267,6 +267,19 @@
<hr>
<article>
+ <h2 class="blog-title"><a href="/news/2020/01/15/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Case Study of a Fraud Detection System</a></h2>
+
+ <p>15 Jan 2020
+ Alexander Fedulov (<a href="https://twitter.com/alex_fedulov">@alex_fedulov</a>)</p>
+
+ <p>In this series of blog posts you will learn about powerful Flink patterns for building streaming applications.</p>
+
+ <p><a href="/news/2020/01/15/demo-fraud-detection-2.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2020/01/15/demo-fraud-detection.html">Advanced Flink Application Patterns Vol.1: Case Study of a Fraud Detection System</a></h2>
<p>15 Jan 2020
@@ -307,19 +320,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to query Pulsar Streams using Apache Flink</a></h2>
-
- <p>25 Nov 2019
- Sijie Guo (<a href="https://twitter.com/sijieg">@sijieg</a>) & Markos Sfikas (<a href="https://twitter.com/MarkSfik">@MarkSfik</a>)</p>
-
- <p>This blog post discusses the new developments and integrations between Apache Flink and Apache Pulsar and showcases how you can leverage Pulsar’s built-in schema to query Pulsar streams in real time using Apache Flink.</p>
-
- <p><a href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -412,6 +412,16 @@
+ <li><a href="/news/2020/01/15/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Case Study of a Fraud Detection System</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/01/15/demo-fraud-detection.html">Advanced Flink Application Patterns Vol.1: Case Study of a Fraud Detection System</a></li>
diff --git a/content/blog/page10/index.html b/content/blog/page10/index.html
index c98574d..be5f2ff 100644
--- a/content/blog/page10/index.html
+++ b/content/blog/page10/index.html
@@ -185,6 +185,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2015/04/07/march-in-flink.html">March 2015 in the Flink community</a></h2>
+
+ <p>07 Apr 2015
+ </p>
+
+ <p><p>March has been a busy month in the Flink community.</p>
+
+</p>
+
+ <p><a href="/news/2015/04/07/march-in-flink.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2015/03/13/peeking-into-Apache-Flinks-Engine-Room.html">Peeking into Apache Flink's Engine Room</a></h2>
<p>13 Mar 2015 by Fabian Hüske (<a href="https://twitter.com/">@fhueske</a>)
@@ -324,21 +339,6 @@ and offers a new API including definition of flexible windows.</p>
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2014/09/26/release-0.6.1.html">Apache Flink 0.6.1 available</a></h2>
-
- <p>26 Sep 2014
- </p>
-
- <p><p>We are happy to announce the availability of Flink 0.6.1.</p>
-
-</p>
-
- <p><a href="/news/2014/09/26/release-0.6.1.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -371,6 +371,16 @@ and offers a new API including definition of flexible windows.</p>
<ul id="markdown-toc">
+ <li><a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
diff --git a/content/blog/page11/index.html b/content/blog/page11/index.html
index 020602d..08e9918 100644
--- a/content/blog/page11/index.html
+++ b/content/blog/page11/index.html
@@ -185,6 +185,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2014/09/26/release-0.6.1.html">Apache Flink 0.6.1 available</a></h2>
+
+ <p>26 Sep 2014
+ </p>
+
+ <p><p>We are happy to announce the availability of Flink 0.6.1.</p>
+
+</p>
+
+ <p><a href="/news/2014/09/26/release-0.6.1.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2014/08/26/release-0.6.html">Apache Flink 0.6 available</a></h2>
<p>26 Aug 2014
@@ -234,6 +249,16 @@ academic and open source project that Flink originates from.</p>
<ul id="markdown-toc">
+ <li><a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
diff --git a/content/blog/page2/index.html b/content/blog/page2/index.html
index e044453..7735782 100644
--- a/content/blog/page2/index.html
+++ b/content/blog/page2/index.html
@@ -185,6 +185,19 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to query Pulsar Streams using Apache Flink</a></h2>
+
+ <p>25 Nov 2019
+ Sijie Guo (<a href="https://twitter.com/sijieg">@sijieg</a>) & Markos Sfikas (<a href="https://twitter.com/MarkSfik">@MarkSfik</a>)</p>
+
+ <p>This blog post discusses the new developments and integrations between Apache Flink and Apache Pulsar and showcases how you can leverage Pulsar’s built-in schema to query Pulsar streams in real time using Apache Flink.</p>
+
+ <p><a href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1 Released</a></h2>
<p>18 Oct 2019
@@ -310,19 +323,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/2019/05/19/state-ttl.html">State TTL in Flink 1.8.0: How to Automatically Cleanup Application State in Apache Flink</a></h2>
-
- <p>19 May 2019
- Fabian Hueske (<a href="https://twitter.com/fhueske">@fhueske</a>) & Andrey Zagrebin </p>
-
- <p>A common requirement for many stateful streaming applications is to automatically cleanup application state for effective management of your state size, or to control how long the application state can be accessed. State TTL enables application state cleanup and efficient state size management in Apache Flink</p>
-
- <p><a href="/2019/05/19/state-ttl.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -355,6 +355,16 @@
<ul id="markdown-toc">
+ <li><a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
diff --git a/content/blog/page3/index.html b/content/blog/page3/index.html
index 3538fd4..7ecead4 100644
--- a/content/blog/page3/index.html
+++ b/content/blog/page3/index.html
@@ -185,6 +185,19 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/2019/05/19/state-ttl.html">State TTL in Flink 1.8.0: How to Automatically Cleanup Application State in Apache Flink</a></h2>
+
+ <p>19 May 2019
+ Fabian Hueske (<a href="https://twitter.com/fhueske">@fhueske</a>) & Andrey Zagrebin </p>
+
+ <p>A common requirement for many stateful streaming applications is to automatically cleanup application state for effective management of your state size, or to control how long the application state can be accessed. State TTL enables application state cleanup and efficient state size management in Apache Flink</p>
+
+ <p><a href="/2019/05/19/state-ttl.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/2019/05/14/temporal-tables.html">Flux capacitor, huh? Temporal Tables and Joins in Streaming SQL</a></h2>
<p>14 May 2019
@@ -313,19 +326,6 @@ for more details.</p>
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2019/02/13/unified-batch-streaming-blink.html">Batch as a Special Case of Streaming and Alibaba's contribution of Blink</a></h2>
-
- <p>13 Feb 2019
- Stephan Ewen (<a href="https://twitter.com/stephanewen">@stephanewen</a>), Fabian Hueske (<a href="https://twitter.com/fhueske">@fhueske</a>), & Xiaowei Jiang (<a href="https://twitter.com/XiaoweiJ">@XiaoweiJ</a>)</p>
-
- <p>A few weeks ago, Alibaba contributed its Flink-fork 'Blink' back to Apache Flink. In this blog post we discuss how Blink's features will help the Flink community to make a big step towards its vision to build a truly unified system for stream and batch processing.</p>
-
- <p><a href="/news/2019/02/13/unified-batch-streaming-blink.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -358,6 +358,16 @@ for more details.</p>
<ul id="markdown-toc">
+ <li><a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
diff --git a/content/blog/page4/index.html b/content/blog/page4/index.html
index 9f039db..62c1a3e 100644
--- a/content/blog/page4/index.html
+++ b/content/blog/page4/index.html
@@ -185,6 +185,19 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2019/02/13/unified-batch-streaming-blink.html">Batch as a Special Case of Streaming and Alibaba's contribution of Blink</a></h2>
+
+ <p>13 Feb 2019
+ Stephan Ewen (<a href="https://twitter.com/stephanewen">@stephanewen</a>), Fabian Hueske (<a href="https://twitter.com/fhueske">@fhueske</a>), & Xiaowei Jiang (<a href="https://twitter.com/XiaoweiJ">@XiaoweiJ</a>)</p>
+
+ <p>A few weeks ago, Alibaba contributed its Flink-fork 'Blink' back to Apache Flink. In this blog post we discuss how Blink's features will help the Flink community to make a big step towards its vision to build a truly unified system for stream and batch processing.</p>
+
+ <p><a href="/news/2019/02/13/unified-batch-streaming-blink.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2018/12/26/release-1.5.6.html">Apache Flink 1.5.6 Released</a></h2>
<p>26 Dec 2018
@@ -321,21 +334,6 @@ Please check the <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2018/08/09/release-1.6.0.html">Apache Flink 1.6.0 Release Announcement</a></h2>
-
- <p>09 Aug 2018
- Till Rohrmann (<a href="https://twitter.com/stsffap">@stsffap</a>)</p>
-
- <p><p>The Apache Flink community is proud to announce the 1.6.0 release. Over the past 2 months, the Flink community has worked hard to resolve more than 360 issues. Please check the <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12342760">complete changelog</a> for more details.</p>
-
-</p>
-
- <p><a href="/news/2018/08/09/release-1.6.0.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -368,6 +366,16 @@ Please check the <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa
<ul id="markdown-toc">
+ <li><a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
diff --git a/content/blog/page5/index.html b/content/blog/page5/index.html
index cc976af..6288910 100644
--- a/content/blog/page5/index.html
+++ b/content/blog/page5/index.html
@@ -185,6 +185,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2018/08/09/release-1.6.0.html">Apache Flink 1.6.0 Release Announcement</a></h2>
+
+ <p>09 Aug 2018
+ Till Rohrmann (<a href="https://twitter.com/stsffap">@stsffap</a>)</p>
+
+ <p><p>The Apache Flink community is proud to announce the 1.6.0 release. Over the past 2 months, the Flink community has worked hard to resolve more than 360 issues. Please check the <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12342760">complete changelog</a> for more details.</p>
+
+</p>
+
+ <p><a href="/news/2018/08/09/release-1.6.0.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2018/07/31/release-1.5.2.html">Apache Flink 1.5.2 Released</a></h2>
<p>31 Jul 2018
@@ -313,23 +328,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2017/12/12/release-1.4.0.html">Apache Flink 1.4.0 Release Announcement</a></h2>
-
- <p>12 Dec 2017
- Aljoscha Krettek (<a href="https://twitter.com/aljoscha">@aljoscha</a>) & Mike Winters (<a href="https://twitter.com/wints">@wints</a>)</p>
-
- <p><p>The Apache Flink community is pleased to announce the 1.4.0 release. Over the past 5 months, the
-Flink community has been working hard to resolve more than 900 issues. See the <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12340533">complete changelog</a>
-for more detail.</p>
-
-</p>
-
- <p><a href="/news/2017/12/12/release-1.4.0.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -362,6 +360,16 @@ for more detail.</p>
<ul id="markdown-toc">
+ <li><a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
diff --git a/content/blog/page6/index.html b/content/blog/page6/index.html
index dd3cfb9..4e02b6a 100644
--- a/content/blog/page6/index.html
+++ b/content/blog/page6/index.html
@@ -185,6 +185,23 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2017/12/12/release-1.4.0.html">Apache Flink 1.4.0 Release Announcement</a></h2>
+
+ <p>12 Dec 2017
+ Aljoscha Krettek (<a href="https://twitter.com/aljoscha">@aljoscha</a>) & Mike Winters (<a href="https://twitter.com/wints">@wints</a>)</p>
+
+ <p><p>The Apache Flink community is pleased to announce the 1.4.0 release. Over the past 5 months, the
+Flink community has been working hard to resolve more than 900 issues. See the <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12340533">complete changelog</a>
+for more detail.</p>
+
+</p>
+
+ <p><a href="/news/2017/12/12/release-1.4.0.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2017/11/22/release-1.4-and-1.5-timeline.html">Looking Ahead to Apache Flink 1.4.0 and 1.5.0</a></h2>
<p>22 Nov 2017
@@ -317,21 +334,6 @@ what’s coming in Flink 1.4.0 as well as a preview of what the Flink community
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2017/03/23/release-1.1.5.html">Apache Flink 1.1.5 Released</a></h2>
-
- <p>23 Mar 2017
- </p>
-
- <p><p>The Apache Flink community released the next bugfix version of the Apache Flink 1.1 series.</p>
-
-</p>
-
- <p><a href="/news/2017/03/23/release-1.1.5.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -364,6 +366,16 @@ what’s coming in Flink 1.4.0 as well as a preview of what the Flink community
<ul id="markdown-toc">
+ <li><a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
diff --git a/content/blog/page7/index.html b/content/blog/page7/index.html
index d60fc28..b07af4e 100644
--- a/content/blog/page7/index.html
+++ b/content/blog/page7/index.html
@@ -185,6 +185,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2017/03/23/release-1.1.5.html">Apache Flink 1.1.5 Released</a></h2>
+
+ <p>23 Mar 2017
+ </p>
+
+ <p><p>The Apache Flink community released the next bugfix version of the Apache Flink 1.1 series.</p>
+
+</p>
+
+ <p><a href="/news/2017/03/23/release-1.1.5.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2017/02/06/release-1.2.0.html">Announcing Apache Flink 1.2.0</a></h2>
<p>06 Feb 2017 by Robert Metzger
@@ -314,21 +329,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2016/05/11/release-1.0.3.html">Flink 1.0.3 Released</a></h2>
-
- <p>11 May 2016
- </p>
-
- <p><p>Today, the Flink community released Flink version <strong>1.0.3</strong>, the third bugfix release of the 1.0 series.</p>
-
-</p>
-
- <p><a href="/news/2016/05/11/release-1.0.3.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -361,6 +361,16 @@
<ul id="markdown-toc">
+ <li><a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
diff --git a/content/blog/page8/index.html b/content/blog/page8/index.html
index 042071d..2707cf4 100644
--- a/content/blog/page8/index.html
+++ b/content/blog/page8/index.html
@@ -185,6 +185,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2016/05/11/release-1.0.3.html">Flink 1.0.3 Released</a></h2>
+
+ <p>11 May 2016
+ </p>
+
+ <p><p>Today, the Flink community released Flink version <strong>1.0.3</strong>, the third bugfix release of the 1.0 series.</p>
+
+</p>
+
+ <p><a href="/news/2016/05/11/release-1.0.3.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2016/04/22/release-1.0.2.html">Flink 1.0.2 Released</a></h2>
<p>22 Apr 2016
@@ -312,21 +327,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2015/11/27/release-0.10.1.html">Flink 0.10.1 released</a></h2>
-
- <p>27 Nov 2015
- </p>
-
- <p><p>Today, the Flink community released the first bugfix release of the 0.10 series of Flink.</p>
-
-</p>
-
- <p><a href="/news/2015/11/27/release-0.10.1.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -359,6 +359,16 @@
<ul id="markdown-toc">
+ <li><a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
diff --git a/content/blog/page9/index.html b/content/blog/page9/index.html
index 0375c43..d3e4763 100644
--- a/content/blog/page9/index.html
+++ b/content/blog/page9/index.html
@@ -185,6 +185,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2015/11/27/release-0.10.1.html">Flink 0.10.1 released</a></h2>
+
+ <p>27 Nov 2015
+ </p>
+
+ <p><p>Today, the Flink community released the first bugfix release of the 0.10 series of Flink.</p>
+
+</p>
+
+ <p><a href="/news/2015/11/27/release-0.10.1.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2015/11/16/release-0.10.0.html">Announcing Apache Flink 0.10.0</a></h2>
<p>16 Nov 2015
@@ -327,21 +342,6 @@ release is a preview release that contains known issues.</p>
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2015/04/07/march-in-flink.html">March 2015 in the Flink community</a></h2>
-
- <p>07 Apr 2015
- </p>
-
- <p><p>March has been a busy month in the Flink community.</p>
-
-</p>
-
- <p><a href="/news/2015/04/07/march-in-flink.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -374,6 +374,16 @@ release is a preview release that contains known issues.</p>
<ul id="markdown-toc">
+ <li><a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
diff --git a/content/zh/index.html b/content/zh/index.html
index 057f7ad..3e6c98f 100644
--- a/content/zh/index.html
+++ b/content/zh/index.html
@@ -554,6 +554,9 @@
<dl>
+ <dt> <a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Case Study of a Fraud Detection System</a></dt>
+ <dd>In this series of blog posts you will learn about powerful Flink patterns for building streaming applications.</dd>
+
<dt> <a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></dt>
<dd>This blog post discusses the reasons to use Flink together with Beam for your stream processing needs and takes a closer look at how Flink works with Beam under the hood.</dd>
@@ -567,11 +570,6 @@
<dt> <a href="/news/2020/02/07/a-guide-for-unit-testing-in-apache-flink.html">A Guide for Unit Testing in Apache Flink</a></dt>
<dd>This post provides a detailed guide for unit testing of Apache Flink applications.</dd>
-
- <dt> <a href="/news/2020/01/30/release-1.9.2.html">Apache Flink 1.9.2 Released</a></dt>
- <dd><p>The Apache Flink community released the second bugfix version of the Apache Flink 1.9 series.</p>
-
-</dd>
</dl>
diff --git a/img/blog/patterns-blog-2/broadcast.png b/img/blog/patterns-blog-2/broadcast.png
new file mode 100644
index 0000000..7ed8c47
Binary files /dev/null and b/img/blog/patterns-blog-2/broadcast.png differ
diff --git a/img/blog/patterns-blog-2/forward.png b/img/blog/patterns-blog-2/forward.png
new file mode 100644
index 0000000..343899d
Binary files /dev/null and b/img/blog/patterns-blog-2/forward.png differ
diff --git a/img/blog/patterns-blog-2/hash.png b/img/blog/patterns-blog-2/hash.png
new file mode 100644
index 0000000..d586293
Binary files /dev/null and b/img/blog/patterns-blog-2/hash.png differ
diff --git a/img/blog/patterns-blog-2/job-graph.png b/img/blog/patterns-blog-2/job-graph.png
new file mode 100644
index 0000000..f97c1a2
Binary files /dev/null and b/img/blog/patterns-blog-2/job-graph.png differ
diff --git a/img/blog/patterns-blog-2/rebalance.png b/img/blog/patterns-blog-2/rebalance.png
new file mode 100644
index 0000000..61d5797
Binary files /dev/null and b/img/blog/patterns-blog-2/rebalance.png differ
diff --git a/img/blog/patterns-blog-2/rule-dsl.png b/img/blog/patterns-blog-2/rule-dsl.png
new file mode 100644
index 0000000..ce1b547
Binary files /dev/null and b/img/blog/patterns-blog-2/rule-dsl.png differ
[flink-web] 02/02: rebuild site
Posted by rm...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
rmetzger pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
commit 5541234e60c8bc40e1fa72c2307e8f35340e227b
Author: Robert Metzger <rm...@apache.org>
AuthorDate: Tue Mar 24 15:11:35 2020 +0100
rebuild site
---
content/blog/index.html | 40 +-
content/img/blog/patterns-blog-2/broadcast.png | Bin 0 -> 156450 bytes
content/img/blog/patterns-blog-2/forward.png | Bin 0 -> 112715 bytes
content/img/blog/patterns-blog-2/hash.png | Bin 0 -> 259743 bytes
content/img/blog/patterns-blog-2/job-graph.png | Bin 0 -> 426837 bytes
content/img/blog/patterns-blog-2/rebalance.png | Bin 0 -> 142683 bytes
content/img/blog/patterns-blog-2/rule-dsl.png | Bin 0 -> 166981 bytes
content/index.html | 8 +-
.../news/2020/02/26/demo-fraud-detection-2.html | 426 +++++++++++++++++++++
content/zh/index.html | 2 +-
10 files changed, 450 insertions(+), 26 deletions(-)
diff --git a/content/blog/index.html b/content/blog/index.html
index 390d053..83f8232 100644
--- a/content/blog/index.html
+++ b/content/blog/index.html
@@ -185,6 +185,19 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</a></h2>
+
+ <p>26 Feb 2020
+ Alexander Fedulov (<a href="https://twitter.com/alex_fedulov">@alex_fedulov</a>)</p>
+
+ <p>In this series of blog posts you will learn about powerful Flink patterns for building streaming applications.</p>
+
+ <p><a href="/news/2020/02/26/demo-fraud-detection-2.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></h2>
<p>22 Feb 2020
@@ -267,19 +280,6 @@
<hr>
<article>
- <h2 class="blog-title"><a href="/news/2020/01/15/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Case Study of a Fraud Detection System</a></h2>
-
- <p>15 Jan 2020
- Alexander Fedulov (<a href="https://twitter.com/alex_fedulov">@alex_fedulov</a>)</p>
-
- <p>In this series of blog posts you will learn about powerful Flink patterns for building streaming applications.</p>
-
- <p><a href="/news/2020/01/15/demo-fraud-detection-2.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
- <article>
<h2 class="blog-title"><a href="/news/2020/01/15/demo-fraud-detection.html">Advanced Flink Application Patterns Vol.1: Case Study of a Fraud Detection System</a></h2>
<p>15 Jan 2020
@@ -352,7 +352,7 @@
<ul id="markdown-toc">
- <li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
+ <li><a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</a></li>
@@ -362,7 +362,7 @@
- <li><a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></li>
+ <li><a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></li>
@@ -372,7 +372,7 @@
- <li><a href="/news/2020/02/11/release-1.10.0.html">Apache Flink 1.10.0 Release Announcement</a></li>
+ <li><a href="/news/2020/02/20/ddl.html">No Java Required: Configuring Sources and Sinks in SQL</a></li>
@@ -382,7 +382,7 @@
- <li><a href="/news/2020/02/07/a-guide-for-unit-testing-in-apache-flink.html">A Guide for Unit Testing in Apache Flink</a></li>
+ <li><a href="/news/2020/02/11/release-1.10.0.html">Apache Flink 1.10.0 Release Announcement</a></li>
@@ -392,7 +392,7 @@
- <li><a href="/news/2020/01/30/release-1.9.2.html">Apache Flink 1.9.2 Released</a></li>
+ <li><a href="/news/2020/02/07/a-guide-for-unit-testing-in-apache-flink.html">A Guide for Unit Testing in Apache Flink</a></li>
@@ -402,7 +402,7 @@
- <li><a href="/news/2020/01/29/state-unlocked-interacting-with-state-in-apache-flink.html">State Unlocked: Interacting with State in Apache Flink</a></li>
+ <li><a href="/news/2020/01/30/release-1.9.2.html">Apache Flink 1.9.2 Released</a></li>
@@ -412,7 +412,7 @@
- <li><a href="/news/2020/01/15/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Case Study of a Fraud Detection System</a></li>
+ <li><a href="/news/2020/01/29/state-unlocked-interacting-with-state-in-apache-flink.html">State Unlocked: Interacting with State in Apache Flink</a></li>
diff --git a/content/img/blog/patterns-blog-2/broadcast.png b/content/img/blog/patterns-blog-2/broadcast.png
new file mode 100644
index 0000000..7ed8c47
Binary files /dev/null and b/content/img/blog/patterns-blog-2/broadcast.png differ
diff --git a/content/img/blog/patterns-blog-2/forward.png b/content/img/blog/patterns-blog-2/forward.png
new file mode 100644
index 0000000..343899d
Binary files /dev/null and b/content/img/blog/patterns-blog-2/forward.png differ
diff --git a/content/img/blog/patterns-blog-2/hash.png b/content/img/blog/patterns-blog-2/hash.png
new file mode 100644
index 0000000..d586293
Binary files /dev/null and b/content/img/blog/patterns-blog-2/hash.png differ
diff --git a/content/img/blog/patterns-blog-2/job-graph.png b/content/img/blog/patterns-blog-2/job-graph.png
new file mode 100644
index 0000000..f97c1a2
Binary files /dev/null and b/content/img/blog/patterns-blog-2/job-graph.png differ
diff --git a/content/img/blog/patterns-blog-2/rebalance.png b/content/img/blog/patterns-blog-2/rebalance.png
new file mode 100644
index 0000000..61d5797
Binary files /dev/null and b/content/img/blog/patterns-blog-2/rebalance.png differ
diff --git a/content/img/blog/patterns-blog-2/rule-dsl.png b/content/img/blog/patterns-blog-2/rule-dsl.png
new file mode 100644
index 0000000..ce1b547
Binary files /dev/null and b/content/img/blog/patterns-blog-2/rule-dsl.png differ
diff --git a/content/index.html b/content/index.html
index cf1c79d..99c2cdb 100644
--- a/content/index.html
+++ b/content/index.html
@@ -557,6 +557,9 @@
<dl>
+ <dt> <a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</a></dt>
+ <dd>In this series of blog posts you will learn about powerful Flink patterns for building streaming applications.</dd>
+
<dt> <a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></dt>
<dd>This blog post discusses the reasons to use Flink together with Beam for your stream processing needs and takes a closer look at how Flink works with Beam under the hood.</dd>
@@ -570,11 +573,6 @@
<dt> <a href="/news/2020/02/07/a-guide-for-unit-testing-in-apache-flink.html">A Guide for Unit Testing in Apache Flink</a></dt>
<dd>This post provides a detailed guide for unit testing of Apache Flink applications.</dd>
-
- <dt> <a href="/news/2020/01/30/release-1.9.2.html">Apache Flink 1.9.2 Released</a></dt>
- <dd><p>The Apache Flink community released the second bugfix version of the Apache Flink 1.9 series.</p>
-
-</dd>
</dl>
diff --git a/content/news/2020/02/26/demo-fraud-detection-2.html b/content/news/2020/02/26/demo-fraud-detection-2.html
new file mode 100644
index 0000000..bc98c28
--- /dev/null
+++ b/content/news/2020/02/26/demo-fraud-detection-2.html
@@ -0,0 +1,426 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>Apache Flink: Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</title>
+ <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+ <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+ <!-- Bootstrap -->
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.1/css/bootstrap.min.css">
+ <link rel="stylesheet" href="/css/flink.css">
+ <link rel="stylesheet" href="/css/syntax.css">
+
+ <!-- Blog RSS feed -->
+ <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" />
+
+ <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+ <!-- We need to load Jquery in the header for custom google analytics event tracking-->
+ <script src="/js/jquery.min.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+
+
+ <!-- Main content. -->
+ <div class="container">
+ <div class="row">
+
+
+ <div id="sidebar" class="col-sm-3">
+
+
+<!-- Top navbar. -->
+ <nav class="navbar navbar-default">
+ <!-- The logo. -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <div class="navbar-logo">
+ <a href="/">
+ <img alt="Apache Flink" src="/img/flink-header-logo.svg" width="147px" height="73px">
+ </a>
+ </div>
+ </div><!-- /.navbar-header -->
+
+ <!-- The navigation links. -->
+ <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
+ <ul class="nav navbar-nav navbar-main">
+
+ <!-- First menu section explains visitors what Flink is -->
+
+ <!-- What is Stream Processing? -->
+ <!--
+ <li><a href="/streamprocessing1.html">What is Stream Processing?</a></li>
+ -->
+
+ <!-- What is Flink? -->
+ <li><a href="/flink-architecture.html">What is Apache Flink?</a></li>
+
+
+
+ <!-- Use cases -->
+ <li><a href="/usecases.html">Use Cases</a></li>
+
+ <!-- Powered by -->
+ <li><a href="/poweredby.html">Powered By</a></li>
+
+
+
+ <!-- Second menu section aims to support Flink users -->
+
+ <!-- Downloads -->
+ <li><a href="/downloads.html">Downloads</a></li>
+
+ <!-- Getting Started -->
+ <li>
+ <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.10/getting-started/index.html" target="_blank">Getting Started <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+ <!-- Documentation -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.10" target="_blank">1.10 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-master" target="_blank">Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ </ul>
+ </li>
+
+ <!-- getting help -->
+ <li><a href="/gettinghelp.html">Getting Help</a></li>
+
+ <!-- Blog -->
+ <li class="active"><a href="/blog/"><b>Flink Blog</b></a></li>
+
+
+ <!-- Flink-packages -->
+ <li>
+ <a href="https://flink-packages.org" target="_blank">flink-packages.org <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+ <!-- Third menu section aim to support community and contributors -->
+
+ <!-- Community -->
+ <li><a href="/community.html">Community & Project Info</a></li>
+
+ <!-- Roadmap -->
+ <li><a href="/roadmap.html">Roadmap</a></li>
+
+ <!-- Contribute -->
+ <li><a href="/contributing/how-to-contribute.html">How to Contribute</a></li>
+
+
+ <!-- GitHub -->
+ <li>
+ <a href="https://github.com/apache/flink" target="_blank">Flink on GitHub <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+
+ <!-- Language Switcher -->
+ <li>
+
+
+ <!-- link to the Chinese home page when current is blog page -->
+ <a href="/zh">中文版</a>
+
+
+ </li>
+
+ </ul>
+
+ <ul class="nav navbar-nav navbar-bottom">
+ <hr />
+
+ <!-- Twitter -->
+ <li><a href="https://twitter.com/apacheflink" target="_blank">@ApacheFlink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <!-- Visualizer -->
+ <li class=" hidden-md hidden-sm"><a href="/visualizer/" target="_blank">Plan Visualizer <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <hr />
+
+ <li><a href="https://apache.org" target="_blank">Apache Software Foundation <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <li>
+ <style>
+ .smalllinks:link {
+ display: inline-block !important; background: none; padding-top: 0px; padding-bottom: 0px; padding-right: 0px; min-width: 75px;
+ }
+ </style>
+
+ <a class="smalllinks" href="https://www.apache.org/licenses/" target="_blank">License</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+
+ <a class="smalllinks" href="https://www.apache.org/security/" target="_blank">Security</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+
+ <a class="smalllinks" href="https://www.apache.org/foundation/sponsorship.html" target="_blank">Donate</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+
+ <a class="smalllinks" href="https://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+ </li>
+
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </nav>
+
+ </div>
+ <div class="col-sm-9">
+ <div class="row-fluid">
+ <div class="col-sm-12">
+ <div class="row">
+ <h1>Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</h1>
+
+ <article>
+ <p>26 Feb 2020 Alexander Fedulov (<a href="https://twitter.com/alex_fedulov">@alex_fedulov</a>)</p>
+
+<p>In the <a href="https://flink.apache.org/news/2020/01/15/demo-fraud-detection.html">first article</a> of the series, we gave a high-level description of the objectives and required functionality of a Fraud Detection engine. We also described how to make data partitioning in Apache Flink customizable based on modifiable rules instead of using a hardcoded <code>KeysExtractor</code> implementation.</p>
+
+<p>We intentionally omitted details of how the applied rules are initialized and what possibilities exist for updating them at runtime. In this post, we will address exactly these details. You will learn how the approach to data partitioning described in <a href="https://flink.apache.org/news/2020/01/15/demo-fraud-detection.html">Part 1</a> can be applied in combination with a dynamic configuration. These two patterns, when used together, can eliminate the need to recompile the code and [...]
+
+<h2 id="rules-broadcasting">Rules Broadcasting</h2>
+
+<p>Let’s first have a look at the <a href="https://flink.apache.org/news/2020/01/15/demo-fraud-detection.html#dynamic-data-partitioning">previously-defined</a> data-processing pipeline:</p>
+
+<div class="highlight"><pre><code class="language-java"><span class="n">DataStream</span><span class="o"><</span><span class="n">Alert</span><span class="o">></span> <span class="n">alerts</span> <span class="o">=</span>
+ <span class="n">transactions</span>
+ <span class="o">.</span><span class="na">process</span><span class="o">(</span><span class="k">new</span> <span class="nf">DynamicKeyFunction</span><span class="o">())</span>
+ <span class="o">.</span><span class="na">keyBy</span><span class="o">((</span><span class="n">keyed</span><span class="o">)</span> <span class="o">-></span> <span class="n">keyed</span><span class="o">.</span><span class="na">getKey</span><span class="o">());</span>
+ <span class="o">.</span><span class="na">process</span><span class="o">(</span><span class="k">new</span> <span class="nf">DynamicAlertFunction</span><span class="o">())</span></code></pre></div>
+
+<p><code>DynamicKeyFunction</code> provides dynamic data partitioning while <code>DynamicAlertFunction</code> is responsible for executing the main logic of processing transactions and sending alert messages according to defined rules.</p>
+
+<p>Vol.1 of this series simplified the use case and assumed that the applied set of rules is pre-initialized and accessible via the <code>List<Rules></code> within <code>DynamicKeyFunction</code>.</p>
+
+<div class="highlight"><pre><code class="language-java"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">DynamicKeyFunction</span>
+ <span class="kd">extends</span> <span class="n">ProcessFunction</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="o">{</span>
+
+ <span class="cm">/* Simplified */</span>
+ <span class="n">List</span><span class="o"><</span><span class="n">Rule</span><span class="o">></span> <span class="n">rules</span> <span class="o">=</span> <span class="cm">/* Rules that are initialized somehow.*/</span><span class="o">;</span>
+ <span class="o">...</span>
+<span class="o">}</span></code></pre></div>
+
+<p>Adding rules to this list is obviously possible directly inside the code of the Flink Job at the stage of its initialization (Create a <code>List</code> object; use it’s <code>add</code> method). A major drawback of doing so is that it will require recompilation of the job with each rule modification. In a real Fraud Detection system, rules are expected to change on a frequent basis, making this approach unacceptable from the point of view of business and operational requirements. A d [...]
+
+<p>Next, let’s take a look at a sample rule definition that we introduced in the previous post of the series:</p>
+
+<center>
+<img src="/img/blog/patterns-blog-2/rule-dsl.png" width="800px" alt="Figure 1: Rule definition" />
+<br />
+<i><small>Figure 1: Rule definition</small></i>
+</center>
+<p><br /></p>
+
+<p>The previous post covered use of <code>groupingKeyNames</code> by <code>DynamicKeyFunction</code> to extract message keys. Parameters from the second part of this rule are used by <code>DynamicAlertFunction</code>: they define the actual logic of the performed operations and their parameters (such as the alert-triggering limit). This means that the same rule must be present in both <code>DynamicKeyFunction</code> and <code>DynamicAlertFunction</code>. To achieve this result, we will u [...]
+
+<p>Figure 2 presents the final job graph of the system that we are building:</p>
+
+<center>
+<img src="/img/blog/patterns-blog-2/job-graph.png" width="800px" alt="Figure 2: Job Graph of the Fraud Detection Flink Job" />
+<br />
+<i><small>Figure 2: Job Graph of the Fraud Detection Flink Job</small></i>
+</center>
+<p><br /></p>
+
+<p>The main blocks of the Transactions processing pipeline are:<br /></p>
+
+<ul>
+ <li>
+ <p><strong>Transaction Source</strong> that consumes transaction messages from Kafka partitions in parallel. <br /></p>
+ </li>
+ <li>
+ <p><strong>Dynamic Key Function</strong> that performs data enrichment with a dynamic key. The subsequent <code>keyBy</code> hashes this dynamic key and partitions the data accordingly among all parallel instances of the following operator.</p>
+ </li>
+ <li>
+ <p><strong>Dynamic Alert Function</strong> that accumulates a data window and creates Alerts based on it.</p>
+ </li>
+</ul>
+
+<h2 id="data-exchange-inside-apache-flink">Data Exchange inside Apache Flink</h2>
+
+<p>The job graph above also indicates various data exchange patterns between the operators. In order to understand how the broadcast pattern works, let’s take a short detour and discuss what methods of message propagation exist in Apache Flink’s distributed runtime.</p>
+
+<ul>
+ <li>The <strong>FORWARD</strong> connection after the Transaction Source means that all data consumed by one of the parallel instances of the Transaction Source operator is transferred to exactly one instance of the subsequent <code>DynamicKeyFunction</code> operator. It also indicates the same level of parallelism of the two connected operators (12 in the above case). This communication pattern is illustrated in Figure 3. Orange circles represent transactions, and dotted rectangles de [...]
+</ul>
+
+<center>
+<img src="/img/blog/patterns-blog-2/forward.png" width="800px" alt="Figure 3: FORWARD message passing across operator instances" />
+<br />
+<i><small>Figure 3: FORWARD message passing across operator instances</small></i>
+</center>
+<p><br /></p>
+
+<ul>
+ <li>The <strong>HASH</strong> connection between <code>DynamicKeyFunction</code> and <code>DynamicAlertFunction</code> means that for each message a hash code is calculated and messages are evenly distributed among available parallel instances of the next operator. Such a connection needs to be explicitly “requested” from Flink by using <code>keyBy</code>.</li>
+</ul>
+
+<center>
+<img src="/img/blog/patterns-blog-2/hash.png" width="800px" alt="Figure 4: HASHED message passing across operator instances (via `keyBy`)" />
+<br />
+<i><small>Figure 4: HASHED message passing across operator instances (via `keyBy`)</small></i>
+</center>
+<p><br /></p>
+
+<ul>
+ <li>A <strong>REBALANCE</strong> distribution is either caused by an explicit call to <code>rebalance()</code> or by a change of parallelism (12 -> 1 in the case of the job graph from Figure 2). Calling <code>rebalance()</code> causes data to be repartitioned in a round-robin fashion and can help to mitigate data skew in certain scenarios.</li>
+</ul>
+
+<center>
+<img src="/img/blog/patterns-blog-2/rebalance.png" width="800px" alt="Figure 5: REBALANCE message passing across operator instances" />
+<br />
+<i><small>Figure 5: REBALANCE message passing across operator instances</small></i>
+</center>
+<p><br /></p>
+
+<p>The Fraud Detection job graph in Figure 2 contains an additional data source: <em>Rules Source</em>. It also consumes from Kafka. Rules are “mixed into” the main processing data flow through the <strong>BROADCAST</strong> channel. Unlike other methods of transmitting data between operators, such as <code>forward</code>, <code>hash</code> or <code>rebalance</code> that make each message available for processing in only one of the parallel instances of the receiving operator, <code>broa [...]
+
+<center>
+ <img src="/img/blog/patterns-blog-2/broadcast.png" width="800px" alt="Figure 6: BROADCAST message passing across operator instances" />
+ <br />
+ <i><small>Figure 6: BROADCAST message passing across operator instances</small></i>
+ </center>
+<p><br /></p>
+
+<div class="alert alert-info">
+ <p><span class="label label-info" style="display: inline-block"><span class="glyphicon glyphicon-info-sign" aria-hidden="true"></span> Note</span>
+There are actually a few more specialized data partitioning schemes in Flink which we did not mention here. If you want to find out more, please refer to Flink’s documentation on <strong><a href="https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/#physical-partitioning">stream partitioning</a></strong>.</p>
+</div>
+
+<h2 id="broadcast-state-pattern">Broadcast State Pattern</h2>
+
+<p>In order to make use of the Rules Source, we need to “connect” it to the main data stream:</p>
+
+<div class="highlight"><pre><code class="language-java"><span class="c1">// Streams setup</span>
+<span class="n">DataStream</span><span class="o"><</span><span class="n">Transaction</span><span class="o">></span> <span class="n">transactions</span> <span class="o">=</span> <span class="o">[...]</span>
+<span class="n">DataStream</span><span class="o"><</span><span class="n">Rule</span><span class="o">></span> <span class="n">rulesUpdateStream</span> <span class="o">=</span> <span class="o">[...]</span>
+
+<span class="n">BroadcastStream</span><span class="o"><</span><span class="n">Rule</span><span class="o">></span> <span class="n">rulesStream</span> <span class="o">=</span> <span class="n">rulesUpdateStream</span><span class="o">.</span><span class="na">broadcast</span><span class="o">(</span><span class="n">RULES_STATE_DESCRIPTOR</span><span class="o">);</span>
+
+<span class="c1">// Processing pipeline setup</span>
+ <span class="n">DataStream</span><span class="o"><</span><span class="n">Alert</span><span class="o">></span> <span class="n">alerts</span> <span class="o">=</span>
+ <span class="n">transactions</span>
+ <span class="o">.</span><span class="na">connect</span><span class="o">(</span><span class="n">rulesStream</span><span class="o">)</span>
+ <span class="o">.</span><span class="na">process</span><span class="o">(</span><span class="k">new</span> <span class="nf">DynamicKeyFunction</span><span class="o">())</span>
+ <span class="o">.</span><span class="na">keyBy</span><span class="o">((</span><span class="n">keyed</span><span class="o">)</span> <span class="o">-></span> <span class="n">keyed</span><span class="o">.</span><span class="na">getKey</span><span class="o">())</span>
+ <span class="o">.</span><span class="na">connect</span><span class="o">(</span><span class="n">rulesStream</span><span class="o">)</span>
+ <span class="o">.</span><span class="na">process</span><span class="o">(</span><span class="k">new</span> <span class="nf">DynamicAlertFunction</span><span class="o">())</span></code></pre></div>
+
+<p>As you can see, the broadcast stream can be created from any regular stream by calling the <code>broadcast</code> method and specifying a state descriptor. Flink assumes that broadcasted data needs to be stored and retrieved while processing events of the main data flow and, therefore, always automatically creates a corresponding <em>broadcast state</em> from this state descriptor. This is different from any other Apache Flink state type in which you need to initialize it in the <code [...]
+
+<div class="highlight"><pre><code class="language-java"><span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">MapStateDescriptor</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Rule</span><span class="o">></span> <span class="n">RULES_STATE_DESCRIPTOR</span> <span class="o">=</span>
+ <span class="k">new</span> <span class="n">MapStateDescriptor</span><span class="o"><>(</span><span class="s">"rules"</span><span class="o">,</span> <span class="n">Integer</span><span class="o">.</span><span class="na">class</span><span class="o">,</span> <span class="n">Rule</span><span class="o">.</span><span class="na">class</span><span class="o">);</span></code></pre></div>
+
+<p>Connecting to <code>rulesStream</code> causes some changes in the signature of the processing functions. The previous article presented it in a slightly simplified way as a <code>ProcessFunction</code>. However, <code>DynamicKeyFunction</code> is actually a <code>BroadcastProcessFunction</code>.</p>
+
+<div class="highlight"><pre><code class="language-java"><span class="kd">public</span> <span class="kd">abstract</span> <span class="kd">class</span> <span class="nc">BroadcastProcessFunction</span><span class="o"><</span><span class="n">IN1</span><span class="o">,</span> <span class="n">IN2</span><span class="o">,</span> <span class="n">OUT</span><span class="o">></span> <span class="o">{</span>
+
+ <span class="kd">public</span> <span class="kd">abstract</span> <span class="kt">void</span> <span class="nf">processElement</span><span class="o">(</span><span class="n">IN1</span> <span class="n">value</span><span class="o">,</span>
+ <span class="n">ReadOnlyContext</span> <span class="n">ctx</span><span class="o">,</span>
+ <span class="n">Collector</span><span class="o"><</span><span class="n">OUT</span><span class="o">></span> <span class="n">out</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span><span class="o">;</span>
+
+ <span class="kd">public</span> <span class="kd">abstract</span> <span class="kt">void</span> <span class="nf">processBroadcastElement</span><span class="o">(</span><span class="n">IN2</span> <span class="n">value</span><span class="o">,</span>
+ <span class="n">Context</span> <span class="n">ctx</span><span class="o">,</span>
+ <span class="n">Collector</span><span class="o"><</span><span class="n">OUT</span><span class="o">></span> <span class="n">out</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span><span class="o">;</span>
+
+<span class="o">}</span></code></pre></div>
+
+<p>The difference is the addition of the <code>processBroadcastElement</code> method through which messages of the rules stream will arrive. The following new version of <code>DynamicKeyFunction</code> allows modifying the list of data-distribution keys at runtime through this stream:</p>
+
+<div class="highlight"><pre><code class="language-java"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">DynamicKeyFunction</span>
+ <span class="kd">extends</span> <span class="n">BroadcastProcessFunction</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">Rule</span><span class="o">,</span> <span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="o">{</span>
+
+
+ <span class="nd">@Override</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">processBroadcastElement</span><span class="o">(</span><span class="n">Rule</span> <span class="n">rule</span><span class="o">,</span>
+ <span class="n">Context</span> <span class="n">ctx</span><span class="o">,</span>
+ <span class="n">Collector</span><span class="o"><</span><span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="n">out</span><span class="o">)</span> <span class="o">{</span>
+ <span class="n">BroadcastState</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Rule</span><span class="o">></span> <span class="n">broadcastState</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="na">getBroadcastState</span><span class="o">(</span><span class="n">RULES_STATE_DESCRIPTOR</span><span class="o">);</span>
+ <span class="n">broadcastState</span><span class="o">.</span><span class="na">put</span><span class="o">(</span><span class="n">rule</span><span class="o">.</span><span class="na">getRuleId</span><span class="o">(),</span> <span class="n">rule</span><span class="o">);</span>
+ <span class="o">}</span>
+
+ <span class="nd">@Override</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">processElement</span><span class="o">(</span><span class="n">Transaction</span> <span class="n">event</span><span class="o">,</span>
+ <span class="n">ReadOnlyContext</span> <span class="n">ctx</span><span class="o">,</span>
+ <span class="n">Collector</span><span class="o"><</span><span class="n">Keyed</span><span class="o"><</span><span class="n">Transaction</span><span class="o">,</span> <span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="n">out</span><span class="o">){</span>
+ <span class="n">ReadOnlyBroadcastState</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Rule</span><span class="o">></span> <span class="n">rulesState</span> <span class="o">=</span>
+ <span class="n">ctx</span><span class="o">.</span><span class="na">getBroadcastState</span><span class="o">(</span><span class="n">RULES_STATE_DESCRIPTOR</span><span class="o">);</span>
+ <span class="k">for</span> <span class="o">(</span><span class="n">Map</span><span class="o">.</span><span class="na">Entry</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Rule</span><span class="o">></span> <span class="n">entry</span> <span class="o">:</span> <span class="n">rulesState</span><span class="o">.</span><span class="na">immutableEntries</span><span class="o">())</span> <span class="o">{</span>
+ <span class="kd">final</span> <span class="n">Rule</span> <span class="n">rule</span> <span class="o">=</span> <span class="n">entry</span><span class="o">.</span><span class="na">getValue</span><span class="o">();</span>
+ <span class="n">out</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span>
+ <span class="k">new</span> <span class="n">Keyed</span><span class="o"><>(</span>
+ <span class="n">event</span><span class="o">,</span> <span class="n">KeysExtractor</span><span class="o">.</span><span class="na">getKey</span><span class="o">(</span><span class="n">rule</span><span class="o">.</span><span class="na">getGroupingKeyNames</span><span class="o">(),</span> <span class="n">event</span><span class="o">),</span> <span class="n">rule</span><span class="o">.</span><span class="na">getRuleId</span><span class="o">()));</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+<span class="o">}</span></code></pre></div>
+
+<p>In the above code, <code>processElement()</code> receives Transactions, and <code>processBroadcastElement()</code> receives Rule updates. When a new rule is created, it is distributed as depicted in Figure 6 and saved in all parallel instances of the operator using <code>processBroadcastState</code>. We use a Rule’s ID as the key to store and reference individual rules. Instead of iterating over a hardcoded <code>List<Rules></code>, we iterate over entries in the dynamically-upd [...]
+
+<p><code>DynamicAlertFunction</code> follows the same logic with respect to storing the rules in the broadcast <code>MapState</code>. As described in <a href="https://flink.apache.org/news/2020/01/15/demo-fraud-detection.html">Part 1</a>, each message in the <code>processElement</code> input is intended to be processed by one specific rule and comes “pre-marked” with a corresponding ID by <code>DynamicKeyFunction</code>. All we need to do is retrieve the definition of the corresponding [...]
+
+<h1 id="summary">Summary</h1>
+
+<p>In this blog post, we continued our investigation of the use case of a Fraud Detection System built with Apache Flink. We looked into different ways in which data can be distributed between parallel operator instances and, most importantly, examined broadcast state. We demonstrated how dynamic partitioning — a pattern described in the <a href="https://flink.apache.org/news/2020/01/15/demo-fraud-detection.html">first part</a> of the series — can be combined and enhanced by the function [...]
+
+ </article>
+ </div>
+
+ <div class="row">
+ <div id="disqus_thread"></div>
+ <script type="text/javascript">
+ /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE * * */
+ var disqus_shortname = 'stratosphere-eu'; // required: replace example with your forum shortname
+
+ /* * * DON'T EDIT BELOW THIS LINE * * */
+ (function() {
+ var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
+ dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';
+ (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
+ })();
+ </script>
+ </div>
+ </div>
+</div>
+ </div>
+ </div>
+
+ <hr />
+
+ <div class="row">
+ <div class="footer text-center col-sm-12">
+ <p>Copyright © 2014-2019 <a href="http://apache.org">The Apache Software Foundation</a>. All Rights Reserved.</p>
+ <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p>
+ <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p>
+ </div>
+ </div>
+ </div><!-- /.container -->
+
+ <!-- Include all compiled plugins (below), or include individual files as needed -->
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery.matchHeight/0.7.0/jquery.matchHeight-min.js"></script>
+ <script src="/js/codetabs.js"></script>
+ <script src="/js/stickysidebar.js"></script>
+
+ <!-- Google Analytics -->
+ <script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-52545728-1', 'auto');
+ ga('send', 'pageview');
+ </script>
+ </body>
+</html>
diff --git a/content/zh/index.html b/content/zh/index.html
index 3e6c98f..4955950 100644
--- a/content/zh/index.html
+++ b/content/zh/index.html
@@ -554,7 +554,7 @@
<dl>
- <dt> <a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Case Study of a Fraud Detection System</a></dt>
+ <dt> <a href="/news/2020/02/26/demo-fraud-detection-2.html">Advanced Flink Application Patterns Vol.2: Dynamic Updates of Application Logic</a></dt>
<dd>In this series of blog posts you will learn about powerful Flink patterns for building streaming applications.</dd>
<dt> <a href="/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-top-of-flink.html">Apache Beam: How Beam Runs on Top of Flink</a></dt>