You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2020/09/18 00:41:48 UTC
[beam] branch asf-site updated: Publishing website 2020/09/18
00:41:21 at commit f4c2734
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 389272c Publishing website 2020/09/18 00:41:21 at commit f4c2734
389272c is described below
commit 389272c63844360455899a19e3cc673aee1ec5f0
Author: jenkins <bu...@apache.org>
AuthorDate: Fri Sep 18 00:41:30 2020 +0000
Publishing website 2020/09/18 00:41:21 at commit f4c2734
---
.../blog/2020/08/27/pattern-match-beam-sql.html | 1 +
website/generated-content/blog/index.html | 4 +-
website/generated-content/blog/index.xml | 215 ++++++++++++++--
.../blog/pattern-match-beam-sql/index.html | 56 +++++
.../generated-content/categories/blog/index.xml | 206 +++++++++++++--
website/generated-content/categories/index.xml | 2 +-
website/generated-content/feed.xml | 278 ++++++++++++++-------
website/generated-content/index.html | 2 +-
website/generated-content/sitemap.xml | 2 +-
9 files changed, 633 insertions(+), 133 deletions(-)
diff --git a/website/generated-content/blog/2020/08/27/pattern-match-beam-sql.html b/website/generated-content/blog/2020/08/27/pattern-match-beam-sql.html
new file mode 100644
index 0000000..c938f7b
--- /dev/null
+++ b/website/generated-content/blog/2020/08/27/pattern-match-beam-sql.html
@@ -0,0 +1 @@
+<!doctype html><html><head><title>/blog/pattern-match-beam-sql/</title><link rel=canonical href=/blog/pattern-match-beam-sql/><meta name=robots content="noindex"><meta charset=utf-8><meta http-equiv=refresh content="0; url=/blog/pattern-match-beam-sql/"></head></html>
\ No newline at end of file
diff --git a/website/generated-content/blog/index.html b/website/generated-content/blog/index.html
index f43817f..0cffecb 100644
--- a/website/generated-content/blog/index.html
+++ b/website/generated-content/blog/index.html
@@ -1,7 +1,9 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Blogs</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languag [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+•
+Mark-Zeng</i></p>Introduction SQL is becoming increasingly powerful and useful in the field of data analysis.<p><a class="btn btn-default btn-sm" href=/blog/pattern-match-beam-sql/ role=button>Read more <span class="glyphicon glyphicon-menu-right" aria-hidden=true></span></a></p><hr><h3><a class=post-link href=/blog/python-improved-annotations/>Improved Annotation Support for the Python SDK</a></h3><p><i>Aug 21, 2020
•
Saavan Nanavati</i></p>The importance of static type checking in a dynamically typed language like Python is not up for debate.<p><a class="btn btn-default btn-sm" href=/blog/python-improved-annotations/ role=button>Read more <span class="glyphicon glyphicon-menu-right" aria-hidden=true></span></a></p><hr><h3><a class=post-link href=/blog/python-performance-runtime-type-checking/>Performance-Driven Runtime Type Checking for the Python SDK</a></h3><p><i>Aug 21, 2020
•
diff --git a/website/generated-content/blog/index.xml b/website/generated-content/blog/index.xml
index 89c39f1..bcab2a6 100644
--- a/website/generated-content/blog/index.xml
+++ b/website/generated-content/blog/index.xml
@@ -1,4 +1,192 @@
-<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Apache Beam – Blogs</title><link>/blog/</link><description>Recent content in Blogs on Apache Beam</description><generator>Hugo -- gohugo.io</generator><lastBuildDate>Fri, 21 Aug 2020 00:00:01 -0800</lastBuildDate><atom:link href="/blog/index.xml" rel="self" type="application/rss+xml"/><item><title>Blog: Improved Annotation Support for the Python SDK</title><link>/blog/python-improved-annotations/</link><pubDate>F [...]
+<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Apache Beam – Blogs</title><link>/blog/</link><description>Recent content in Blogs on Apache Beam</description><generator>Hugo -- gohugo.io</generator><lastBuildDate>Thu, 27 Aug 2020 00:00:01 +0800</lastBuildDate><atom:link href="/blog/index.xml" rel="self" type="application/rss+xml"/><item><title>Blog: Pattern Matching with Beam SQL</title><link>/blog/pattern-match-beam-sql/</link><pubDate>Thu, 27 Aug 2020 00:00 [...]
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h2 id="introduction">Introduction</h2>
+<p>SQL is becoming increasingly powerful and useful in the field of data analysis. MATCH_RECOGNIZE,
+a new SQL component introduced in 2016, brings extra analytical functionality. This project,
+as part of Google Summer of Code, aims to support basic MATCH_RECOGNIZE functionality. A basic MATCH_RECOGNIZE
+query would be something like this:
+<div class=language-sql>
+<div class="highlight"><pre class="chroma"><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">T</span><span class="p">.</span><span class="n">aid</span><span class="p">,</span> <span class="n">T</span><span class="p">.</span><span class="n">bid</span><span class="p">,</span> <span class="n">T</span><span class="p">.</span><span class="n">cid</span>
+<span class="k">FROM</span> <span class="n">MyTable</span>
+<span class="n">MATCH_RECOGNIZE</span> <span class="p">(</span>
+<span class="n">PARTITION</span> <span class="k">BY</span> <span class="n">userid</span>
+<span class="k">ORDER</span> <span class="k">BY</span> <span class="n">proctime</span>
+<span class="n">MEASURES</span>
+<span class="n">A</span><span class="p">.</span><span class="n">id</span> <span class="k">AS</span> <span class="n">aid</span><span class="p">,</span>
+<span class="n">B</span><span class="p">.</span><span class="n">id</span> <span class="k">AS</span> <span class="n">bid</span><span class="p">,</span>
+<span class="k">C</span><span class="p">.</span><span class="n">id</span> <span class="k">AS</span> <span class="n">cid</span>
+<span class="n">PATTERN</span> <span class="p">(</span><span class="n">A</span> <span class="n">B</span> <span class="k">C</span><span class="p">)</span>
+<span class="n">DEFINE</span>
+<span class="n">A</span> <span class="k">AS</span> <span class="n">name</span> <span class="o">=</span> <span class="s1">&#39;a&#39;</span><span class="p">,</span>
+<span class="n">B</span> <span class="k">AS</span> <span class="n">name</span> <span class="o">=</span> <span class="s1">&#39;b&#39;</span><span class="p">,</span>
+<span class="k">C</span> <span class="k">AS</span> <span class="n">name</span> <span class="o">=</span> <span class="s1">&#39;c&#39;</span>
+<span class="p">)</span> <span class="k">AS</span> <span class="n">T</span></code></pre></div>
+</div>
+</p>
+<p>The above query finds out ordered sets of events that have names &lsquo;a&rsquo;, &lsquo;b&rsquo; and &lsquo;c&rsquo;. Apart from this basic usage of
+MATCH_RECOGNIZE, I supported a few of other crucial features such as quantifiers and row pattern navigation. I will spell out
+the details in later sections.</p>
+<h2 id="approach--discussion">Approach &amp; Discussion</h2>
+<p>The implementation is strongly based on BEAM core transforms. Specifically, one MATCH_RECOGNIZE execution composes the
+following series of transforms:</p>
+<ol>
+<li>A <code>ParDo</code> transform and then a <code>GroupByKey</code> transform that build up the partitions (PARTITION BY).</li>
+<li>A <code>ParDo</code> transform that sorts within each partition (ORDER BY).</li>
+<li>A <code>ParDo</code> transform that applies pattern-match in each sorted partition.</li>
+</ol>
+<p>A pattern-match operation was first done with the java regex library. That is, I first transform rows within a partition into
+a string and then apply regex pattern-match routines. If a row satisfies a condition, then I output the corresponding pattern variable.
+This is ok under the assumption that the pattern definitions are mutually exclusive. That is, a pattern definition like <code>A AS A.price &gt; 0, B AS b.price &lt; 0</code> is allowed while
+a pattern definition like <code>A AS A.price &gt; 0, B AS B.proctime &gt; 0</code> might results in an incomplete match. For the latter case,
+an event can satisfy the conditions A and B at the same time. Mutually exclusive conditions gives deterministic pattern-match:
+each event can only belong to at most one pattern class.</p>
+<p>As specified in the SQL 2016 document, MATCH_RECOGNIZE defines a richer set of expression than regular expression. Specifically,
+it introduces <em>Row Pattern Navigation Operations</em> such as <code>PREV</code> and <code>NEXT</code>. This is perhaps one of the most intriguing feature of
+MATCH_RECOGNIZE. A regex library would no longer suffice the need since the pattern definition could be back-referencing (<code>PREV</code>) or
+forward-referencing (<code>NEXT</code>). So for the second version of implementation, we chose to use an NFA regex engine. An NFA brings more flexibility
+in terms of non-determinism (see Chapter 6 of SQL 2016 Part 5 for a more thorough discussion). My proposed NFA is based on a paper of UMASS.</p>
+<p>This is a working project. Many of the components are still not supported. I will list some unimplemented work in the section
+of future work.</p>
+<h2 id="usages">Usages</h2>
+<p>For now, the components I supported are:</p>
+<ul>
+<li>PARTITION BY</li>
+<li>ORDER BY</li>
+<li>MEASURES
+<ol>
+<li>LAST</li>
+<li>FIRST</li>
+</ol>
+</li>
+<li>ONE ROW PER MATCH/ALL ROWS PER MATCH</li>
+<li>DEFINE
+<ol>
+<li>Left side of the condition
+<ol>
+<li>LAST</li>
+</ol>
+</li>
+<li>Right side of the condition
+<ol>
+<li>PREV</li>
+</ol>
+</li>
+</ol>
+</li>
+<li>Quantifier
+<ol>
+<li>Kleene plus</li>
+</ol>
+</li>
+</ul>
+<p>The pattern definition evaluation is hard coded. To be more specific, it expects the column reference of the incoming row
+to be on the left side of a comparator. Additionally, PREV function can only appear on the right side of the comparator.</p>
+<p>With these limited tools, we could already write some slightly more complicated queries. Imagine we have the following
+table:</p>
+<table>
+<thead>
+<tr>
+<th align="center">transTime</th>
+<th align="center">price</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td align="center">1</td>
+<td align="center">3</td>
+</tr>
+<tr>
+<td align="center">2</td>
+<td align="center">2</td>
+</tr>
+<tr>
+<td align="center">3</td>
+<td align="center">1</td>
+</tr>
+<tr>
+<td align="center">4</td>
+<td align="center">5</td>
+</tr>
+<tr>
+<td align="center">5</td>
+<td align="center">6</td>
+</tr>
+</tbody>
+</table>
+<p>This table reflects the price changes of a product with respect to the transaction time. We could write the following
+query:</p>
+<div class=language-sql>
+<div class="highlight"><pre class="chroma"><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="o">*</span>
+<span class="k">FROM</span> <span class="n">MyTable</span>
+<span class="n">MATCH_RECOGNIZE</span> <span class="p">(</span>
+<span class="k">ORDER</span> <span class="k">BY</span> <span class="n">transTime</span>
+<span class="n">MEASURES</span>
+<span class="k">LAST</span><span class="p">(</span><span class="n">A</span><span class="p">.</span><span class="n">price</span><span class="p">)</span> <span class="k">AS</span> <span class="n">beforePrice</span><span class="p">,</span>
+<span class="k">FIRST</span><span class="p">(</span><span class="n">B</span><span class="p">.</span><span class="n">price</span><span class="p">)</span> <span class="k">AS</span> <span class="n">afterPrice</span>
+<span class="n">PATTERN</span> <span class="p">(</span><span class="n">A</span><span class="o">+</span> <span class="n">B</span><span class="o">+</span><span class="p">)</span>
+<span class="n">DEFINE</span>
+<span class="n">A</span> <span class="k">AS</span> <span class="n">price</span> <span class="o">&lt;</span> <span class="n">PREV</span><span class="p">(</span><span class="n">A</span><span class="p">.</span><span class="n">price</span><span class="p">),</span>
+<span class="n">B</span> <span class="k">AS</span> <span class="n">price</span> <span class="o">&gt;</span> <span class="n">PREV</span><span class="p">(</span><span class="n">B</span><span class="p">.</span><span class="n">price</span><span class="p">)</span>
+<span class="p">)</span> <span class="k">AS</span> <span class="n">T</span></code></pre></div>
+</div>
+<p>This will find the local minimum price and the price after it. For the example dataset, the first 3 rows will be
+mapped to A and the rest of the rows will be mapped to B. Thus, we will have (1, 5) as the result.</p>
+<blockquote>
+<p>Very important: For my NFA implementation, it slightly breaks the rule in the SQL standard. Since the buffered NFA
+only stores an event to the buffer if the event is a match to some pattern class, There would be no way to get the
+previous event back if the previous row is discarded. So the first row would always be a match (different from the standard)
+if PREV is used.</p>
+</blockquote>
+<h2 id="progress">Progress</h2>
+<ol>
+<li>PRs
+<ol>
+<li><a href="https://github.com/apache/beam/pull/12232">Support MATCH_RECOGNIZE using regex library</a> (merged)</li>
+<li><a href="https://github.com/apache/beam/pull/12532">Support MATCH_RECOGNIZE using NFA</a> (pending)</li>
+</ol>
+</li>
+<li>Commits
+<ol>
+<li>partition by: <a href="https://github.com/apache/beam/pull/12232/commits/064ada7257970bcb1d35530be1b88cb3830f242b">commit 064ada7</a></li>
+<li>order by: <a href="https://github.com/apache/beam/pull/12232/commits/9cd1a82bec7b2f7c44aacfbd72f5f775bb58b650">commit 9cd1a82</a></li>
+<li>regex pattern match: <a href="https://github.com/apache/beam/pull/12232/commits/8d6ffcc213e30999fc495c119b68da4f62fad258">commit 8d6ffcc</a></li>
+<li>support quantifiers: <a href="https://github.com/apache/beam/pull/12232/commits/f529b876a2c2e43d012c71b3a83ebd55eb16f4ff">commit f529b87</a></li>
+<li>measures: <a href="https://github.com/apache/beam/pull/12232/commits/87935746647611aa139d664ebed10c8e638bb024">commit 8793574</a></li>
+<li>added NFA implementation: <a href="https://github.com/apache/beam/pull/12532/commits/fc731f2b0699d11853e7b76da86456427d434a2a">commit fc731f2</a></li>
+<li>implemented functions PREV and LAST: <a href="https://github.com/apache/beam/pull/12532/commits/fc731f2b0699d11853e7b76da86456427d434a2a">commit 35323da</a></li>
+</ol>
+</li>
+</ol>
+<h2 id="future-work">Future Work</h2>
+<ul>
+<li>Support FINAL/RUNNING keywords.</li>
+<li>Support more quantifiers.</li>
+<li>Add optimization to the NFA.</li>
+<li>A better way to realize MATCH_RECOGNIZE might be having a Complex Event Processing library at BEAM core (rather than using BEAM transforms).</li>
+</ul>
+<!-- Related Documents:
+- proposal
+- design doc
+- SQL 2016 standard
+- UMASS NFA^b paper
+-->
+<h2 id="references">References</h2>
+<ul>
+<li><a href="https://drive.google.com/file/d/1ZuFZV4dCFVPZW_-RiqbU0w-vShaZh_jX/view?usp=sharing">Project Proposal</a></li>
+<li><a href="https://s.apache.org/beam-sql-pattern-recognization">Design Documentation</a></li>
+<li><a href="https://www.iso.org/standard/65143.html">SQL 2016 documentation Part 5</a></li>
+<li><a href="https://dl.acm.org/doi/10.1145/1376616.1376634">UMASS paper on NFA with shared buffer</a></li>
+</ul></description></item><item><title>Blog: Improved Annotation Support for the Python SDK</title><link>/blog/python-improved-annotations/</link><pubDate>Fri, 21 Aug 2020 00:00:01 -0800</pubDate><guid>/blog/python-improved-annotations/</guid><description>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -5157,27 +5345,4 @@ anticipated, perhaps one every 1-2 months.</p>
the developer experience will be our focus for the next several months. If you
have any comments or discover any issues, I’d like to invite you to reach out
to us via <a href="/get-started/support/">user’s mailing list</a> or the
-<a href="https://issues.apache.org/jira/browse/BEAM/">Apache JIRA issue tracker</a>.</p></description></item><item><title>Blog: How We Added Windowing to the Apache Flink Batch Runner</title><link>/blog/flink-batch-runner-milestone/</link><pubDate>Mon, 13 Jun 2016 09:00:00 -0700</pubDate><guid>/blog/flink-batch-runner-milestone/</guid><description>
-<!--
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-http://www.apache.org/licenses/LICENSE-2.0
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
--->
-<p>We recently achieved a major milestone by adding support for windowing to the <a href="https://flink.apache.org">Apache Flink</a> Batch runner. In this post we would like to explain what this means for users of Apache Beam and highlight some of the implementation details.</p>
-<p>Before we start, though, let’s quickly talk about the execution of Beam programs and how this is relevant to today’s post. A Beam pipeline can contain bounded and unbounded sources. If the pipeline only contains bounded sources it can be executed in a batch fashion, if it contains some unbounded sources it must be executed in a streaming fashion. When executing a Beam pipeline on Flink, you don’t have to choose the execution mode. Internally, the Flink runner either translates the [...]
-<h2 id="what-does-this-mean-for-users">What does this mean for users?</h2>
-<p>Support for windowing was the last missing puzzle piece for making the Flink Batch runner compatible with the Beam model. With the latest change to the Batch runner users can now run any pipeline that only contains bounded sources and be certain that the results match those of the original reference-implementation runners that were provided by Google as part of the initial code drop coming from the Google Dataflow SDK.</p>
-<p>The most obvious part of the change is that windows can now be assigned to elements and that the runner respects these windows for the <code>GroupByKey</code> and <code>Combine</code> operations. A not-so-obvious change concerns side-inputs. In the Beam model, side inputs respect windows; when a value of the main input is being processed only the side input that corresponds to the correct window is available to the processing function, the <code>DoFn</code>.</p>
-<p>Getting side-input semantics right is an important milestone in it’s own because it allows to use a big suite of unit tests for verifying the correctness of a runner implementation. These tests exercise every obscure detail of the Beam programming model and verify that the results produced by a runner match what you would expect from a correct implementation. In the suite, side inputs are used to compare the expected result to the actual result. With these tests being executed regu [...]
-<h2 id="under-the-hood">Under the Hood</h2>
-<p>The basis for the changes is the introduction of <code>WindowedValue</code> in the generated Flink transformations. Before, a Beam <code>PCollection&lt;T&gt;</code> would be transformed to a <code>DataSet&lt;T&gt;</code>. Now, we instead create a <code>DataSet&lt;WindowedValue&lt;T&gt;&gt;</code>. The <code>WindowedValue&lt;T&gt;</code> stores meta data about the value, such as the timestamp and the windows to wh [...]
-<p>With this basic change out of the way we just had to make sure that windows were respected for side inputs and that <code>Combine</code> and <code>GroupByKey</code> correctly handled windows. The tricky part there is the handling of merging windows such as session windows. For these we essentially emulate the behavior of a merging <code>WindowFn</code> in our own code.</p>
-<p>After we got side inputs working we could enable the aforementioned suite of tests to check how well the runner behaves with respect to the Beam model. As can be expected there were quite some discrepancies but we managed to resolve them all. In the process, we also slimmed down the runner implementation. For example, we removed all custom translations for sources and sinks and are now relying only on Beam code for these, thereby greatly reducing the maintenance overhead.</p>
-<h2 id="summary">Summary</h2>
-<p>We reached a major milestone in adding windowing support to the Flink Batch runner, thereby making it compatible with the Beam model. Because of the large suite of tests that can now be executed on the runner we are also confident about the correctness of the implementation and about it staying that way in the future.</p></description></item></channel></rss>
\ No newline at end of file
+<a href="https://issues.apache.org/jira/browse/BEAM/">Apache JIRA issue tracker</a>.</p></description></item></channel></rss>
\ No newline at end of file
diff --git a/website/generated-content/blog/pattern-match-beam-sql/index.html b/website/generated-content/blog/pattern-match-beam-sql/index.html
new file mode 100644
index 0000000..6e12b2f
--- /dev/null
+++ b/website/generated-content/blog/pattern-match-beam-sql/index.html
@@ -0,0 +1,56 @@
+<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Pattern Matching with Beam SQL</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) an [...]
+<span class=sr-only>Toggle navigation</span>
+<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+•</p></header><div class=post-content itemprop=articleBody><h2 id=introduction>Introduction</h2><p>SQL is becoming increasingly powerful and useful in the field of data analysis. MATCH_RECOGNIZE,
+a new SQL component introduced in 2016, brings extra analytical functionality. This project,
+as part of Google Summer of Code, aims to support basic MATCH_RECOGNIZE functionality. A basic MATCH_RECOGNIZE
+query would be something like this:<div class=language-sql><div class=highlight><pre class=chroma><code class=language-sql data-lang=sql><span class=k>SELECT</span> <span class=n>T</span><span class=p>.</span><span class=n>aid</span><span class=p>,</span> <span class=n>T</span><span class=p>.</span><span class=n>bid</span><span class=p>,</span> <span class=n>T</span><span class=p>.</span><span class=n>cid</span>
+<span class=k>FROM</span> <span class=n>MyTable</span>
+ <span class=n>MATCH_RECOGNIZE</span> <span class=p>(</span>
+ <span class=n>PARTITION</span> <span class=k>BY</span> <span class=n>userid</span>
+ <span class=k>ORDER</span> <span class=k>BY</span> <span class=n>proctime</span>
+ <span class=n>MEASURES</span>
+ <span class=n>A</span><span class=p>.</span><span class=n>id</span> <span class=k>AS</span> <span class=n>aid</span><span class=p>,</span>
+ <span class=n>B</span><span class=p>.</span><span class=n>id</span> <span class=k>AS</span> <span class=n>bid</span><span class=p>,</span>
+ <span class=k>C</span><span class=p>.</span><span class=n>id</span> <span class=k>AS</span> <span class=n>cid</span>
+ <span class=n>PATTERN</span> <span class=p>(</span><span class=n>A</span> <span class=n>B</span> <span class=k>C</span><span class=p>)</span>
+ <span class=n>DEFINE</span>
+ <span class=n>A</span> <span class=k>AS</span> <span class=n>name</span> <span class=o>=</span> <span class=s1>'a'</span><span class=p>,</span>
+ <span class=n>B</span> <span class=k>AS</span> <span class=n>name</span> <span class=o>=</span> <span class=s1>'b'</span><span class=p>,</span>
+ <span class=k>C</span> <span class=k>AS</span> <span class=n>name</span> <span class=o>=</span> <span class=s1>'c'</span>
+ <span class=p>)</span> <span class=k>AS</span> <span class=n>T</span></code></pre></div></div></p><p>The above query finds out ordered sets of events that have names ‘a’, ‘b’ and ‘c’. Apart from this basic usage of
+MATCH_RECOGNIZE, I supported a few of other crucial features such as quantifiers and row pattern navigation. I will spell out
+the details in later sections.</p><h2 id=approach--discussion>Approach & Discussion</h2><p>The implementation is strongly based on BEAM core transforms. Specifically, one MATCH_RECOGNIZE execution composes the
+following series of transforms:</p><ol><li>A <code>ParDo</code> transform and then a <code>GroupByKey</code> transform that build up the partitions (PARTITION BY).</li><li>A <code>ParDo</code> transform that sorts within each partition (ORDER BY).</li><li>A <code>ParDo</code> transform that applies pattern-match in each sorted partition.</li></ol><p>A pattern-match operation was first done with the java regex library. That is, I first transform rows within a partition into
+a string and then apply regex pattern-match routines. If a row satisfies a condition, then I output the corresponding pattern variable.
+This is ok under the assumption that the pattern definitions are mutually exclusive. That is, a pattern definition like <code>A AS A.price > 0, B AS b.price < 0</code> is allowed while
+a pattern definition like <code>A AS A.price > 0, B AS B.proctime > 0</code> might results in an incomplete match. For the latter case,
+an event can satisfy the conditions A and B at the same time. Mutually exclusive conditions gives deterministic pattern-match:
+each event can only belong to at most one pattern class.</p><p>As specified in the SQL 2016 document, MATCH_RECOGNIZE defines a richer set of expression than regular expression. Specifically,
+it introduces <em>Row Pattern Navigation Operations</em> such as <code>PREV</code> and <code>NEXT</code>. This is perhaps one of the most intriguing feature of
+MATCH_RECOGNIZE. A regex library would no longer suffice the need since the pattern definition could be back-referencing (<code>PREV</code>) or
+forward-referencing (<code>NEXT</code>). So for the second version of implementation, we chose to use an NFA regex engine. An NFA brings more flexibility
+in terms of non-determinism (see Chapter 6 of SQL 2016 Part 5 for a more thorough discussion). My proposed NFA is based on a paper of UMASS.</p><p>This is a working project. Many of the components are still not supported. I will list some unimplemented work in the section
+of future work.</p><h2 id=usages>Usages</h2><p>For now, the components I supported are:</p><ul><li>PARTITION BY</li><li>ORDER BY</li><li>MEASURES<ol><li>LAST</li><li>FIRST</li></ol></li><li>ONE ROW PER MATCH/ALL ROWS PER MATCH</li><li>DEFINE<ol><li>Left side of the condition<ol><li>LAST</li></ol></li><li>Right side of the condition<ol><li>PREV</li></ol></li></ol></li><li>Quantifier<ol><li>Kleene plus</li></ol></li></ul><p>The pattern definition evaluation is hard coded. To be more specif [...]
+to be on the left side of a comparator. Additionally, PREV function can only appear on the right side of the comparator.</p><p>With these limited tools, we could already write some slightly more complicated queries. Imagine we have the following
+table:</p><table><thead><tr><th align=center>transTime</th><th align=center>price</th></tr></thead><tbody><tr><td align=center>1</td><td align=center>3</td></tr><tr><td align=center>2</td><td align=center>2</td></tr><tr><td align=center>3</td><td align=center>1</td></tr><tr><td align=center>4</td><td align=center>5</td></tr><tr><td align=center>5</td><td align=center>6</td></tr></tbody></table><p>This table reflects the price changes of a product with respect to the transaction time. We [...]
+query:</p><div class=language-sql><div class=highlight><pre class=chroma><code class=language-sql data-lang=sql><span class=k>SELECT</span> <span class=o>*</span>
+<span class=k>FROM</span> <span class=n>MyTable</span>
+ <span class=n>MATCH_RECOGNIZE</span> <span class=p>(</span>
+ <span class=k>ORDER</span> <span class=k>BY</span> <span class=n>transTime</span>
+ <span class=n>MEASURES</span>
+ <span class=k>LAST</span><span class=p>(</span><span class=n>A</span><span class=p>.</span><span class=n>price</span><span class=p>)</span> <span class=k>AS</span> <span class=n>beforePrice</span><span class=p>,</span>
+ <span class=k>FIRST</span><span class=p>(</span><span class=n>B</span><span class=p>.</span><span class=n>price</span><span class=p>)</span> <span class=k>AS</span> <span class=n>afterPrice</span>
+ <span class=n>PATTERN</span> <span class=p>(</span><span class=n>A</span><span class=o>+</span> <span class=n>B</span><span class=o>+</span><span class=p>)</span>
+ <span class=n>DEFINE</span>
+ <span class=n>A</span> <span class=k>AS</span> <span class=n>price</span> <span class=o><</span> <span class=n>PREV</span><span class=p>(</span><span class=n>A</span><span class=p>.</span><span class=n>price</span><span class=p>),</span>
+ <span class=n>B</span> <span class=k>AS</span> <span class=n>price</span> <span class=o>></span> <span class=n>PREV</span><span class=p>(</span><span class=n>B</span><span class=p>.</span><span class=n>price</span><span class=p>)</span>
+ <span class=p>)</span> <span class=k>AS</span> <span class=n>T</span></code></pre></div></div><p>This will find the local minimum price and the price after it. For the example dataset, the first 3 rows will be
+mapped to A and the rest of the rows will be mapped to B. Thus, we will have (1, 5) as the result.</p><blockquote><p>Very important: For my NFA implementation, it slightly breaks the rule in the SQL standard. Since the buffered NFA
+only stores an event to the buffer if the event is a match to some pattern class, There would be no way to get the
+previous event back if the previous row is discarded. So the first row would always be a match (different from the standard)
+if PREV is used.</p></blockquote><h2 id=progress>Progress</h2><ol><li>PRs<ol><li><a href=https://github.com/apache/beam/pull/12232>Support MATCH_RECOGNIZE using regex library</a> (merged)</li><li><a href=https://github.com/apache/beam/pull/12532>Support MATCH_RECOGNIZE using NFA</a> (pending)</li></ol></li><li>Commits<ol><li>partition by: <a href=https://github.com/apache/beam/pull/12232/commits/064ada7257970bcb1d35530be1b88cb3830f242b>commit 064ada7</a></li><li>order by: <a href=https:/ [...]
+<a href=http://www.apache.org>The Apache Software Foundation</a>
+| <a href=/privacy_policy>Privacy Policy</a>
+| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/categories/blog/index.xml b/website/generated-content/categories/blog/index.xml
index b424dd2..f615297 100644
--- a/website/generated-content/categories/blog/index.xml
+++ b/website/generated-content/categories/blog/index.xml
@@ -1,4 +1,192 @@
-<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Apache Beam – blog</title><link>/categories/blog/</link><description>Recent content in blog on Apache Beam</description><generator>Hugo -- gohugo.io</generator><lastBuildDate>Fri, 21 Aug 2020 00:00:01 -0800</lastBuildDate><atom:link href="/categories/blog/index.xml" rel="self" type="application/rss+xml"/><item><title>Blog: Improved Annotation Support for the Python SDK</title><link>/blog/python-improved-annotatio [...]
+<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Apache Beam – blog</title><link>/categories/blog/</link><description>Recent content in blog on Apache Beam</description><generator>Hugo -- gohugo.io</generator><lastBuildDate>Thu, 27 Aug 2020 00:00:01 +0800</lastBuildDate><atom:link href="/categories/blog/index.xml" rel="self" type="application/rss+xml"/><item><title>Blog: Pattern Matching with Beam SQL</title><link>/blog/pattern-match-beam-sql/</link><pubDate>Th [...]
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h2 id="introduction">Introduction</h2>
+<p>SQL is becoming increasingly powerful and useful in the field of data analysis. MATCH_RECOGNIZE,
+a new SQL component introduced in 2016, brings extra analytical functionality. This project,
+as part of Google Summer of Code, aims to support basic MATCH_RECOGNIZE functionality. A basic MATCH_RECOGNIZE
+query would be something like this:
+<div class=language-sql>
+<div class="highlight"><pre class="chroma"><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">T</span><span class="p">.</span><span class="n">aid</span><span class="p">,</span> <span class="n">T</span><span class="p">.</span><span class="n">bid</span><span class="p">,</span> <span class="n">T</span><span class="p">.</span><span class="n">cid</span>
+<span class="k">FROM</span> <span class="n">MyTable</span>
+<span class="n">MATCH_RECOGNIZE</span> <span class="p">(</span>
+<span class="n">PARTITION</span> <span class="k">BY</span> <span class="n">userid</span>
+<span class="k">ORDER</span> <span class="k">BY</span> <span class="n">proctime</span>
+<span class="n">MEASURES</span>
+<span class="n">A</span><span class="p">.</span><span class="n">id</span> <span class="k">AS</span> <span class="n">aid</span><span class="p">,</span>
+<span class="n">B</span><span class="p">.</span><span class="n">id</span> <span class="k">AS</span> <span class="n">bid</span><span class="p">,</span>
+<span class="k">C</span><span class="p">.</span><span class="n">id</span> <span class="k">AS</span> <span class="n">cid</span>
+<span class="n">PATTERN</span> <span class="p">(</span><span class="n">A</span> <span class="n">B</span> <span class="k">C</span><span class="p">)</span>
+<span class="n">DEFINE</span>
+<span class="n">A</span> <span class="k">AS</span> <span class="n">name</span> <span class="o">=</span> <span class="s1">&#39;a&#39;</span><span class="p">,</span>
+<span class="n">B</span> <span class="k">AS</span> <span class="n">name</span> <span class="o">=</span> <span class="s1">&#39;b&#39;</span><span class="p">,</span>
+<span class="k">C</span> <span class="k">AS</span> <span class="n">name</span> <span class="o">=</span> <span class="s1">&#39;c&#39;</span>
+<span class="p">)</span> <span class="k">AS</span> <span class="n">T</span></code></pre></div>
+</div>
+</p>
+<p>The above query finds out ordered sets of events that have names &lsquo;a&rsquo;, &lsquo;b&rsquo; and &lsquo;c&rsquo;. Apart from this basic usage of
+MATCH_RECOGNIZE, I supported a few of other crucial features such as quantifiers and row pattern navigation. I will spell out
+the details in later sections.</p>
+<h2 id="approach--discussion">Approach &amp; Discussion</h2>
+<p>The implementation is strongly based on BEAM core transforms. Specifically, one MATCH_RECOGNIZE execution composes the
+following series of transforms:</p>
+<ol>
+<li>A <code>ParDo</code> transform and then a <code>GroupByKey</code> transform that build up the partitions (PARTITION BY).</li>
+<li>A <code>ParDo</code> transform that sorts within each partition (ORDER BY).</li>
+<li>A <code>ParDo</code> transform that applies pattern-match in each sorted partition.</li>
+</ol>
+<p>A pattern-match operation was first done with the java regex library. That is, I first transform rows within a partition into
+a string and then apply regex pattern-match routines. If a row satisfies a condition, then I output the corresponding pattern variable.
+This is ok under the assumption that the pattern definitions are mutually exclusive. That is, a pattern definition like <code>A AS A.price &gt; 0, B AS b.price &lt; 0</code> is allowed while
+a pattern definition like <code>A AS A.price &gt; 0, B AS B.proctime &gt; 0</code> might results in an incomplete match. For the latter case,
+an event can satisfy the conditions A and B at the same time. Mutually exclusive conditions gives deterministic pattern-match:
+each event can only belong to at most one pattern class.</p>
+<p>As specified in the SQL 2016 document, MATCH_RECOGNIZE defines a richer set of expression than regular expression. Specifically,
+it introduces <em>Row Pattern Navigation Operations</em> such as <code>PREV</code> and <code>NEXT</code>. This is perhaps one of the most intriguing feature of
+MATCH_RECOGNIZE. A regex library would no longer suffice the need since the pattern definition could be back-referencing (<code>PREV</code>) or
+forward-referencing (<code>NEXT</code>). So for the second version of implementation, we chose to use an NFA regex engine. An NFA brings more flexibility
+in terms of non-determinism (see Chapter 6 of SQL 2016 Part 5 for a more thorough discussion). My proposed NFA is based on a paper of UMASS.</p>
+<p>This is a working project. Many of the components are still not supported. I will list some unimplemented work in the section
+of future work.</p>
+<h2 id="usages">Usages</h2>
+<p>For now, the components I supported are:</p>
+<ul>
+<li>PARTITION BY</li>
+<li>ORDER BY</li>
+<li>MEASURES
+<ol>
+<li>LAST</li>
+<li>FIRST</li>
+</ol>
+</li>
+<li>ONE ROW PER MATCH/ALL ROWS PER MATCH</li>
+<li>DEFINE
+<ol>
+<li>Left side of the condition
+<ol>
+<li>LAST</li>
+</ol>
+</li>
+<li>Right side of the condition
+<ol>
+<li>PREV</li>
+</ol>
+</li>
+</ol>
+</li>
+<li>Quantifier
+<ol>
+<li>Kleene plus</li>
+</ol>
+</li>
+</ul>
+<p>The pattern definition evaluation is hard coded. To be more specific, it expects the column reference of the incoming row
+to be on the left side of a comparator. Additionally, PREV function can only appear on the right side of the comparator.</p>
+<p>With these limited tools, we could already write some slightly more complicated queries. Imagine we have the following
+table:</p>
+<table>
+<thead>
+<tr>
+<th align="center">transTime</th>
+<th align="center">price</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td align="center">1</td>
+<td align="center">3</td>
+</tr>
+<tr>
+<td align="center">2</td>
+<td align="center">2</td>
+</tr>
+<tr>
+<td align="center">3</td>
+<td align="center">1</td>
+</tr>
+<tr>
+<td align="center">4</td>
+<td align="center">5</td>
+</tr>
+<tr>
+<td align="center">5</td>
+<td align="center">6</td>
+</tr>
+</tbody>
+</table>
+<p>This table reflects the price changes of a product with respect to the transaction time. We could write the following
+query:</p>
+<div class=language-sql>
+<div class="highlight"><pre class="chroma"><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="o">*</span>
+<span class="k">FROM</span> <span class="n">MyTable</span>
+<span class="n">MATCH_RECOGNIZE</span> <span class="p">(</span>
+<span class="k">ORDER</span> <span class="k">BY</span> <span class="n">transTime</span>
+<span class="n">MEASURES</span>
+<span class="k">LAST</span><span class="p">(</span><span class="n">A</span><span class="p">.</span><span class="n">price</span><span class="p">)</span> <span class="k">AS</span> <span class="n">beforePrice</span><span class="p">,</span>
+<span class="k">FIRST</span><span class="p">(</span><span class="n">B</span><span class="p">.</span><span class="n">price</span><span class="p">)</span> <span class="k">AS</span> <span class="n">afterPrice</span>
+<span class="n">PATTERN</span> <span class="p">(</span><span class="n">A</span><span class="o">+</span> <span class="n">B</span><span class="o">+</span><span class="p">)</span>
+<span class="n">DEFINE</span>
+<span class="n">A</span> <span class="k">AS</span> <span class="n">price</span> <span class="o">&lt;</span> <span class="n">PREV</span><span class="p">(</span><span class="n">A</span><span class="p">.</span><span class="n">price</span><span class="p">),</span>
+<span class="n">B</span> <span class="k">AS</span> <span class="n">price</span> <span class="o">&gt;</span> <span class="n">PREV</span><span class="p">(</span><span class="n">B</span><span class="p">.</span><span class="n">price</span><span class="p">)</span>
+<span class="p">)</span> <span class="k">AS</span> <span class="n">T</span></code></pre></div>
+</div>
+<p>This will find the local minimum price and the price after it. For the example dataset, the first 3 rows will be
+mapped to A and the rest of the rows will be mapped to B. Thus, we will have (1, 5) as the result.</p>
+<blockquote>
+<p>Very important: For my NFA implementation, it slightly breaks the rule in the SQL standard. Since the buffered NFA
+only stores an event to the buffer if the event is a match to some pattern class, There would be no way to get the
+previous event back if the previous row is discarded. So the first row would always be a match (different from the standard)
+if PREV is used.</p>
+</blockquote>
+<h2 id="progress">Progress</h2>
+<ol>
+<li>PRs
+<ol>
+<li><a href="https://github.com/apache/beam/pull/12232">Support MATCH_RECOGNIZE using regex library</a> (merged)</li>
+<li><a href="https://github.com/apache/beam/pull/12532">Support MATCH_RECOGNIZE using NFA</a> (pending)</li>
+</ol>
+</li>
+<li>Commits
+<ol>
+<li>partition by: <a href="https://github.com/apache/beam/pull/12232/commits/064ada7257970bcb1d35530be1b88cb3830f242b">commit 064ada7</a></li>
+<li>order by: <a href="https://github.com/apache/beam/pull/12232/commits/9cd1a82bec7b2f7c44aacfbd72f5f775bb58b650">commit 9cd1a82</a></li>
+<li>regex pattern match: <a href="https://github.com/apache/beam/pull/12232/commits/8d6ffcc213e30999fc495c119b68da4f62fad258">commit 8d6ffcc</a></li>
+<li>support quantifiers: <a href="https://github.com/apache/beam/pull/12232/commits/f529b876a2c2e43d012c71b3a83ebd55eb16f4ff">commit f529b87</a></li>
+<li>measures: <a href="https://github.com/apache/beam/pull/12232/commits/87935746647611aa139d664ebed10c8e638bb024">commit 8793574</a></li>
+<li>added NFA implementation: <a href="https://github.com/apache/beam/pull/12532/commits/fc731f2b0699d11853e7b76da86456427d434a2a">commit fc731f2</a></li>
+<li>implemented functions PREV and LAST: <a href="https://github.com/apache/beam/pull/12532/commits/fc731f2b0699d11853e7b76da86456427d434a2a">commit 35323da</a></li>
+</ol>
+</li>
+</ol>
+<h2 id="future-work">Future Work</h2>
+<ul>
+<li>Support FINAL/RUNNING keywords.</li>
+<li>Support more quantifiers.</li>
+<li>Add optimization to the NFA.</li>
+<li>A better way to realize MATCH_RECOGNIZE might be having a Complex Event Processing library at BEAM core (rather than using BEAM transforms).</li>
+</ul>
+<!-- Related Documents:
+- proposal
+- design doc
+- SQL 2016 standard
+- UMASS NFA^b paper
+-->
+<h2 id="references">References</h2>
+<ul>
+<li><a href="https://drive.google.com/file/d/1ZuFZV4dCFVPZW_-RiqbU0w-vShaZh_jX/view?usp=sharing">Project Proposal</a></li>
+<li><a href="https://s.apache.org/beam-sql-pattern-recognization">Design Documentation</a></li>
+<li><a href="https://www.iso.org/standard/65143.html">SQL 2016 documentation Part 5</a></li>
+<li><a href="https://dl.acm.org/doi/10.1145/1376616.1376634">UMASS paper on NFA with shared buffer</a></li>
+</ul></description></item><item><title>Blog: Improved Annotation Support for the Python SDK</title><link>/blog/python-improved-annotations/</link><pubDate>Fri, 21 Aug 2020 00:00:01 -0800</pubDate><guid>/blog/python-improved-annotations/</guid><description>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -5200,18 +5388,4 @@ PCollection&lt;O&gt; output = input
<i>Three different visualizations of a simple WordCount pipeline which computes the number of occurrences of every word in a set of text files. The flat view gives the full DAG of all operations performed. The execution view groups operations according to how they're executed, e.g. after performing runner-specific optimizations like function composition. The structured view nests operations according to their grouping in PTransforms.</i>
</div>
<h2 id="summary">Summary</h2>
-<p>Although it&rsquo;s tempting to add methods to PCollections, such an approach is not scalable, extensible, or sufficiently expressive. Putting a single apply method on PCollection and all the logic into the operation itself lets us have the best of both worlds, and avoids hard cliffs of complexity by having a single consistent style across simple and complex pipelines, and between predefined and user-defined operations.</p></description></item><item><title>Blog: Dynamic work [...]
-<!--
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-http://www.apache.org/licenses/LICENSE-2.0
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
--->
-<p>This morning, Eugene and Malo from the Google Cloud Dataflow team posted <a href="https://cloud.google.com/blog/big-data/2016/05/no-shard-left-behind-dynamic-work-rebalancing-in-google-cloud-dataflow"><em>No shard left behind: dynamic work rebalancing in Google Cloud Dataflow</em></a>. This article discusses Cloud Dataflow’s solution to the well-known straggler problem.</p>
-<p>In a large batch processing job with many tasks executing in parallel, some of the tasks &ndash; the stragglers &ndash; can take a much longer time to complete than others, perhaps due to imperfect splitting of the work into parallel chunks when issuing the job. Typically, waiting for stragglers means that the overall job completes later than it should, and may also reserve too many machines that may be underutilized at the end. Cloud Dataflow’s dynamic work rebalancing can [...]
-<p>What I’d like to highlight for the Apache Beam (incubating) community is that Cloud Dataflow’s dynamic work rebalancing is implemented using <em>runner-specific</em> control logic on top of Beam’s <em>runner-independent</em> <a href="https://github.com/apache/beam/blob/9fa97fb2491bc784df53fb0f044409dbbc2af3d7/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BoundedSource.java"><code>BoundedSource API</code></a>. Specifically, to steal work from a straggle [...]
\ No newline at end of file
+<p>Although it&rsquo;s tempting to add methods to PCollections, such an approach is not scalable, extensible, or sufficiently expressive. Putting a single apply method on PCollection and all the logic into the operation itself lets us have the best of both worlds, and avoids hard cliffs of complexity by having a single consistent style across simple and complex pipelines, and between predefined and user-defined operations.</p></description></item></channel></rss>
\ No newline at end of file
diff --git a/website/generated-content/categories/index.xml b/website/generated-content/categories/index.xml
index b9555ba..5ac594b 100644
--- a/website/generated-content/categories/index.xml
+++ b/website/generated-content/categories/index.xml
@@ -1 +1 @@
-<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Apache Beam – Categories</title><link>/categories/</link><description>Recent content in Categories on Apache Beam</description><generator>Hugo -- gohugo.io</generator><lastBuildDate>Fri, 21 Aug 2020 00:00:01 -0800</lastBuildDate><atom:link href="/categories/index.xml" rel="self" type="application/rss+xml"/></channel></rss>
\ No newline at end of file
+<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Apache Beam – Categories</title><link>/categories/</link><description>Recent content in Categories on Apache Beam</description><generator>Hugo -- gohugo.io</generator><lastBuildDate>Thu, 27 Aug 2020 00:00:01 +0800</lastBuildDate><atom:link href="/categories/index.xml" rel="self" type="application/rss+xml"/></channel></rss>
\ No newline at end of file
diff --git a/website/generated-content/feed.xml b/website/generated-content/feed.xml
index 18053c6..6125c7b 100644
--- a/website/generated-content/feed.xml
+++ b/website/generated-content/feed.xml
@@ -1,4 +1,192 @@
-<rss xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title>Apache Beam</title><description>Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number [...]
+<rss xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title>Apache Beam</title><description>Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number [...]
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<h2 id="introduction">Introduction</h2>
+<p>SQL is becoming increasingly powerful and useful in the field of data analysis. MATCH_RECOGNIZE,
+a new SQL component introduced in 2016, brings extra analytical functionality. This project,
+as part of Google Summer of Code, aims to support basic MATCH_RECOGNIZE functionality. A basic MATCH_RECOGNIZE
+query would be something like this:
+<div class=language-sql>
+<div class="highlight"><pre class="chroma"><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">T</span><span class="p">.</span><span class="n">aid</span><span class="p">,</span> <span class="n">T</span><span class="p">.</span><span class="n">bid</span><span class="p">,</span> <span class="n">T</span><span class="p">.</span><span class="n">cid</span>
+<span class="k">FROM</span> <span class="n">MyTable</span>
+<span class="n">MATCH_RECOGNIZE</span> <span class="p">(</span>
+<span class="n">PARTITION</span> <span class="k">BY</span> <span class="n">userid</span>
+<span class="k">ORDER</span> <span class="k">BY</span> <span class="n">proctime</span>
+<span class="n">MEASURES</span>
+<span class="n">A</span><span class="p">.</span><span class="n">id</span> <span class="k">AS</span> <span class="n">aid</span><span class="p">,</span>
+<span class="n">B</span><span class="p">.</span><span class="n">id</span> <span class="k">AS</span> <span class="n">bid</span><span class="p">,</span>
+<span class="k">C</span><span class="p">.</span><span class="n">id</span> <span class="k">AS</span> <span class="n">cid</span>
+<span class="n">PATTERN</span> <span class="p">(</span><span class="n">A</span> <span class="n">B</span> <span class="k">C</span><span class="p">)</span>
+<span class="n">DEFINE</span>
+<span class="n">A</span> <span class="k">AS</span> <span class="n">name</span> <span class="o">=</span> <span class="s1">&#39;a&#39;</span><span class="p">,</span>
+<span class="n">B</span> <span class="k">AS</span> <span class="n">name</span> <span class="o">=</span> <span class="s1">&#39;b&#39;</span><span class="p">,</span>
+<span class="k">C</span> <span class="k">AS</span> <span class="n">name</span> <span class="o">=</span> <span class="s1">&#39;c&#39;</span>
+<span class="p">)</span> <span class="k">AS</span> <span class="n">T</span></code></pre></div>
+</div>
+</p>
+<p>The above query finds out ordered sets of events that have names &lsquo;a&rsquo;, &lsquo;b&rsquo; and &lsquo;c&rsquo;. Apart from this basic usage of
+MATCH_RECOGNIZE, I supported a few of other crucial features such as quantifiers and row pattern navigation. I will spell out
+the details in later sections.</p>
+<h2 id="approach--discussion">Approach &amp; Discussion</h2>
+<p>The implementation is strongly based on BEAM core transforms. Specifically, one MATCH_RECOGNIZE execution composes the
+following series of transforms:</p>
+<ol>
+<li>A <code>ParDo</code> transform and then a <code>GroupByKey</code> transform that build up the partitions (PARTITION BY).</li>
+<li>A <code>ParDo</code> transform that sorts within each partition (ORDER BY).</li>
+<li>A <code>ParDo</code> transform that applies pattern-match in each sorted partition.</li>
+</ol>
+<p>A pattern-match operation was first done with the java regex library. That is, I first transform rows within a partition into
+a string and then apply regex pattern-match routines. If a row satisfies a condition, then I output the corresponding pattern variable.
+This is ok under the assumption that the pattern definitions are mutually exclusive. That is, a pattern definition like <code>A AS A.price &gt; 0, B AS b.price &lt; 0</code> is allowed while
+a pattern definition like <code>A AS A.price &gt; 0, B AS B.proctime &gt; 0</code> might results in an incomplete match. For the latter case,
+an event can satisfy the conditions A and B at the same time. Mutually exclusive conditions gives deterministic pattern-match:
+each event can only belong to at most one pattern class.</p>
+<p>As specified in the SQL 2016 document, MATCH_RECOGNIZE defines a richer set of expression than regular expression. Specifically,
+it introduces <em>Row Pattern Navigation Operations</em> such as <code>PREV</code> and <code>NEXT</code>. This is perhaps one of the most intriguing feature of
+MATCH_RECOGNIZE. A regex library would no longer suffice the need since the pattern definition could be back-referencing (<code>PREV</code>) or
+forward-referencing (<code>NEXT</code>). So for the second version of implementation, we chose to use an NFA regex engine. An NFA brings more flexibility
+in terms of non-determinism (see Chapter 6 of SQL 2016 Part 5 for a more thorough discussion). My proposed NFA is based on a paper of UMASS.</p>
+<p>This is a working project. Many of the components are still not supported. I will list some unimplemented work in the section
+of future work.</p>
+<h2 id="usages">Usages</h2>
+<p>For now, the components I supported are:</p>
+<ul>
+<li>PARTITION BY</li>
+<li>ORDER BY</li>
+<li>MEASURES
+<ol>
+<li>LAST</li>
+<li>FIRST</li>
+</ol>
+</li>
+<li>ONE ROW PER MATCH/ALL ROWS PER MATCH</li>
+<li>DEFINE
+<ol>
+<li>Left side of the condition
+<ol>
+<li>LAST</li>
+</ol>
+</li>
+<li>Right side of the condition
+<ol>
+<li>PREV</li>
+</ol>
+</li>
+</ol>
+</li>
+<li>Quantifier
+<ol>
+<li>Kleene plus</li>
+</ol>
+</li>
+</ul>
+<p>The pattern definition evaluation is hard coded. To be more specific, it expects the column reference of the incoming row
+to be on the left side of a comparator. Additionally, PREV function can only appear on the right side of the comparator.</p>
+<p>With these limited tools, we could already write some slightly more complicated queries. Imagine we have the following
+table:</p>
+<table>
+<thead>
+<tr>
+<th align="center">transTime</th>
+<th align="center">price</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td align="center">1</td>
+<td align="center">3</td>
+</tr>
+<tr>
+<td align="center">2</td>
+<td align="center">2</td>
+</tr>
+<tr>
+<td align="center">3</td>
+<td align="center">1</td>
+</tr>
+<tr>
+<td align="center">4</td>
+<td align="center">5</td>
+</tr>
+<tr>
+<td align="center">5</td>
+<td align="center">6</td>
+</tr>
+</tbody>
+</table>
+<p>This table reflects the price changes of a product with respect to the transaction time. We could write the following
+query:</p>
+<div class=language-sql>
+<div class="highlight"><pre class="chroma"><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="o">*</span>
+<span class="k">FROM</span> <span class="n">MyTable</span>
+<span class="n">MATCH_RECOGNIZE</span> <span class="p">(</span>
+<span class="k">ORDER</span> <span class="k">BY</span> <span class="n">transTime</span>
+<span class="n">MEASURES</span>
+<span class="k">LAST</span><span class="p">(</span><span class="n">A</span><span class="p">.</span><span class="n">price</span><span class="p">)</span> <span class="k">AS</span> <span class="n">beforePrice</span><span class="p">,</span>
+<span class="k">FIRST</span><span class="p">(</span><span class="n">B</span><span class="p">.</span><span class="n">price</span><span class="p">)</span> <span class="k">AS</span> <span class="n">afterPrice</span>
+<span class="n">PATTERN</span> <span class="p">(</span><span class="n">A</span><span class="o">+</span> <span class="n">B</span><span class="o">+</span><span class="p">)</span>
+<span class="n">DEFINE</span>
+<span class="n">A</span> <span class="k">AS</span> <span class="n">price</span> <span class="o">&lt;</span> <span class="n">PREV</span><span class="p">(</span><span class="n">A</span><span class="p">.</span><span class="n">price</span><span class="p">),</span>
+<span class="n">B</span> <span class="k">AS</span> <span class="n">price</span> <span class="o">&gt;</span> <span class="n">PREV</span><span class="p">(</span><span class="n">B</span><span class="p">.</span><span class="n">price</span><span class="p">)</span>
+<span class="p">)</span> <span class="k">AS</span> <span class="n">T</span></code></pre></div>
+</div>
+<p>This will find the local minimum price and the price after it. For the example dataset, the first 3 rows will be
+mapped to A and the rest of the rows will be mapped to B. Thus, we will have (1, 5) as the result.</p>
+<blockquote>
+<p>Very important: For my NFA implementation, it slightly breaks the rule in the SQL standard. Since the buffered NFA
+only stores an event to the buffer if the event is a match to some pattern class, There would be no way to get the
+previous event back if the previous row is discarded. So the first row would always be a match (different from the standard)
+if PREV is used.</p>
+</blockquote>
+<h2 id="progress">Progress</h2>
+<ol>
+<li>PRs
+<ol>
+<li><a href="https://github.com/apache/beam/pull/12232">Support MATCH_RECOGNIZE using regex library</a> (merged)</li>
+<li><a href="https://github.com/apache/beam/pull/12532">Support MATCH_RECOGNIZE using NFA</a> (pending)</li>
+</ol>
+</li>
+<li>Commits
+<ol>
+<li>partition by: <a href="https://github.com/apache/beam/pull/12232/commits/064ada7257970bcb1d35530be1b88cb3830f242b">commit 064ada7</a></li>
+<li>order by: <a href="https://github.com/apache/beam/pull/12232/commits/9cd1a82bec7b2f7c44aacfbd72f5f775bb58b650">commit 9cd1a82</a></li>
+<li>regex pattern match: <a href="https://github.com/apache/beam/pull/12232/commits/8d6ffcc213e30999fc495c119b68da4f62fad258">commit 8d6ffcc</a></li>
+<li>support quantifiers: <a href="https://github.com/apache/beam/pull/12232/commits/f529b876a2c2e43d012c71b3a83ebd55eb16f4ff">commit f529b87</a></li>
+<li>measures: <a href="https://github.com/apache/beam/pull/12232/commits/87935746647611aa139d664ebed10c8e638bb024">commit 8793574</a></li>
+<li>added NFA implementation: <a href="https://github.com/apache/beam/pull/12532/commits/fc731f2b0699d11853e7b76da86456427d434a2a">commit fc731f2</a></li>
+<li>implemented functions PREV and LAST: <a href="https://github.com/apache/beam/pull/12532/commits/fc731f2b0699d11853e7b76da86456427d434a2a">commit 35323da</a></li>
+</ol>
+</li>
+</ol>
+<h2 id="future-work">Future Work</h2>
+<ul>
+<li>Support FINAL/RUNNING keywords.</li>
+<li>Support more quantifiers.</li>
+<li>Add optimization to the NFA.</li>
+<li>A better way to realize MATCH_RECOGNIZE might be having a Complex Event Processing library at BEAM core (rather than using BEAM transforms).</li>
+</ul>
+<!-- Related Documents:
+- proposal
+- design doc
+- SQL 2016 standard
+- UMASS NFA^b paper
+-->
+<h2 id="references">References</h2>
+<ul>
+<li><a href="https://drive.google.com/file/d/1ZuFZV4dCFVPZW_-RiqbU0w-vShaZh_jX/view?usp=sharing">Project Proposal</a></li>
+<li><a href="https://s.apache.org/beam-sql-pattern-recognization">Design Documentation</a></li>
+<li><a href="https://www.iso.org/standard/65143.html">SQL 2016 documentation Part 5</a></li>
+<li><a href="https://dl.acm.org/doi/10.1145/1376616.1376634">UMASS paper on NFA with shared buffer</a></li>
+</ul></description><link>/blog/pattern-match-beam-sql/</link><pubDate>Thu, 27 Aug 2020 00:00:01 +0800</pubDate><guid>/blog/pattern-match-beam-sql/</guid><category>blog</category></item><item><title>Improved Annotation Support for the Python SDK</title><description>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -667,90 +855,4 @@ See the <a href="/get-started/downloads/#2190-2020-02-04">download page</a
, Kengo Seki, Kenneth Jung, Kenneth Knowles, Kyle Weaver, Kyle Winkelman, Lukas Drbal, Marek Simunek, Mark Liu, Maximilian Michels, Melissa Pashniak
, Michael Luckey, Michal Walenia, Mike Pedersen, Mikhail Gryzykhin, Niel Markwick, Pablo Estrada, Pascal Gula, Rehman Murad Ali, Reuven Lax, Rob, Robbe Sneyders
, Robert Bradshaw, Robert Burke, Rui Wang, Ruoyun Huang, Ryan Williams, Sam Rohde, Sam Whittle, Scott Wegner, Shoaib Zafar, Thomas Weise, Tianyang Hu, Tyler Akidau
-, Udi Meiri, Valentyn Tymofieiev, Xinyu Liu, XuMingmin, ttanay, tvalentyn, Łukasz Gajowy</p></description><link>/blog/beam-2.20.0/</link><pubDate>Wed, 15 Apr 2020 00:00:01 -0800</pubDate><guid>/blog/beam-2.20.0/</guid><category>blog</category></item><item><title>Apache Beam 2.19.0</title><description>
-<!--
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-http://www.apache.org/licenses/LICENSE-2.0
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
--->
-<p>We are happy to present the new 2.19.0 release of Beam. This release includes both improvements and new functionality.
-See the <a href="/get-started/downloads/#2190-2020-02-04">download page</a> for this release.</p>
-<p>For more information on changes in 2.19.0, check out the
-<a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&amp;version=12346582">detailed release notes</a>.</p>
-<h2 id="highlights">Highlights</h2>
-<ul>
-<li>Multiple improvements made into Python SDK harness:
-<a href="https://issues.apache.org/jira/browse/BEAM-8624">BEAM-8624</a>,
-<a href="https://issues.apache.org/jira/browse/BEAM-8623">BEAM-8623</a>,
-<a href="https://issues.apache.org/jira/browse/BEAM-7949">BEAM-7949</a>,
-<a href="https://issues.apache.org/jira/browse/BEAM-8935">BEAM-8935</a>,
-<a href="https://issues.apache.org/jira/browse/BEAM-8816">BEAM-8816</a></li>
-</ul>
-<h3 id="ios">I/Os</h3>
-<ul>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-1440">BEAM-1440</a> Create a BigQuery source (that implements iobase.BoundedSource) for Python SDK</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-2572">BEAM-2572</a> Implement an S3 filesystem for Python SDK</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-5192">BEAM-5192</a> Support Elasticsearch 7.x</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8745">BEAM-8745</a> More fine-grained controls for the size of a BigQuery Load job</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8801">BEAM-8801</a> PubsubMessageToRow should not check useFlatSchema() in processElement</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8953">BEAM-8953</a> Extend ParquetIO.Read/ReadFiles.Builder to support Avro GenericData model</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8946">BEAM-8946</a> Report collection size from MongoDBIOIT</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8978">BEAM-8978</a> Report saved data size from HadoopFormatIOIT</li>
-</ul>
-<h3 id="new-features--improvements">New Features / Improvements</h3>
-<ul>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-6008">BEAM-6008</a> Improve error reporting in Java/Python PortableRunner</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8296">BEAM-8296</a> Containerize the Spark job server</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8746">BEAM-8746</a> Allow the local job service to work from inside docker</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8837">BEAM-8837</a> PCollectionVisualizationTest: possible bug</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8139">BEAM-8139</a> Execute portable Spark application jar</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-9019">BEAM-9019</a> Improve Spark Encoders (wrappers of beam coders)</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-9053">BEAM-9053</a> Improve error message when unable to get the correct filesystem for specified path in Python SDK) Improve error message when unable to get the correct filesystem for specified path in Python SDK</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-9055">BEAM-9055</a> Unify the config names of Fn Data API across languages</li>
-</ul>
-<h3 id="sql">SQL</h3>
-<ul>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-5690">BEAM-5690</a> Issue with GroupByKey in BeamSql using SparkRunner</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8993">BEAM-8993</a> [SQL] MongoDb should use predicate push-down</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8844">BEAM-8844</a> [SQL] Create performance tests for BigQueryTable</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-9023">BEAM-9023</a> Upgrade to ZetaSQL 2019.12.1</li>
-</ul>
-<h3 id="breaking-changes">Breaking Changes</h3>
-<ul>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8989">BEAM-8989</a> Backwards incompatible change in ParDo.getSideInputs (caught by failure when running Apache Nemo quickstart)</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8402">BEAM-8402</a> Backwards incompatible change related to how Environments are represented in Python <code>DirectRunner</code>.</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-9218">BEAM-9218</a> Template staging broken on Beam 2.18.0</li>
-</ul>
-<h3 id="dependency-changes">Dependency Changes</h3>
-<ul>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8696">BEAM-8696</a> Beam Dependency Update Request: com.google.protobuf:protobuf-java</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8701">BEAM-8701</a> Beam Dependency Update Request: commons-io:commons-io</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8716">BEAM-8716</a> Beam Dependency Update Request: org.apache.commons:commons-csv</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8717">BEAM-8717</a> Beam Dependency Update Request: org.apache.commons:commons-lang3</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8749">BEAM-8749</a> Beam Dependency Update Request: com.datastax.cassandra:cassandra-driver-mapping</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-5546">BEAM-5546</a> Beam Dependency Update Request: commons-codec:commons-codec</li>
-</ul>
-<h3 id="bugfixes">Bugfixes</h3>
-<ul>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-9123">BEAM-9123</a> HadoopResourceId returns wrong directory name</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8962">BEAM-8962</a> FlinkMetricContainer causes churn in the JobManager and lets the web frontend malfunction</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-5495">BEAM-5495</a> PipelineResources algorithm is not working in most environments</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8025">BEAM-8025</a> Cassandra IO classMethod test is flaky</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8577">BEAM-8577</a> FileSystems may have not be initialized during ResourceId deserialization</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8582">BEAM-8582</a> Python SDK emits duplicate records for Default and AfterWatermark triggers</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8943">BEAM-8943</a> SDK harness servers don&rsquo;t shut down properly when SDK harness environment cleanup fails</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8995">BEAM-8995</a> apache_beam.io.gcp.bigquery_read_it_test failing on Py3.5 PC with: TypeError: the JSON object must be str, not &lsquo;bytes&rsquo;</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-8999">BEAM-8999</a> PGBKCVOperation does not respect timestamp combiners</li>
-<li><a href="https://issues.apache.org/jira/browse/BEAM-9050">BEAM-9050</a> Beam pickler doesn&rsquo;t pickle classes that have <strong>module</strong> set to None.</li>
-<li></li>
-<li>Various bug fixes and performance improvements.</li>
-</ul>
-<h2 id="list-of-contributors">List of Contributors</h2>
-<p>According to git shortlog, the following people contributed to the 2.19.0 release. Thank you to all contributors!</p>
-<p>Ahmet Altay, Alex Amato, Alexey Romanenko, Andrew Pilloud, Ankur Goenka, Anton Kedin, Boyuan Zhang, Brian Hulette, Brian Martin, Chamikara Jayalath, Charles Chen, Craig Chambers, Daniel Oliveira, David Moravek, David Rieber, Dustin Rhodes, Etienne Chauchot, Gleb Kanterov, Hai Lu, Heejong Lee, Ismaël Mejía, Jan Lukavský, Jason Kuster, Jean-Baptiste Onofré, Jeff Klukas, João Cabrita, J Ross Thomson, Juan Rael, Juta, Kasia Kucharczyk, Kengo Seki, Kenneth Jung, Kenneth Knowles, Kyle We [...]
\ No newline at end of file
+, Udi Meiri, Valentyn Tymofieiev, Xinyu Liu, XuMingmin, ttanay, tvalentyn, Łukasz Gajowy</p></description><link>/blog/beam-2.20.0/</link><pubDate>Wed, 15 Apr 2020 00:00:01 -0800</pubDate><guid>/blog/beam-2.20.0/</guid><category>blog</category></item></channel></rss>
\ No newline at end of file
diff --git a/website/generated-content/index.html b/website/generated-content/index.html
index 931e8dd..cbd6ef0 100644
--- a/website/generated-content/index.html
+++ b/website/generated-content/index.html
@@ -5,7 +5,7 @@
<a class="button button--primary" href=/get-started/try-apache-beam/>Try Beam</a>
<a class="button button--primary" href=/get-started/downloads/>Download Beam SDK 2.23.0</a></div><div class=hero__ctas><a class=button href=/get-started/quickstart-java/>Java Quickstart</a>
<a class=button href=/get-started/quickstart-py/>Python Quickstart</a>
-<a class=button href=/get-started/quickstart-go/>Go Quickstart</a></div></div></div><div class=hero__cols__col><div class=hero__blog><div class=hero__blog__title>The latest from the blog</div><div class=hero__blog__cards><a class=hero__blog__cards__card href=/blog/python-improved-annotations/><div class=hero__blog__cards__card__title>Improved Annotation Support for the Python SDK</div><div class=hero__blog__cards__card__date>Aug 21, 2020</div></a><a class=hero__blog__cards__card href=/bl [...]
+<a class=button href=/get-started/quickstart-go/>Go Quickstart</a></div></div></div><div class=hero__cols__col><div class=hero__blog><div class=hero__blog__title>The latest from the blog</div><div class=hero__blog__cards><a class=hero__blog__cards__card href=/blog/pattern-match-beam-sql/><div class=hero__blog__cards__card__title>Pattern Matching with Beam SQL</div><div class=hero__blog__cards__card__date>Aug 27, 2020</div></a><a class=hero__blog__cards__card href=/blog/python-improved-an [...]
<a class="button button--primary" href=/get-started/downloads/>Download Beam SDK 2.23.0</a></div><div class=ctas__ctas><a class=button href=/get-started/quickstart-java/>Java Quickstart</a>
<a class=button href=/get-started/quickstart-py/>Python Quickstart</a>
<a class=button href=/get-started/quickstart-go/>Go Quickstart</a></div></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class=footer__cols__col><div class=footer__cols__col__logo><img src=/images/beam_logo_circle.svg class=footer__logo alt="Beam logo"></div><div class=footer__cols__col__logo><img src=/images/apache_logo_circle.svg class=footer__logo alt="Apache logo"></div></div><div class="footer__cols__col footer__cols__col--md"><div class=foo [...]
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index 0b4c6a8..261cbcf 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/categories/blog/</loc><lastmod>2020-08-26T13:09:05-05:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-08-26T13:09:05-05:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-08-26T13:09:05-05:00</lastmod></url><url><loc>/blog/python-improved-annotations/</loc><lastmod>2020-08-26T13:09:05-05:00</lastmod></url>< [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/categories/blog/</loc><lastmod>2020-09-18T04:29:24+08:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-09-18T04:29:24+08:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-09-18T04:29:24+08:00</lastmod></url><url><loc>/blog/pattern-match-beam-sql/</loc><lastmod>2020-09-18T04:29:24+08:00</lastmod></url><url>< [...]
\ No newline at end of file