You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by me...@apache.org on 2018/07/01 04:26:10 UTC
[beam-site] 01/01: Prepare repository for deployment.
This is an automated email from the ASF dual-hosted git repository.
mergebot-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam-site.git
commit e4990b6324433bbadaa170892b833601de243126
Author: Mergebot <me...@apache.org>
AuthorDate: Sat Jun 30 21:26:07 2018 -0700
Prepare repository for deployment.
---
.../2018/06/26/beam-2.5.0.html} | 313 ++++++++-------------
content/blog/index.html | 30 ++
content/feed.xml | 156 ++++++----
content/index.html | 10 +-
4 files changed, 250 insertions(+), 259 deletions(-)
diff --git a/content/index.html b/content/blog/2018/06/26/beam-2.5.0.html
similarity index 56%
copy from content/index.html
copy to content/blog/2018/06/26/beam-2.5.0.html
index d37f1cd..fe2f393 100644
--- a/content/index.html
+++ b/content/blog/2018/06/26/beam-2.5.0.html
@@ -28,9 +28,8 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Apache Beam</title>
- <meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
-">
+ <title>Apache Beam 2.5.0</title>
+ <meta name="description" content="We are glad to present the new 2.5.0 release of Beam. This release includesmultiple fixes and new functionalities.">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
<link rel="stylesheet" href="/css/site.css">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.2.0/jquery.min.js"></script>
@@ -39,7 +38,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/blog/2018/06/26/beam-2.5.0.html" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<script>
@@ -52,7 +51,7 @@
</script>
</head>
- <body class="body body--index">
+ <body class="body ">
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -121,6 +120,24 @@
<div class="body__contained">
<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+
+
+<article class="post" itemscope itemtype="http://schema.org/BlogPosting">
+
+ <header class="post-header">
+ <h1 class="post-title" itemprop="name headline">Apache Beam 2.5.0</h1>
+ <p class="post-meta"><time datetime="2018-06-26T01:00:01-07:00" itemprop="datePublished">Jun 26, 2018</time> • <!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -133,206 +150,104 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<div class="hero-bg">
- <div class="hero section">
- <div class="hero__cols">
- <div class="hero__cols__col">
- <div class="hero__cols__col__content">
- <div class="hero__title">
- Apache Beam: An advanced unified programming model
- </div>
- <div class="hero__subtitle">
- Implement batch and streaming data processing jobs that run on any execution engine.
- </div>
- <div class="hero__ctas hero__ctas--first">
- <a class="button button--primary" href="/get-started/beam-overview/">Learn more</a>
- <a class="button button--primary" href="/get-started/downloads/">Download Beam SDK 2.5.0</a>
- </div>
- <div class="hero__ctas">
- <a class="button" href="/get-started/quickstart-java/">Java Quickstart</a>
- <a class="button" href="/get-started/quickstart-py/">Python Quickstart</a>
- <a class="button" href="/get-started/quickstart-go/">Go Quickstart</a>
- </div>
- </div>
- </div>
- <div class="hero__cols__col">
- <div class="hero__blog">
- <div class="hero__blog__title">
- The latest from the blog
- </div>
- <div class="hero__blog__cards">
-
- <a class="hero__blog__cards__card" href="/blog/2018/02/19/beam-2.3.0.html">
- <div class="hero__blog__cards__card__title">Apache Beam 2.3.0</div>
- <div class="hero__blog__cards__card__date">Feb 19, 2018</div>
- </a>
-
- <a class="hero__blog__cards__card" href="/blog/2018/01/09/beam-a-look-back.html">
- <div class="hero__blog__cards__card__title">Apache Beam: A Look Back at 2017</div>
- <div class="hero__blog__cards__card__date">Jan 9, 2018</div>
- </a>
-
- <a class="hero__blog__cards__card" href="/blog/2017/08/28/timely-processing.html">
- <div class="hero__blog__cards__card__title">Timely (and Stateful) Processing with Apache Beam</div>
- <div class="hero__blog__cards__card__date">Aug 28, 2017</div>
- </a>
-
- </div>
- </div>
- </div>
- </div>
- </div>
-</div>
+ Alexey Romanenko [<a href="https://twitter.com/alexromdev">@alexromdev</a>]
+</p>
+ </header>
-<div class="pillars section">
- <div class="pillars__title">
- All about Apache Beam
- </div>
- <div class="pillars__cols">
-
- <div class="pillars__cols__col">
- <div class="pillars__cols__col__title">
- Unified
- </div>
- <div class="pillars__cols__col__body">
- Use a single programming model for both batch and streaming use cases.
- </div>
- </div>
-
- <div class="pillars__cols__col">
- <div class="pillars__cols__col__title">
- Portable
- </div>
- <div class="pillars__cols__col__body">
- Execute pipelines on multiple execution environments.
- </div>
- </div>
-
- <div class="pillars__cols__col">
- <div class="pillars__cols__col__title">
- Extensible
- </div>
- <div class="pillars__cols__col__body">
- Write and share new SDKs, IO connectors, and transformation libraries.
- </div>
- </div>
-
- </div>
-</div>
+ <div class="post-content" itemprop="articleBody">
+ <p>We are glad to present the new 2.5.0 release of Beam. This release includes
+multiple fixes and new functionalities. <!--more--> For more information
+please check the detailed release notes.</p>
-<div class="graphic section">
-<div class="graphic__image">
-<img src="/images/beam_architecture.png" alt="Beam architecture" />
-</div>
-</div>
+<h1 id="new-features--improvements">New Features / Improvements</h1>
-<div class="logos section">
- <div class="logos__title">
- Works with
- </div>
- <div class="logos__logos">
-
- <div class="logos__logos__logo">
- <a href="http://apex.apache.org"><img src="/images/logo_apex.png" alt="APEX" /></a>
- </div>
-
- <div class="logos__logos__logo">
- <a href="http://flink.apache.org"><img src="/images/logo_flink.png" alt="Flink" /></a>
- </div>
-
- <div class="logos__logos__logo">
- <a href="http://spark.apache.org/"><img src="/images/logo_spark.png" alt="Spark" /></a>
- </div>
-
- <div class="logos__logos__logo">
- <a href="https://cloud.google.com/dataflow/"><img src="/images/logo_google_cloud.png" alt="Google Cloud Dataflow" /></a>
- </div>
-
- <div class="logos__logos__logo">
- <a href="http://gearpump.apache.org/"><img src="/images/logo_gearpump.png" alt="Gearpump" /></a>
- </div>
-
- <div class="logos__logos__logo">
- <a href="http://samza.apache.org/"><img src="/images/logo_samza.png" alt="Samza" /></a>
- </div>
-
- </div>
-</div>
+<h2 id="go-sdk-support">Go SDK support</h2>
+<p>The Go SDK has been officially accepted into the project, after an incubation period and community effort. Go pipelines run on Dataflow runner. More details are <a href="https://beam.apache.org/documentation/sdks/go/">here</a>.</p>
-<div class="cards section section--wide">
- <div class="section__contained">
- <div class="cards__title">
- Testimonials
- </div>
- <div class="cards__cards">
-
- <div class="cards__cards__card">
- <div class="cards__cards__card__body">
- A framework that delivers the flexibility and advanced functionality our customers need.
- </div>
- <div class="cards__cards__card__user">
- <!-- TODO: Implement icons.
- <div class="cards__cards__card__user__icon">
- </div>
- -->
- <div class="cards__cards__card__user__name">
- –Talend
- </div>
- </div>
- </div>
-
- <div class="cards__cards__card">
- <div class="cards__cards__card__body">
- Apache Beam has powerful semantics that solve real-world challenges of stream processing.
- </div>
- <div class="cards__cards__card__user">
- <!-- TODO: Implement icons.
- <div class="cards__cards__card__user__icon">
- </div>
- -->
- <div class="cards__cards__card__user__name">
- –PayPal
- </div>
- </div>
- </div>
-
- <div class="cards__cards__card">
- <div class="cards__cards__card__body">
- Apache Beam represents a principled approach for analyzing data streams.
- </div>
- <div class="cards__cards__card__user">
- <!-- TODO: Implement icons.
- <div class="cards__cards__card__user__icon">
- </div>
- -->
- <div class="cards__cards__card__user__name">
- –data Artisans
- </div>
- </div>
- </div>
-
- </div>
- <div class="cards__body">
- Beam is an open source community and contributions are greatly appreciated!
- If you’d like to contribute, please see the <a href="/contribute/">Contribute</a> section.
- </div>
- </div>
-</div>
+<h2 id="parquet-support">Parquet support</h2>
+<p>Support for Apache Parquet format was added. It uses Parquet 1.10 release which, thanks to AvroParquerWriter’s API changes, allows FileIO.Sink implementation.</p>
+
+<h2 id="performanceintegration-tests">Performance/Integration Tests</h2>
+<ul>
+ <li>Added new integration tests - HCatalogIOIT (Hive), HBaseIOIT, ParquetIOIT (with the IO itself, local filesystem, HDFS)</li>
+ <li>Multinode (3 data node) HDFS cluster is used for running tests on HDFS.</li>
+ <li>Several improvements on performance tests running and results analysis.</li>
+ <li>Scaled up Kubernetes cluster from 1 to 3 nodes.</li>
+ <li>Added metrics in Spark streaming.</li>
+</ul>
+
+<h2 id="internal-build-system-migrated-to-gradle">Internal Build System: Migrated to Gradle</h2>
+<p>After a months-long community effort, the internal Beam build has been migrated from Maven to Gradle. The new build system was chosen because of dependency-driven build support, incremental build/test, and support for non-Java languages.</p>
+
+<h2 id="nexmark-improvements">Nexmark Improvements</h2>
+<ul>
+ <li>Kafka support as a source/sink for events and results.</li>
+ <li>Translation of some queries to Beam SQL.</li>
+</ul>
+
+<h2 id="beam-sql">Beam SQL</h2>
+<ul>
+ <li>Support for MAP, ROW, ARRAY data types</li>
+ <li>Support UNNEST on array fields</li>
+ <li>Improved optimizations</li>
+ <li>Upgrade Calcite to 1.16</li>
+ <li>Support SQL on POJOs via automatic conversion</li>
+ <li>Schema moved into core Beam</li>
+ <li>UDAFs can be indirect suclasses of CombineFn</li>
+ <li>Many other small bugfixes</li>
+</ul>
+
+<h2 id="portability">Portability</h2>
+<ul>
+ <li>Common shared code related to supporting portable execution for runners.</li>
+ <li>Python SDK supporting side inputs over the portability APIs.</li>
+</ul>
+
+<h2 id="extract-metrics-in-a-runner-agnostic-way">Extract metrics in a runner agnostic way</h2>
+<p>Metrics are pushed by the runners to configurable sinks (Http REST sink available). It is already enabled in Flink and Spark runner, work is in progress for Dataflow.</p>
+
+<h1 id="miscellaneous-fixes">Miscellaneous Fixes</h1>
+
+<h2 id="sdks">SDKs</h2>
+
+<ul>
+ <li>Implemented HDFS FileSystem for Python SDK.</li>
+ <li>Python SDK adds support for side inputs for streaming execution.</li>
+</ul>
+
+<h2 id="runners">Runners</h2>
+<ul>
+ <li>Updated Spark runner to Spark version 2.3.1</li>
+ <li>Fixed issue with late elements windowed into expired fixed windows get dropped in Directrunner.</li>
+</ul>
+
+<h2 id="ios">IOs</h2>
+
+<ul>
+ <li>CassandraIO gained a better split algorithm based on overlapping regions.</li>
+ <li>ElasticsearchIO supports partial updates.</li>
+ <li>ElasticsearchIO allows to pass id, type and index per document.</li>
+ <li>SolrIO supports a more robust retry on write strategy.</li>
+ <li>S3 FileSystem supports encryption (SSE-S3, SSE-C and SSE-KMS).</li>
+ <li>Improved connection management in JdbcIO.</li>
+ <li>Added support the element timestamps while publishing to Kafka.</li>
+</ul>
+
+<h2 id="other">Other</h2>
+
+<ul>
+ <li>Use Java ErrorProne for static analysis.</li>
+</ul>
+
+<h1 id="list-of-contributors">List of Contributors</h1>
+
+<p>According to git shortlog, the following 84 people contributed to the 2.5.0 release. Thank you to all contributors!</p>
+
+<p>Ahmet Altay, Alan Myrvold, Alex Amato, Alex Van Boxel, Alexander Dejanovski, Alexey Romanenko, Aljoscha Krettek, ananvay, Andreas Ehrencrona, Andrew Pilloud, Ankur Goenka, Anton Kedin, arkash, Austin Bennett, Axel Magnuson, Ben Chambers, Ben Sidhom, Bill Neubauer, Boyuan Zhang, Braden Bassingthwaite, Cade Markegard, cclauss, Chamikara Jayalath, Charles Chen, Chuan Yu Foo, Cody Schroeder, Colm O hEigeartaigh, Daniel Oliveira, Dariusz Aniszewski, David Cavazos, Dawid Wysakowicz, Eric Ro [...]
-<div class="ctas section">
- <div class="ctas__title">
- Get started
- </div>
- <div class="ctas__ctas ctas__ctas--top">
- <a class="button button--primary" href="/get-started/beam-overview/">Learn more</a>
- <a class="button button--primary" href="/get-started/downloads/">Download Beam SDK 2.5.0</a>
- </div>
- <div class="ctas__ctas">
- <a class="button" href="/get-started/quickstart-java/">Java Quickstart</a>
- <a class="button" href="/get-started/quickstart-py/">Python Quickstart</a>
- <a class="button" href="/get-started/quickstart-go/">Go Quickstart</a>
</div>
-</div>
+
+</article>
</div>
<!--
diff --git a/content/blog/index.html b/content/blog/index.html
index bae0af4..160e07f 100644
--- a/content/blog/index.html
+++ b/content/blog/index.html
@@ -139,6 +139,36 @@ limitations under the License.
<p>This is the blog for the Apache Beam project. This blog contains news and updates
for the project.</p>
+<h3 id="apache-beam-250"><a class="post-link" href="/blog/2018/06/26/beam-2.5.0.html">Apache Beam 2.5.0</a></h3>
+<p><i>Jun 26, 2018 • <!–
+Licensed under the Apache License, Version 2.0 (the “License”);
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at</i></p>
+
+<p>http://www.apache.org/licenses/LICENSE-2.0</p>
+
+<p>Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an “AS IS” BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+–>
+ Alexey Romanenko [<a href="https://twitter.com/alexromdev">@alexromdev</a>]
+</i></p>
+
+<p>We are glad to present the new 2.5.0 release of Beam. This release includes
+multiple fixes and new functionalities.</p>
+
+<!-- Render a "read more" button if the post is longer than the excerpt -->
+
+<p>
+<a class="btn btn-default btn-sm" href="/blog/2018/06/26/beam-2.5.0.html" role="button">
+Read more <span class="glyphicon glyphicon-menu-right" aria-hidden="true"></span>
+</a>
+</p>
+
+<hr />
+
<h3 id="apache-beam-230"><a class="post-link" href="/blog/2018/02/19/beam-2.3.0.html">Apache Beam 2.3.0</a></h3>
<p><i>Feb 19, 2018 • <!–
Licensed under the Apache License, Version 2.0 (the “License”);
diff --git a/content/feed.xml b/content/feed.xml
index 7c588b2..2ad71e2 100644
--- a/content/feed.xml
+++ b/content/feed.xml
@@ -20,6 +20,107 @@
<generator>Jekyll v3.2.0</generator>
<item>
+ <title>Apache Beam 2.5.0</title>
+ <description><p>We are glad to present the new 2.5.0 release of Beam. This release includes
+multiple fixes and new functionalities. <!--more--> For more information
+please check the detailed release notes.</p>
+
+<h1 id="new-features--improvements">New Features / Improvements</h1>
+
+<h2 id="go-sdk-support">Go SDK support</h2>
+<p>The Go SDK has been officially accepted into the project, after an incubation period and community effort. Go pipelines run on Dataflow runner. More details are <a href="https://beam.apache.org/documentation/sdks/go/">here</a>.</p>
+
+<h2 id="parquet-support">Parquet support</h2>
+<p>Support for Apache Parquet format was added. It uses Parquet 1.10 release which, thanks to AvroParquerWriter’s API changes, allows FileIO.Sink implementation.</p>
+
+<h2 id="performanceintegration-tests">Performance/Integration Tests</h2>
+<ul>
+ <li>Added new integration tests - HCatalogIOIT (Hive), HBaseIOIT, ParquetIOIT (with the IO itself, local filesystem, HDFS)</li>
+ <li>Multinode (3 data node) HDFS cluster is used for running tests on HDFS.</li>
+ <li>Several improvements on performance tests running and results analysis.</li>
+ <li>Scaled up Kubernetes cluster from 1 to 3 nodes.</li>
+ <li>Added metrics in Spark streaming.</li>
+</ul>
+
+<h2 id="internal-build-system-migrated-to-gradle">Internal Build System: Migrated to Gradle</h2>
+<p>After a months-long community effort, the internal Beam build has been migrated from Maven to Gradle. The new build system was chosen because of dependency-driven build support, incremental build/test, and support for non-Java languages.</p>
+
+<h2 id="nexmark-improvements">Nexmark Improvements</h2>
+<ul>
+ <li>Kafka support as a source/sink for events and results.</li>
+ <li>Translation of some queries to Beam SQL.</li>
+</ul>
+
+<h2 id="beam-sql">Beam SQL</h2>
+<ul>
+ <li>Support for MAP, ROW, ARRAY data types</li>
+ <li>Support UNNEST on array fields</li>
+ <li>Improved optimizations</li>
+ <li>Upgrade Calcite to 1.16</li>
+ <li>Support SQL on POJOs via automatic conversion</li>
+ <li>Schema moved into core Beam</li>
+ <li>UDAFs can be indirect suclasses of CombineFn</li>
+ <li>Many other small bugfixes</li>
+</ul>
+
+<h2 id="portability">Portability</h2>
+<ul>
+ <li>Common shared code related to supporting portable execution for runners.</li>
+ <li>Python SDK supporting side inputs over the portability APIs.</li>
+</ul>
+
+<h2 id="extract-metrics-in-a-runner-agnostic-way">Extract metrics in a runner agnostic way</h2>
+<p>Metrics are pushed by the runners to configurable sinks (Http REST sink available). It is already enabled in Flink and Spark runner, work is in progress for Dataflow.</p>
+
+<h1 id="miscellaneous-fixes">Miscellaneous Fixes</h1>
+
+<h2 id="sdks">SDKs</h2>
+
+<ul>
+ <li>Implemented HDFS FileSystem for Python SDK.</li>
+ <li>Python SDK adds support for side inputs for streaming execution.</li>
+</ul>
+
+<h2 id="runners">Runners</h2>
+<ul>
+ <li>Updated Spark runner to Spark version 2.3.1</li>
+ <li>Fixed issue with late elements windowed into expired fixed windows get dropped in Directrunner.</li>
+</ul>
+
+<h2 id="ios">IOs</h2>
+
+<ul>
+ <li>CassandraIO gained a better split algorithm based on overlapping regions.</li>
+ <li>ElasticsearchIO supports partial updates.</li>
+ <li>ElasticsearchIO allows to pass id, type and index per document.</li>
+ <li>SolrIO supports a more robust retry on write strategy.</li>
+ <li>S3 FileSystem supports encryption (SSE-S3, SSE-C and SSE-KMS).</li>
+ <li>Improved connection management in JdbcIO.</li>
+ <li>Added support the element timestamps while publishing to Kafka.</li>
+</ul>
+
+<h2 id="other">Other</h2>
+
+<ul>
+ <li>Use Java ErrorProne for static analysis.</li>
+</ul>
+
+<h1 id="list-of-contributors">List of Contributors</h1>
+
+<p>According to git shortlog, the following 84 people contributed to the 2.5.0 release. Thank you to all contributors!</p>
+
+<p>Ahmet Altay, Alan Myrvold, Alex Amato, Alex Van Boxel, Alexander Dejanovski, Alexey Romanenko, Aljoscha Krettek, ananvay, Andreas Ehrencrona, Andrew Pilloud, Ankur Goenka, Anton Kedin, arkash, Austin Bennett, Axel Magnuson, Ben Chambers, Ben Sidhom, Bill Neubauer, Boyuan Zhang, Braden Bassingthwaite, Cade Markegard, cclauss, Chamikara Jayalath, Charles Chen, Chuan Yu Foo, Cody Schroeder, Colm O hEigeartaigh, Daniel Oliveira, Dariusz Aniszewski, David Cavazos, Dawid Wysakowicz, E [...]
+</description>
+ <pubDate>Tue, 26 Jun 2018 01:00:01 -0700</pubDate>
+ <link>https://beam.apache.org/blog/2018/06/26/beam-2.5.0.html</link>
+ <guid isPermaLink="true">https://beam.apache.org/blog/2018/06/26/beam-2.5.0.html</guid>
+
+
+ <category>blog</category>
+
+ </item>
+
+ <item>
<title>Apache Beam 2.3.0</title>
<description><!--
Licensed under the Apache License, Version 2.0 (the "License");
@@ -2346,60 +2447,5 @@ tracker, pull requests, and events.</p>
</item>
- <item>
- <title>Release 0.4.0 adds a runner for Apache Apex</title>
- <description><!--
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
--->
-
-<p>The latest release 0.4.0 of <a href="/">Apache Beam</a> adds a new runner for <a href="http://apex.apache.org/">Apache Apex</a>. We are excited to reach this initial milestone and are looking forward to continued collaboration between the Beam and Apex communities to advance the runner.</p>
-
-<!--more-->
-
-<p>Beam evolved from the Google Dataflow SDK and as incubator project has quickly adapted the Apache way, grown the community and attracts increasing interest from users that hope to benefit from a conceptual strong unified programming model that is portable between different big data processing frameworks (see <a href="https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101">Streaming-101</a> and <a href="https://www.oreilly.com/ideas/the-w [...]
-
-<p>Apex is a stream processing framework for low-latency, high-throughput, stateful and reliable processing of complex analytics pipelines on clusters. Apex was developed since 2012 and is used in production by large companies for real-time and batch processing at scale.</p>
-
-<p>The initial revision of the runner was focussed on broad coverage of the Beam model on a functional level. That means, there will be follow up work in several areas to take the runner from functional to scalable and high performance to match the capabilities of Apex and its native API. The runner capabilities matrix shows that the Apex capabilities are well aligned with the Beam model. Specifically, the ability to track computational state in a fault tolerant and efficient manne [...]
-
-<h2 id="stateful-stream-processor">Stateful Stream Processor</h2>
-
-<p>Apex was built as stateful stream processor from the ground up. Operators <a href="https://www.datatorrent.com/blog/blog-introduction-to-checkpoint/">checkpoint</a> state in a distributed and asynchronous manner that produces a consistent snapshot for the entire processing graph, which can be used for recovery. Apex also supports such recovery in an incremental, or fine grained, manner. This means only the portion of the DAG that is actually affected by a f [...]
-
-<h2 id="translation-to-apex-dag">Translation to Apex DAG</h2>
-
-<p>A Beam runner needs to implement the translation from the Beam model to the underlying frameworks execution model. In the case of Apex, the runner will translate the pipeline into the <a href="https://www.datatorrent.com/blog/tracing-dags-from-specification-to-execution/">native (compositional, low level) DAG API</a> (which is also the base for a number of other API that are available to specify applications that run on Apex). The DAG consists of operators [...]
-
-<h2 id="execution-and-testing">Execution and Testing</h2>
-
-<p>In this release, the Apex runner executes the pipelines in embedded mode, where, similar to the direct runner, everything is executed in a single JVM. See <a href="/get-started/quickstart/">quickstart</a> on how to run the Beam examples with the Apex runner.</p>
-
-<p>Embedded mode is useful for development and debugging. Apex in production runs distributed on Apache Hadoop YARN clusters. An example how a Beam pipeline can be embedded into an Apex application package to run on YARN can be found <a href="https://github.com/tweise/apex-samples/tree/master/beam-apex-wordcount">here</a> and support for direct launch in the runner is currently being worked on.</p>
-
-<p>The Beam project has a strong focus on development process and tooling, including testing. For the runners, there is a comprehensive test suite with more than 200 integration tests that are executed against each runner to ensure they don’t break as changes are made. The tests cover the capabilities of the matrix and thus are a measure of completeness and correctness of the runner implementations. The suite was very helpful when developing the Apex runner.</p>
-
-<h2 id="outlook">Outlook</h2>
-
-<p>The next step is to take the Apex runner from functional to ready for real applications that run distributed, leveraging the scalability and performance features of Apex, similar to its native API. This includes chaining of ParDos, partitioning, optimizing combine operations etc. To get involved, please see <a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20and%20component%20%3D%20runner-apex%20and%20resolution%20%3D%20unresolved">JIRA< [...]
-</description>
- <pubDate>Mon, 09 Jan 2017 09:00:01 -0800</pubDate>
- <link>https://beam.apache.org/blog/2017/01/09/added-apex-runner.html</link>
- <guid isPermaLink="true">https://beam.apache.org/blog/2017/01/09/added-apex-runner.html</guid>
-
-
- <category>blog</category>
-
- </item>
-
</channel>
</rss>
diff --git a/content/index.html b/content/index.html
index d37f1cd..ec600c0 100644
--- a/content/index.html
+++ b/content/index.html
@@ -162,6 +162,11 @@ limitations under the License.
</div>
<div class="hero__blog__cards">
+ <a class="hero__blog__cards__card" href="/blog/2018/06/26/beam-2.5.0.html">
+ <div class="hero__blog__cards__card__title">Apache Beam 2.5.0</div>
+ <div class="hero__blog__cards__card__date">Jun 26, 2018</div>
+ </a>
+
<a class="hero__blog__cards__card" href="/blog/2018/02/19/beam-2.3.0.html">
<div class="hero__blog__cards__card__title">Apache Beam 2.3.0</div>
<div class="hero__blog__cards__card__date">Feb 19, 2018</div>
@@ -172,11 +177,6 @@ limitations under the License.
<div class="hero__blog__cards__card__date">Jan 9, 2018</div>
</a>
- <a class="hero__blog__cards__card" href="/blog/2017/08/28/timely-processing.html">
- <div class="hero__blog__cards__card__title">Timely (and Stateful) Processing with Apache Beam</div>
- <div class="hero__blog__cards__card__date">Aug 28, 2017</div>
- </a>
-
</div>
</div>
</div>