You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by da...@apache.org on 2016/11/14 00:20:28 UTC
[1/3] incubator-beam-site git commit: Update Apex runner info after
merge to master.
Repository: incubator-beam-site
Updated Branches:
refs/heads/asf-site 41e553944 -> 2bd194414
Update Apex runner info after merge to master.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/b372fd9c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/tree/b372fd9c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/diff/b372fd9c
Branch: refs/heads/asf-site
Commit: b372fd9c797bc716424a6f80fd8fa2e934c091b7
Parents: 41e5539
Author: Thomas Weise <th...@apache.org>
Authored: Sat Nov 12 02:29:37 2016 -0800
Committer: Thomas Weise <th...@apache.org>
Committed: Sun Nov 13 16:12:21 2016 -0800
----------------------------------------------------------------------
src/_data/capability-matrix.yml | 2 +-
src/documentation/runners/apex.md | 2 +-
src/get-started/quickstart.md | 26 ++++++++++++++++++++++++--
src/index.md | 6 +++++-
4 files changed, 31 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/b372fd9c/src/_data/capability-matrix.yml
----------------------------------------------------------------------
diff --git a/src/_data/capability-matrix.yml b/src/_data/capability-matrix.yml
index 375fdf4..48d2abd 100644
--- a/src/_data/capability-matrix.yml
+++ b/src/_data/capability-matrix.yml
@@ -8,7 +8,7 @@ columns:
- class: spark
name: Apache Spark
- class: apex
- name: Apache Apex (on feature branch)
+ name: Apache Apex
categories:
- description: What is being computed?
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/b372fd9c/src/documentation/runners/apex.md
----------------------------------------------------------------------
diff --git a/src/documentation/runners/apex.md b/src/documentation/runners/apex.md
index 408e6de..050eb83 100644
--- a/src/documentation/runners/apex.md
+++ b/src/documentation/runners/apex.md
@@ -5,5 +5,5 @@ permalink: /documentation/runners/apex/
---
# Using the Apache Apex Runner
-This page is under construction ([BEAM-825](https://issues.apache.org/jira/browse/BEAM-825)). The runner is on a feature branch.
+This page is under construction ([BEAM-825](https://issues.apache.org/jira/browse/BEAM-825)).
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/b372fd9c/src/get-started/quickstart.md
----------------------------------------------------------------------
diff --git a/src/get-started/quickstart.md b/src/get-started/quickstart.md
index f5db479..0a956d2 100644
--- a/src/get-started/quickstart.md
+++ b/src/get-started/quickstart.md
@@ -56,7 +56,7 @@ For a detailed introduction to the Beam concepts used in these examples, see the
## Run WordCount
-A single Beam pipeline can run on multiple Beam [runners]({{ site.baseurl }}/documentation#runners), including the [SparkRunner]({{ site.baseurl }}/documentation/runners/spark), [FlinkRunner]({{ site.baseurl }}/documentation/runners/flink), or [DataflowRunner]({{ site.baseurl }}/documentation/runners/dataflow). The [DirectRunner]({{ site.baseurl }}/documentation/runners/direct) is a common runner for getting started, as it runs locally on your machine and requires no specific setup.
+A single Beam pipeline can run on multiple Beam [runners]({{ site.baseurl }}/documentation#runners), including the [ApexRunner]({{ site.baseurl }}/documentation/runners/apex), [FlinkRunner]({{ site.baseurl }}/documentation/runners/flink), [SparkRunner]({{ site.baseurl }}/documentation/runners/spark) or [DataflowRunner]({{ site.baseurl }}/documentation/runners/dataflow). The [DirectRunner]({{ site.baseurl }}/documentation/runners/direct) is a common runner for getting started, as it runs locally on your machine and requires no specific setup.
After you've chosen which runner you'd like to use:
@@ -73,6 +73,12 @@ $ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args="--inputFile=pom.xml --output=counts"
```
+{:.runner-apex}
+```
+$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+ -Dexec.args="--inputFile=pom.xml --output=counts --runner=ApexRunner" -Papex-runner
+```
+
{:.runner-flink}
```
TODO BEAM-899
@@ -100,6 +106,11 @@ Once the pipeline has completed, you can view the output. You'll notice that the
$ ls counts*
```
+{:.runner-apex}
+```
+$ ls counts*
+```
+
{:.runner-flink}
```
TODO BEAM-899
@@ -131,6 +142,17 @@ Foundation: 1
...
```
+{:.runner-apex}
+```
+$ cat counts*
+BEAM: 1
+have: 1
+simple: 1
+skip: 4
+PAssert: 1
+...
+```
+
{:.runner-flink}
```
TODO BEAM-899
@@ -162,4 +184,4 @@ barrenly: 1
* Join the Beam [users@]({{ site.baseurl }}/get-started/support#mailing-lists) mailing list.
Please don't hesitate to [reach out]({{ site.baseurl }}/get-started/support) if you encounter any issues!
-
\ No newline at end of file
+
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/b372fd9c/src/index.md
----------------------------------------------------------------------
diff --git a/src/index.md b/src/index.md
index 31a8a09..e16d1bf 100644
--- a/src/index.md
+++ b/src/index.md
@@ -9,7 +9,7 @@ The Apache Beam project is in the process of bootstrapping. This includes the we
# Apache Beam (incubating)
-Apache Beam is an open source, unified programming model that you can use to create a data processing **pipeline**. You start by building a program that defines the pipeline using one of the open source Beam SDKs. The pipeline is then executed by one of Beam's supported **distributed processing back-ends**, which include [Apache Flink](http://flink.apache.org), [Apache Spark](http://spark.apache.org), and [Google Cloud Dataflow](https://cloud.google.com/dataflow).
+Apache Beam is an open source, unified programming model that you can use to create a data processing **pipeline**. You start by building a program that defines the pipeline using one of the open source Beam SDKs. The pipeline is then executed by one of Beam's supported **distributed processing back-ends**, which include [Apache Apex](http://apex.apache.org), [Apache Flink](http://flink.apache.org), [Apache Spark](http://spark.apache.org), and [Google Cloud Dataflow](https://cloud.google.com/dataflow).
Beam is particularly useful for [Embarrassingly Parallel](http://en.wikipedia.org/wiki/Embarassingly_parallel) data processing tasks, in which the problem can be decomposed into many smaller bundles of data that can be processed independently and in parallel. You can also use Beam for Extract, Transform, and Load (ETL) tasks and pure data integration. These tasks are useful for moving data between different storage media and data sources, transforming data into a more desirable format, or loading data onto a new system.
@@ -61,6 +61,10 @@ Beam currently supports Runners that work with the following distributed process
<td>Apache Spark</td>
<td>In Development</td>
</tr>
+<tr>
+ <td>Apache Apex</td>
+ <td>In Development</td>
+</tr>
</table>
**Note:** You can always execute your pipeline locally for testing and debugging purposes.
[2/3] incubator-beam-site git commit: Regenerate website
Posted by da...@apache.org.
Regenerate website
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/13d13604
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/tree/13d13604
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/diff/13d13604
Branch: refs/heads/asf-site
Commit: 13d1360478855ac5e8386ddc60b0ca510bf03572
Parents: b372fd9
Author: Davor Bonaci <da...@google.com>
Authored: Sun Nov 13 16:20:05 2016 -0800
Committer: Davor Bonaci <da...@google.com>
Committed: Sun Nov 13 16:20:05 2016 -0800
----------------------------------------------------------------------
content/documentation/runners/apex/index.html | 2 +-
.../runners/capability-matrix/index.html | 16 +++++++--------
content/get-started/quickstart/index.html | 21 +++++++++++++++++++-
content/index.html | 6 +++++-
4 files changed, 34 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/13d13604/content/documentation/runners/apex/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/runners/apex/index.html b/content/documentation/runners/apex/index.html
index f25dec0..fcbb722 100644
--- a/content/documentation/runners/apex/index.html
+++ b/content/documentation/runners/apex/index.html
@@ -142,7 +142,7 @@
<div class="row">
<h1 id="using-the-apache-apex-runner">Using the Apache Apex Runner</h1>
-<p>This page is under construction (<a href="https://issues.apache.org/jira/browse/BEAM-825">BEAM-825</a>). The runner is on a feature branch.</p>
+<p>This page is under construction (<a href="https://issues.apache.org/jira/browse/BEAM-825">BEAM-825</a>).</p>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/13d13604/content/documentation/runners/capability-matrix/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/runners/capability-matrix/index.html b/content/documentation/runners/capability-matrix/index.html
index 30d5532..a5e4190 100644
--- a/content/documentation/runners/capability-matrix/index.html
+++ b/content/documentation/runners/capability-matrix/index.html
@@ -190,7 +190,7 @@
<th class="cap-summary color-platform format-platform" style="color:#ec3">Apache Spark</th>
- <th class="cap-summary color-platform format-platform" style="color:#ec3">Apache Apex (on feature branch)</th>
+ <th class="cap-summary color-platform format-platform" style="color:#ec3">Apache Apex</th>
</tr>
@@ -482,7 +482,7 @@
<th class="cap-summary color-platform format-platform" style="color:#59f">Apache Spark</th>
- <th class="cap-summary color-platform format-platform" style="color:#59f">Apache Apex (on feature branch)</th>
+ <th class="cap-summary color-platform format-platform" style="color:#59f">Apache Apex</th>
</tr>
@@ -714,7 +714,7 @@
<th class="cap-summary color-platform format-platform" style="color:#8c6">Apache Spark</th>
- <th class="cap-summary color-platform format-platform" style="color:#8c6">Apache Apex (on feature branch)</th>
+ <th class="cap-summary color-platform format-platform" style="color:#8c6">Apache Apex</th>
</tr>
@@ -976,7 +976,7 @@
<th class="cap-summary color-platform format-platform" style="color:#d77">Apache Spark</th>
- <th class="cap-summary color-platform format-platform" style="color:#d77">Apache Apex (on feature branch)</th>
+ <th class="cap-summary color-platform format-platform" style="color:#d77">Apache Apex</th>
</tr>
@@ -1102,7 +1102,7 @@
<th class="cap color-platform format-platform" style="color:#ec3">Apache Spark</th>
- <th class="cap color-platform format-platform" style="color:#ec3">Apache Apex (on feature branch)</th>
+ <th class="cap color-platform format-platform" style="color:#ec3">Apache Apex</th>
</tr>
@@ -1394,7 +1394,7 @@
<th class="cap color-platform format-platform" style="color:#59f">Apache Spark</th>
- <th class="cap color-platform format-platform" style="color:#59f">Apache Apex (on feature branch)</th>
+ <th class="cap color-platform format-platform" style="color:#59f">Apache Apex</th>
</tr>
@@ -1626,7 +1626,7 @@
<th class="cap color-platform format-platform" style="color:#8c6">Apache Spark</th>
- <th class="cap color-platform format-platform" style="color:#8c6">Apache Apex (on feature branch)</th>
+ <th class="cap color-platform format-platform" style="color:#8c6">Apache Apex</th>
</tr>
@@ -1888,7 +1888,7 @@
<th class="cap color-platform format-platform" style="color:#d77">Apache Spark</th>
- <th class="cap color-platform format-platform" style="color:#d77">Apache Apex (on feature branch)</th>
+ <th class="cap color-platform format-platform" style="color:#d77">Apache Apex</th>
</tr>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/13d13604/content/get-started/quickstart/index.html
----------------------------------------------------------------------
diff --git a/content/get-started/quickstart/index.html b/content/get-started/quickstart/index.html
index c4f6772..bc7396c 100644
--- a/content/get-started/quickstart/index.html
+++ b/content/get-started/quickstart/index.html
@@ -196,7 +196,7 @@ MinimalWordCount.java WordCount.java
<h2 id="run-wordcount">Run WordCount</h2>
-<p>A single Beam pipeline can run on multiple Beam <a href="/documentation#runners">runners</a>, including the <a href="/documentation/runners/spark">SparkRunner</a>, <a href="/documentation/runners/flink">FlinkRunner</a>, or <a href="/documentation/runners/dataflow">DataflowRunner</a>. The <a href="/documentation/runners/direct">DirectRunner</a> is a common runner for getting started, as it runs locally on your machine and requires no specific setup.</p>
+<p>A single Beam pipeline can run on multiple Beam <a href="/documentation#runners">runners</a>, including the <a href="/documentation/runners/apex">ApexRunner</a>, <a href="/documentation/runners/flink">FlinkRunner</a>, <a href="/documentation/runners/spark">SparkRunner</a> or <a href="/documentation/runners/dataflow">DataflowRunner</a>. The <a href="/documentation/runners/direct">DirectRunner</a> is a common runner for getting started, as it runs locally on your machine and requires no specific setup.</p>
<p>After you\u2019ve chosen which runner you\u2019d like to use:</p>
@@ -217,6 +217,11 @@ MinimalWordCount.java WordCount.java
</code></pre>
</div>
+<div class="runner-apex highlighter-rouge"><pre class="highlight"><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+ -Dexec.args="--inputFile=pom.xml --output=counts --runner=ApexRunner" -Papex-runner
+</code></pre>
+</div>
+
<div class="runner-flink highlighter-rouge"><pre class="highlight"><code>TODO BEAM-899
</code></pre>
</div>
@@ -239,6 +244,10 @@ MinimalWordCount.java WordCount.java
</code></pre>
</div>
+<div class="runner-apex highlighter-rouge"><pre class="highlight"><code>$ ls counts*
+</code></pre>
+</div>
+
<div class="runner-flink highlighter-rouge"><pre class="highlight"><code>TODO BEAM-899
</code></pre>
</div>
@@ -265,6 +274,16 @@ Foundation: 1
</code></pre>
</div>
+<div class="runner-apex highlighter-rouge"><pre class="highlight"><code>$ cat counts*
+BEAM: 1
+have: 1
+simple: 1
+skip: 4
+PAssert: 1
+...
+</code></pre>
+</div>
+
<div class="runner-flink highlighter-rouge"><pre class="highlight"><code>TODO BEAM-899
</code></pre>
</div>
http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/13d13604/content/index.html
----------------------------------------------------------------------
diff --git a/content/index.html b/content/index.html
index 3ded4c5..84e835c 100644
--- a/content/index.html
+++ b/content/index.html
@@ -148,7 +148,7 @@ The Apache Beam project is in the process of bootstrapping. This includes the we
<h1 id="apache-beam-incubating">Apache Beam (incubating)</h1>
-<p>Apache Beam is an open source, unified programming model that you can use to create a data processing <strong>pipeline</strong>. You start by building a program that defines the pipeline using one of the open source Beam SDKs. The pipeline is then executed by one of Beam\u2019s supported <strong>distributed processing back-ends</strong>, which include <a href="http://flink.apache.org">Apache Flink</a>, <a href="http://spark.apache.org">Apache Spark</a>, and <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>.</p>
+<p>Apache Beam is an open source, unified programming model that you can use to create a data processing <strong>pipeline</strong>. You start by building a program that defines the pipeline using one of the open source Beam SDKs. The pipeline is then executed by one of Beam\u2019s supported <strong>distributed processing back-ends</strong>, which include <a href="http://apex.apache.org">Apache Apex</a>, <a href="http://flink.apache.org">Apache Flink</a>, <a href="http://spark.apache.org">Apache Spark</a>, and <a href="https://cloud.google.com/dataflow">Google Cloud Dataflow</a>.</p>
<p>Beam is particularly useful for <a href="http://en.wikipedia.org/wiki/Embarassingly_parallel">Embarrassingly Parallel</a> data processing tasks, in which the problem can be decomposed into many smaller bundles of data that can be processed independently and in parallel. You can also use Beam for Extract, Transform, and Load (ETL) tasks and pure data integration. These tasks are useful for moving data between different storage media and data sources, transforming data into a more desirable format, or loading data onto a new system.</p>
@@ -200,6 +200,10 @@ The Apache Beam project is in the process of bootstrapping. This includes the we
<td>Apache Spark</td>
<td>In Development</td>
</tr>
+<tr>
+ <td>Apache Apex</td>
+ <td>In Development</td>
+</tr>
</table>
<p><strong>Note:</strong> You can always execute your pipeline locally for testing and debugging purposes.</p>
[3/3] incubator-beam-site git commit: This closes #78
Posted by da...@apache.org.
This closes #78
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/2bd19441
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/tree/2bd19441
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/diff/2bd19441
Branch: refs/heads/asf-site
Commit: 2bd1944147c95afc90537af7b5e5406a04eb0974
Parents: 41e5539 13d1360
Author: Davor Bonaci <da...@google.com>
Authored: Sun Nov 13 16:20:06 2016 -0800
Committer: Davor Bonaci <da...@google.com>
Committed: Sun Nov 13 16:20:06 2016 -0800
----------------------------------------------------------------------
content/documentation/runners/apex/index.html | 2 +-
.../runners/capability-matrix/index.html | 16 ++++++------
content/get-started/quickstart/index.html | 21 +++++++++++++++-
content/index.html | 6 ++++-
src/_data/capability-matrix.yml | 2 +-
src/documentation/runners/apex.md | 2 +-
src/get-started/quickstart.md | 26 ++++++++++++++++++--
src/index.md | 6 ++++-
8 files changed, 65 insertions(+), 16 deletions(-)
----------------------------------------------------------------------