You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by ke...@apache.org on 2016/06/13 18:16:53 UTC
[1/2] incubator-beam git commit: [BEAM-336] update examples-java
README
Repository: incubator-beam
Updated Branches:
refs/heads/master dbddb9f3c -> 9e0d7d650
[BEAM-336] update examples-java README
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/99c596eb
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/99c596eb
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/99c596eb
Branch: refs/heads/master
Commit: 99c596ebcfa481c043c3f26b8085ddb4f0867ef5
Parents: 60964b6
Author: manuzhang <ow...@gmail.com>
Authored: Mon Jun 13 11:09:38 2016 +0800
Committer: manuzhang <ow...@gmail.com>
Committed: Mon Jun 13 11:09:38 2016 +0800
----------------------------------------------------------------------
examples/java/README.md | 38 +++++++++++++++++++-------------------
1 file changed, 19 insertions(+), 19 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/99c596eb/examples/java/README.md
----------------------------------------------------------------------
diff --git a/examples/java/README.md b/examples/java/README.md
index 4694103..ef3cf07 100644
--- a/examples/java/README.md
+++ b/examples/java/README.md
@@ -26,42 +26,42 @@ the development of more complex pipelines.
## Word Count
A good starting point for new users is our set of
-[word count](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/examples/src/main/java/com/google/cloud/dataflow/examples) examples, which computes word frequencies. This series of four successively more detailed pipelines is described in detail in the accompanying [walkthrough](https://cloud.google.com/dataflow/examples/wordcount-example).
+[word count](https://github.com/apache/incubator-beam/blob/master/examples/java/src/main/java/org/apache/beam/examples) examples, which computes word frequencies. This series of four successively more detailed pipelines is described in detail in the accompanying [walkthrough](https://cloud.google.com/dataflow/examples/wordcount-example).
-1. [`MinimalWordCount`](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/examples/src/main/java/com/google/cloud/dataflow/examples/MinimalWordCount.java) is the simplest word count pipeline and introduces basic concepts like [Pipelines](https://cloud.google.com/dataflow/model/pipelines),
+1. [`MinimalWordCount`](https://github.com/apache/incubator-beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java) is the simplest word count pipeline and introduces basic concepts like [Pipelines](https://cloud.google.com/dataflow/model/pipelines),
[PCollections](https://cloud.google.com/dataflow/model/pcollection),
[ParDo](https://cloud.google.com/dataflow/model/par-do),
and [reading and writing data](https://cloud.google.com/dataflow/model/reading-and-writing-data) from external storage.
-1. [`WordCount`](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/examples/src/main/java/com/google/cloud/dataflow/examples/WordCount.java) introduces Dataflow best practices like [PipelineOptions](https://cloud.google.com/dataflow/pipelines/constructing-your-pipeline#Creating) and custom [PTransforms](https://cloud.google.com/dataflow/model/composite-transforms).
+1. [`WordCount`](https://github.com/apache/incubator-beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/WordCount.java) introduces Dataflow best practices like [PipelineOptions](https://cloud.google.com/dataflow/pipelines/constructing-your-pipeline#Creating) and custom [PTransforms](https://cloud.google.com/dataflow/model/composite-transforms).
-1. [`DebuggingWordCount`](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/examples/src/main/java/com/google/cloud/dataflow/examples/DebuggingWordCount.java)
+1. [`DebuggingWordCount`](https://github.com/apache/incubator-beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/DebuggingWordCount.java)
shows how to view live aggregators in the [Dataflow Monitoring Interface](https://cloud.google.com/dataflow/pipelines/dataflow-monitoring-intf), get the most out of
[Cloud Logging](https://cloud.google.com/dataflow/pipelines/logging) integration, and start writing
[good tests](https://cloud.google.com/dataflow/pipelines/testing-your-pipeline).
-1. [`WindowedWordCount`](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/examples/src/main/java/com/google/cloud/dataflow/examples/WindowedWordCount.java) shows how to run the same pipeline over either unbounded PCollections in streaming mode or bounded PCollections in batch mode.
+1. [`WindowedWordCount`](https://github.com/apache/incubator-beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/WindowedWordCount.java) shows how to run the same pipeline over either unbounded PCollections in streaming mode or bounded PCollections in batch mode.
## Building and Running
-The examples in this repository can be built and executed from the root directory by running:
+Change directory into `examples/java` and run the examples:
- mvn compile exec:java -pl examples \
+ mvn compile exec:java \
-Dexec.mainClass=<MAIN CLASS> \
-Dexec.args="<EXAMPLE-SPECIFIC ARGUMENTS>"
For example, you can execute the `WordCount` pipeline on your local machine as follows:
- mvn compile exec:java -pl examples \
- -Dexec.mainClass=com.google.cloud.dataflow.examples.WordCount \
+ mvn compile exec:java \
+ -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args="--inputFile=<LOCAL INPUT FILE> --output=<LOCAL OUTPUT FILE>"
Once you have followed the general Cloud Dataflow
[Getting Started](https://cloud.google.com/dataflow/getting-started) instructions, you can execute
the same pipeline on fully managed resources in Google Cloud Platform:
- mvn compile exec:java -pl examples \
- -Dexec.mainClass=com.google.cloud.dataflow.examples.WordCount \
+ mvn compile exec:java \
+ -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args="--project=<YOUR CLOUD PLATFORM PROJECT ID> \
--tempLocation=<YOUR CLOUD STORAGE LOCATION> \
--runner=BlockingDataflowPipelineRunner"
@@ -78,18 +78,18 @@ Platform:
mvn package
- java -cp examples/target/google-cloud-dataflow-java-examples-all-bundled-<VERSION>.jar \
- com.google.cloud.dataflow.examples.WordCount \
+ java -cp examples/java/target/beam-examples-java-bundled-<VERSION>.jar \
+ org.apache.beam.examples.WordCount \
--inputFile=<INPUT FILE PATTERN> --output=<OUTPUT FILE>
- java -cp examples/target/google-cloud-dataflow-java-examples-all-bundled-<VERSION>.jar \
- com.google.cloud.dataflow.examples.WordCount \
+ java -cp examples/java/target/beam-examples-java-bundled-<VERSION>.jar \
+ org.apache.beam.examples.WordCount \
--project=<YOUR CLOUD PLATFORM PROJECT ID> \
--tempLocation=<YOUR CLOUD STORAGE LOCATION> \
--runner=BlockingDataflowPipelineRunner
Other examples can be run similarly by replacing the `WordCount` class path with the example classpath, e.g.
-`com.google.cloud.dataflow.examples.cookbook.BigQueryTornadoes`,
+`org.apache.beam.examples.cookbook.BigQueryTornadoes`,
and adjusting runtime options under the `Dexec.args` parameter, as specified in
the example itself.
@@ -99,14 +99,14 @@ example, input file pattern of `c:\*.txt` should be entered as `c:\\*.txt`.
## Beyond Word Count
-After you've finished running your first few word count pipelines, take a look at the [`cookbook`](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/examples/src/main/java/com/google/cloud/dataflow/examples/cookbook)
+After you've finished running your first few word count pipelines, take a look at the [cookbook](https://github.com/apache/incubator-beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/cookbook)
directory for some common and useful patterns like joining, filtering, and combining.
-The [`complete`](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/blob/master/examples/src/main/java/com/google/cloud/dataflow/examples/complete)
+The [complete](https://github.com/apache/incubator-beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/complete)
directory contains a few realistic end-to-end pipelines.
See the
-[Java 8](https://github.com/GoogleCloudPlatform/DataflowJavaSDK/tree/master/examples/src/main/java8/com/google/cloud/dataflow/examples)
+[Java 8](https://github.com/apache/incubator-beam/tree/master/examples/java8/src/main/java/org/apache/beam/examples)
examples as well. This directory includes a Java 8 version of the
MinimalWordCount example, as well as series of examples in a simple 'mobile
gaming' domain. This series introduces some advanced concepts and provides
[2/2] incubator-beam git commit: This closes #449
Posted by ke...@apache.org.
This closes #449
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9e0d7d65
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9e0d7d65
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/9e0d7d65
Branch: refs/heads/master
Commit: 9e0d7d650a04a63c7e9b392f938990cea90b108e
Parents: dbddb9f 99c596e
Author: Kenneth Knowles <kl...@google.com>
Authored: Mon Jun 13 11:16:46 2016 -0700
Committer: Kenneth Knowles <kl...@google.com>
Committed: Mon Jun 13 11:16:46 2016 -0700
----------------------------------------------------------------------
examples/java/README.md | 38 +++++++++++++++++++-------------------
1 file changed, 19 insertions(+), 19 deletions(-)
----------------------------------------------------------------------