You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by markflyhigh <gi...@git.apache.org> on 2016/07/20 21:11:43 UTC
[GitHub] incubator-beam pull request #703: [BEAM-124] Flink and Spark running Example...
GitHub user markflyhigh opened a pull request:
https://github.com/apache/incubator-beam/pull/703
[BEAM-124] Flink and Spark running Examples WordCountIT
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [x] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [x] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [x] If this contribution is large, please file an Apache
[Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
---
+R: @dhalperi @jasonkuster
This PR is an updated version of an old [PR](https://github.com/apache/incubator-beam/pull/345).
Removes dependencies of Spark and Flink runner on Beam java example in order to run WordCountIT with Spark and Flink runner successfully. The following command is used for different runner:
- Spark runner:
`mvn clean verify -pl examples/java -am -rf :beam-examples-java -DskipITs=false -DintegrationTestPipelineOptions='[ "--tempRoot=/tmp", "--inputFile=/tmp/kinglear.txt", "--runner=org.apache.beam.runners.spark.SparkRunner" ]'`
- Flink runner:
`mvn clean verify -pl examples/java -am -rf :beam-examples-java -DskipITs=false -DintegrationTestPipelineOptions='[ "--tempRoot=gs://clouddfe-testing-temp-storage", "--runner=org.apache.beam.runners.flink.FlinkRunner" ]'`
- Dataflow test runner:
`mvn clean verify -pl examples/java -am -rf :beam-examples-java -DskipIT=false -DintegrationTestPipelineOptions='[ "--tempRoot=gs://clouddfe-testing-temp-storage", "--runner=org.apache.beam.runners.dataflow.testing.TestDataflowRunner" ]'`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/markflyhigh/incubator-beam wordcountit-spark-flink
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/703.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #703
----
commit 37753b0f3c7969ba6315c4326a958cc3c9bbbc50
Author: Mark Liu <ma...@markliu0.mtv.corp.google.com>
Date: 2016-07-20T00:37:50Z
[Beam-124] Run WorldCountIT with Spark and Flink runner
commit d8581c7b1bf8d14910d2f5c335caf25eba56427a
Author: Mark Liu <ma...@markliu0.mtv.corp.google.com>
Date: 2016-07-20T18:57:11Z
Remove depedency of Spark and Flink runner on Beam java examples
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-beam pull request #703: [BEAM-124] Flink and Spark running Example...
Posted by markflyhigh <gi...@git.apache.org>.
Github user markflyhigh closed the pull request at:
https://github.com/apache/incubator-beam/pull/703
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---