You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by markflyhigh <gi...@git.apache.org> on 2016/07/20 21:11:43 UTC

[GitHub] incubator-beam pull request #703: [BEAM-124] Flink and Spark running Example...

GitHub user markflyhigh opened a pull request:

    https://github.com/apache/incubator-beam/pull/703

    [BEAM-124] Flink and Spark running Examples WordCountIT

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
    
     - [x] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [x] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [x] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
    
    ---
    
    +R: @dhalperi @jasonkuster
    
    This PR is an updated version of an old [PR](https://github.com/apache/incubator-beam/pull/345).
    
    Removes dependencies of Spark and Flink runner on Beam java example in order to run WordCountIT with Spark and Flink runner successfully. The following command is used for different runner:
    
    - Spark runner:
    
    `mvn clean verify -pl examples/java -am -rf :beam-examples-java -DskipITs=false -DintegrationTestPipelineOptions='[ "--tempRoot=/tmp", "--inputFile=/tmp/kinglear.txt", "--runner=org.apache.beam.runners.spark.SparkRunner" ]'`
    
    - Flink runner:
    
    `mvn clean verify -pl examples/java -am -rf :beam-examples-java -DskipITs=false -DintegrationTestPipelineOptions='[ "--tempRoot=gs://clouddfe-testing-temp-storage", "--runner=org.apache.beam.runners.flink.FlinkRunner" ]'`
    
    - Dataflow test runner:
    
    `mvn clean verify -pl examples/java -am -rf :beam-examples-java -DskipIT=false -DintegrationTestPipelineOptions='[ "--tempRoot=gs://clouddfe-testing-temp-storage", "--runner=org.apache.beam.runners.dataflow.testing.TestDataflowRunner" ]'`
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/markflyhigh/incubator-beam wordcountit-spark-flink

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/703.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #703
    
----
commit 37753b0f3c7969ba6315c4326a958cc3c9bbbc50
Author: Mark Liu <ma...@markliu0.mtv.corp.google.com>
Date:   2016-07-20T00:37:50Z

    [Beam-124] Run WorldCountIT with Spark and Flink runner

commit d8581c7b1bf8d14910d2f5c335caf25eba56427a
Author: Mark Liu <ma...@markliu0.mtv.corp.google.com>
Date:   2016-07-20T18:57:11Z

    Remove depedency of Spark and Flink runner on Beam java examples

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #703: [BEAM-124] Flink and Spark running Example...

Posted by markflyhigh <gi...@git.apache.org>.
Github user markflyhigh closed the pull request at:

    https://github.com/apache/incubator-beam/pull/703


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---