[GitHub] beam pull request #2154: [BEAM-1620] WIP: maven config for streaming RoS in ...

[kennknowles <gi...@git.apache.org>]

GitHub user kennknowles opened a pull request:

    https://github.com/apache/beam/pull/2154

    [BEAM-1620] WIP: maven config for streaming RoS in Dataflow

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
    
     - [ ] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [ ] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
    
    ---
    
    @jasonkuster this is essentially an inlining of the configuration for batch. Notes:
    
     - I'd love to just provide `-D beamTestPipelineOptions=...` and have things automatically just add `--streaming=true` and `--streaming=false` but don't know that you can do that effectively in a POM.
     - Setting `parallel=all` and `threadCount=4` I did manage to get a 429, which surprised me. TBD why that happens; something is not blocking when it should and/or leaking jobs.
    
    CC @davorbonaci in case you have any vision for the right way for this config to get into the project.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kennknowles/beam streaming-RoS

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/beam/pull/2154.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2154
    
----
commit 32650981f11167756d2ebe8fa5f6531e089970fa
Author: Kenneth Knowles <kl...@google.com>
Date:   2017-01-24T04:30:04Z

    WIP: maven config for streaming RoS in Dataflow

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #2154: [BEAM-1620] Maven config for streaming ValidatesRun...

[kennknowles <gi...@git.apache.org>]

Github user kennknowles closed the pull request at:

    https://github.com/apache/beam/pull/2154


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---