You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by peihe <gi...@git.apache.org> on 2017/11/09 02:56:13 UTC
[GitHub] beam pull request #4107: Merge branch 'master' into jstorm-runner at commit ...
GitHub user peihe opened a pull request:
https://github.com/apache/beam/pull/4107
Merge branch 'master' into jstorm-runner at commit 727253e
Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes.
- [ ] Each commit in the pull request should have a meaningful subject line and body.
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue.
- [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
- [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
---
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/peihe/incubator-beam jstorm-runner-merge
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/4107.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4107
----
commit 3a04dbe46debc2c4aa188b210870a24dc1408bf1
Author: Holden Karau <ho...@us.ibm.com>
Date: 2017-10-10T19:31:16Z
pr/cl-feedback from c-y-koo
commit 9de10e2d80ce15c5881b26f6436d28cccf60e18b
Author: Holden Karau <ho...@us.ibm.com>
Date: 2017-10-12T06:46:28Z
Fix snippets
commit 9c2a6e771d30e3e120b941bd2d450a00d8b7f091
Author: Robert Bradshaw <ro...@gmail.com>
Date: 2017-10-12T22:54:09Z
Closes #3804
commit b63a3eb082be24c74e1cb5f56b60c4c3dd38f728
Author: Scott Wegner <sw...@google.com>
Date: 2017-10-11T21:14:22Z
Increment dataflow client version
commit 6055d280a4ae1d9018e4644d671748341162af4b
Author: Thomas Groh <tg...@google.com>
Date: 2017-10-12T23:31:08Z
This closes #3980
commit 2d147eb21f98f5668ff5e2cc497b8d3e5fae9c20
Author: Luke Cwik <lc...@google.com>
Date: 2017-10-12T16:49:06Z
[BEAM-3048] Remove RAND_RANGE in WindowedWordCount
commit f398e5ad4abff386083c0af87cb29cd87891bbe4
Author: Luke Cwik <lc...@google.com>
Date: 2017-10-13T00:16:29Z
[BEAM-3048] Remove RAND_RANGE in WindowedWordCount
This closes #3987
commit 3feef91761c6f5a44f535e4daf9c39a88320e229
Author: Ahmet Altay <al...@google.com>
Date: 2017-10-13T02:17:28Z
Add an option for dataflow job labels.
commit a3a7807fec68a7954ba136dc00499649d372f5ca
Author: Ahmet Altay <al...@google.com>
Date: 2017-10-13T19:52:20Z
This closes #3990
commit ec192d15d3e83d6fe2127619c8bbd69e83277918
Author: Robert Bradshaw <ro...@google.com>
Date: 2017-10-04T20:57:01Z
Align names with those produced by the dataflow runner harness.
These will be unused once the runner harness produces the correct
transform payloads.
commit d91ebd9f5fa3cf5c250f02096c27c21354dce859
Author: Robert Bradshaw <ro...@google.com>
Date: 2017-10-05T00:33:07Z
Fix from any -> bytes transition.
commit 3dc75599437bc57a1d20a81f137f13b4f25b3719
Author: Robert Bradshaw <ro...@gmail.com>
Date: 2017-10-13T21:41:04Z
Closes #3941
commit d226c7679b9d94a40553609f31ecbfba72559e8a
Author: Robert Bradshaw <ro...@gmail.com>
Date: 2017-10-09T23:46:19Z
Add an element batching transform.
commit 7f5753f1f7f5321f19de51a4d320c1af2d2ad44f
Author: Robert Bradshaw <ro...@gmail.com>
Date: 2017-10-14T00:13:41Z
Closes #3971
commit 4b908c2e693fe9ed44fcb6c67a2d82c7da355259
Author: Eugene Kirpichov <ki...@google.com>
Date: 2017-09-25T20:57:04Z
Introduces Contextful
commit e2ad925dc4d8bb33a264a21c48b8ceef63ac6eb3
Author: Eugene Kirpichov <ki...@google.com>
Date: 2017-10-03T00:36:48Z
Supports side inputs in MapElements and FlatMapElements
commit 014614b695bac0b636aae662977dd3a3fa3b8a1e
Author: Eugene Kirpichov <ek...@gmail.com>
Date: 2017-10-14T01:44:28Z
This closes #3921: [BEAM-3009] Introduces Contextful machinery and uses it to add side input support to Watch
commit 3ad84791d4d85896f46b7956b5bd8045cdc4a0e9
Author: Robert Bradshaw <ro...@gmail.com>
Date: 2017-10-03T00:20:38Z
Add progress metrics to Python SDK.
commit 245d77338be8677df7ea9cbd75b1b8ab8c4da831
Author: Robert Bradshaw <ro...@google.com>
Date: 2017-10-16T20:09:31Z
Closes #3939
commit 6b25948b4f56e4d45e1f9e03eb19c6077413a80e
Author: chamikara@google.com <ch...@google.com>
Date: 2017-10-16T07:50:03Z
Sets user agent in BigTableIO.Read.getBigTableService().
commit ec052bb44ce40ae5fda2fbbd94fbc2b97ac363f5
Author: chamikara@google.com <ch...@google.com>
Date: 2017-10-16T20:14:25Z
This closes #3996
commit e940456bd95da3c8b79eb4666ad09280dccaedcf
Author: Kenneth Knowles <ke...@apache.org>
Date: 2017-10-16T22:13:26Z
Return null when timer not found instead of crashing
commit ec58a80ca0f913c85d5f17cba3535243cd010876
Author: Yunqing Zhou <zh...@zhouyunqing-macbookpro.roam.corp.google.com>
Date: 2017-10-13T22:52:18Z
Add custom tempLocation support to BigQueryIO.
commit a5cbd764bb81ca2aefe225d62e15e222a2266f0b
Author: Eugene Kirpichov <ki...@google.com>
Date: 2017-10-16T22:18:31Z
This closes #3995: Add custom gcs temp location support to BigQueryIO
commit 45c65c557bf79f8b4c5975eee15b5763e45882be
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-10-12T20:38:27Z
Set end-of-window timer in AfterWatermarkStateMachine
commit 3d36f63cbb6de5ce7115d34a1432835407cd666a
Author: Kenneth Knowles <ke...@apache.org>
Date: 2017-10-16T22:03:33Z
Do not set EOW timer in TriggerStateMachineTester
commit 64fb19da42e3d26d0a9dae41b19dd7bf77ff49c7
Author: Kenneth Knowles <ke...@apache.org>
Date: 2017-10-16T22:16:07Z
Allow checking timers set in TriggerStateMachineTester
commit 52863efd7478d406597b5b162c61701314155f25
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-10-16T22:20:27Z
This closes #4001: [BEAM-3052] Set end-of-window timer in AfterWatermarkStateMachine
Do not set EOW timer in TriggerStateMachineTester
Set end-of-window timer in AfterWatermarkStateMachine
Allow checking timers set in TriggerStateMachineTester
Return null when timer not found instead of crashing
commit c8f3a9183aa360cbd2b7e8132791a9fde82f11b7
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-10-16T23:12:26Z
This closes #3981: [BEAM-2878] Use bintray as default for container images
Add license to new file
[BEAM-2878] Use bintray as default container images
commit 6cdea08e06b52bb6a34f41dc6521bcfe8f6f83cb
Author: Luke Cwik <lc...@google.com>
Date: 2017-10-12T23:26:29Z
[BEAM-1487, BEAM-3016] Address termination correctness issues in BufferingStreamObserver & BeamFnLoggingClient
The issue with BeamFnLoggingClient is that we can't arriveAndDeregister during termination since
the onReadyHandler may also arrive at the same time which is why we swap to using forced termination.
Also, I added code that would guarantee that log messages produced by the thread which is shutting
down are guaranteed to make it (this was being caught occassionally by the testLogging test).
The BufferingStreamObserver was incorrectly shutting down since it may attempt to enqueue something
into a full queue with a reading thread that has already exitted for some reason so it would loop
forever attempting to insert the poison pill.
----
---