You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by XuMingmin <gi...@git.apache.org> on 2017/07/20 17:11:09 UTC
[GitHub] beam pull request #3603: redo PR 3553
GitHub user XuMingmin opened a pull request:
https://github.com/apache/beam/pull/3603
redo PR 3553
Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes.
- [ ] Each commit in the pull request should have a meaningful subject line and body.
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue.
- [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
- [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
---
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/beam master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/3603.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3603
----
commit 8cab15338f811f880c6cfb820051cf355f92986b
Author: Robert Bradshaw <ro...@google.com>
Date: 2017-06-22T01:09:48Z
Java Dataflow runner harness compatibility.
commit 2a5520018fb25f96b57e52231a8cc0b191abfce0
Author: Robert Bradshaw <ro...@gmail.com>
Date: 2017-06-22T19:34:27Z
Closes #3418
commit 7471e2736cc22336500f6252ab8448889a2d04d3
Author: Charles Chen <cc...@google.com>
Date: 2017-06-22T18:29:54Z
Clean up test_stream_test and remove stray print statement
commit 3dc454a9bab4683efea080a1cd5484f0838561d3
Author: Ahmet Altay <al...@google.com>
Date: 2017-06-22T20:17:50Z
This closes #3423
commit 0292a24f9c88796542bff55031d84c11f0ab6b16
Author: Colin Phipps <fi...@google.com>
Date: 2017-05-15T14:18:16Z
[BEAM-2439] Dynamic sizing of Datastore write RPCs
This stops the Datastore connector from always sending 500 entities per RPC.
Instead, it starts at a lower number which is more likely to complete within
the deadline even in adverse conditions, and then increases or reduces the
batch size in response to measured latency of past requests.
commit 9ed0af8f2c5dabf3af5d664c473a49166b96d7d2
Author: Luke Cwik <lc...@google.com>
Date: 2017-06-22T20:34:12Z
[BEAM-2439] Dynamic sizing of Datastore write RPCs
This closes #3390
commit c05764454e73ab93d0602f34b8b7622d46e1d892
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-06-22T03:58:35Z
DataflowRunner: Reject SetState and MapState
commit 5506be87dc3b8f4c4425a259cf255a168fd7b127
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-06-22T20:51:46Z
This closes #3420: DataflowRunner: Reject SetState and MapState
commit 497cfabea7d6dcee0c5d327022678c571c3ec487
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-06-22T18:31:28Z
Add window matcher for pane info
commit d4e5db51a025a831ddf4e3bc0e003caebabf647b
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-06-22T18:56:53Z
Tidy LateDataDroppingDoFnRunner
commit 1c1f239501349f5120b0d619c4eea9c435500b78
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-06-22T19:52:42Z
ReduceFnTester can advance clocks without firing timers
commit 795760d370bcbe28e1f0ca373ad4c8c841e6e6b5
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-06-22T19:53:15Z
ReduceFnTester assertion for windows that have data buffered
commit 412fd7eab9e58a4d412f4dff5ffec023610b4f22
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-06-22T19:56:14Z
Drop late data in ReduceFnTester
commit 50c43d96adb8c2523cf38c09f32e241eacc47823
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-06-22T19:56:34Z
Do not GC windows based on processing time timer!
commit fda589c00c8920e76cfc9aaa87cecfa94077599d
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-06-22T20:04:23Z
Add test reproducing BEAM-2505, ignored
commit d2b384a20dbb0213d0f63e74713a06d63bad8d39
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-06-22T20:05:42Z
Add tests for corner cases of processing time timers
commit 7645c44b9bae9a77192fbe6744918cfb95ca1a3a
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-06-22T22:41:02Z
This closes #3425: Fix processing time timer handling in ReduceFnRunner
Add tests for corner cases of processing time timers
Add test reproducing BEAM-2505, ignored
Do not GC windows based on processing time timer!
Drop late data in ReduceFnTester
ReduceFnTester assertion for windows that have data buffered
ReduceFnTester can advance clocks without firing timers
Tidy LateDataDroppingDoFnRunner
Add window matcher for pane info
commit 5d6ad19958d0a2394f9e33720a04cc954279a7e7
Author: Robert Bradshaw <ro...@gmail.com>
Date: 2017-06-22T19:44:23Z
Remove fn api bundle descriptor translation.
commit a882e8f3a33c4a430f55d53b65285123c5a4f50d
Author: Robert Bradshaw <ro...@gmail.com>
Date: 2017-06-22T19:46:13Z
Remove unused (and untested) initial splittling logic.
commit 7a9f762e440168d48e3b3b2930dd83885402e51a
Author: Robert Bradshaw <ro...@gmail.com>
Date: 2017-06-23T00:05:32Z
Closes #3424
commit 799173fac4e07dab4547ba21971922336cd72c62
Author: Eugene Kirpichov <ki...@google.com>
Date: 2017-06-21T22:21:32Z
Uses KV in SplittableParDo expansion instead of ElementAndRestriction
This is a workaround for the following issue.
ElementAndRestriction is in runners-core, which may be shaded by runners
(and is shaded by Dataflow runner), hence it should be *both* produced
and consumed by workers - but currently it's produced by (shaded)
SplittableParDo and consumed by (differently shaded) ProcessFn in the
runner's worker code.
There are several ways out of this, e.g. moving EAR into the SDK (icky
because it's an implementation detail of SplittableParDo), or using
a type that's already in the SDK. There may be other more complicated
ways too.
commit 6f12e7d3d6a9fbd0d7bc1a6136542bd503cb0f2b
Author: Eugene Kirpichov <ki...@google.com>
Date: 2017-06-22T21:59:36Z
Bump Dataflow worker to 0622
commit 8860cceb710f28931fe5ec792b0db09277f0d2a9
Author: Eugene Kirpichov <ki...@google.com>
Date: 2017-06-23T00:56:44Z
This closes #3417: [BEAM-1377] Uses KV in SplittableParDo expansion instead of ElementAndRestriction
commit cab4d8969e7f95b0ece59838ad2d578e75d38823
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-06-22T03:25:31Z
DataflowRunner: Reject merging windowing for stateful ParDo
commit 5f8cfa7412e8f4ce97d6df4413fecbea22b4b5e1
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-06-23T03:05:28Z
This closes #3419: DataflowRunner: Reject merging windowing for stateful ParDo
commit 649994b353afe28c917969609c7a1a47a4f39aaf
Author: Rune Fevang <fe...@exabel.com>
Date: 2017-06-15T11:51:12Z
Allow output from FinishBundle in DoFnTester
commit b3c36256e32f4a4a616d0e152686d29f1174c5cd
Author: Kenneth Knowles <kl...@google.com>
Date: 2017-06-23T03:07:48Z
This closes #3367: Allow output from FinishBundle in DoFnTester
commit 8dcda6e40355af13f4d92fcd44aae4539a225a4a
Author: Sourabh Bajaj <so...@google.com>
Date: 2017-06-23T00:08:20Z
[BEAM-2497] Fix the reading of concat gzip files
commit 336b7f1cffc8709cd480a243a16de95bce39b53e
Author: Chamikara Jayalath <ch...@google.com>
Date: 2017-06-23T05:13:05Z
This closes #3428
commit f291713b28e3ba0246a8c0a710c71506cd0a0f91
Author: Etienne Chauchot <ec...@gmail.com>
Date: 2017-06-21T08:39:39Z
[BEAM-2489] Use dynamic ES port in HIFIOWithElasticTest
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] beam pull request #3603: redo PR 3553
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/beam/pull/3603
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---