You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by ie...@apache.org on 2018/07/16 21:40:25 UTC
[beam] 01/01: Merge pull request #5940: [BEAM-4792] Adds a naive
implementation of bounded SDFs to all runners
This is an automated email from the ASF dual-hosted git repository.
iemejia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 391f63d751fff2198729195e1caf155fabbbe0b7
Merge: 01d8b92 e1bdcac
Author: Ismaël Mejía <ie...@gmail.com>
AuthorDate: Mon Jul 16 23:40:16 2018 +0200
Merge pull request #5940: [BEAM-4792] Adds a naive implementation of bounded SDFs to all runners
runners/apex/build.gradle | 3 +-
.../org/apache/beam/runners/apex/ApexRunner.java | 39 +--
.../core/construction/PTransformMatchers.java | 51 +++
.../runners/core/construction/SplittableParDo.java | 23 ++
.../construction/SplittableParDoNaiveBounded.java | 379 +++++++++++++++++++++
.../core/SplittableParDoViaKeyedWorkItems.java | 6 +-
runners/flink/build.gradle | 28 +-
.../flink/FlinkBatchTransformTranslators.java | 16 +-
.../flink/FlinkStreamingPipelineTranslator.java | 33 --
.../runners/flink/FlinkTransformOverrides.java | 34 +-
runners/gearpump/build.gradle | 3 +-
.../translators/GearpumpPipelineTranslator.java | 9 +
.../beam/runners/dataflow/DataflowRunner.java | 37 +-
runners/samza/build.gradle | 3 +-
.../samza/translation/SamzaTransformOverrides.java | 9 +
.../runtime/SamzaStoreStateInternalsTest.java | 5 +-
runners/spark/build.gradle | 2 +-
.../org/apache/beam/runners/spark/SparkRunner.java | 18 +-
.../runners/spark/SparkTransformOverrides.java} | 25 +-
.../spark/translation/MultiDoFnFunction.java | 9 +-
.../spark/translation/SparkProcessContext.java | 2 -
.../spark/translation/TransformTranslator.java | 6 +-
.../spark/translation/TranslationUtils.java | 9 -
.../streaming/StreamingTransformTranslator.java | 7 +-
...eParDo.java => UsesBoundedSplittableParDo.java} | 8 +-
...arDo.java => UsesUnboundedSplittableParDo.java} | 8 +-
.../java/org/apache/beam/sdk/io/FileIOTest.java | 3 +-
.../org/apache/beam/sdk/io/TextIOReadTest.java | 3 +-
.../beam/sdk/transforms/SplittableDoFnTest.java | 320 +++++++++++++----
.../org/apache/beam/sdk/transforms/WatchTest.java | 17 +-
30 files changed, 858 insertions(+), 257 deletions(-)