You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by ie...@apache.org on 2018/07/16 21:40:25 UTC

[beam] 01/01: Merge pull request #5940: [BEAM-4792] Adds a naive implementation of bounded SDFs to all runners

This is an automated email from the ASF dual-hosted git repository.

iemejia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 391f63d751fff2198729195e1caf155fabbbe0b7
Merge: 01d8b92 e1bdcac
Author: Ismaël Mejía <ie...@gmail.com>
AuthorDate: Mon Jul 16 23:40:16 2018 +0200

    Merge pull request #5940: [BEAM-4792] Adds a naive implementation of bounded SDFs to all runners

 runners/apex/build.gradle                          |   3 +-
 .../org/apache/beam/runners/apex/ApexRunner.java   |  39 +--
 .../core/construction/PTransformMatchers.java      |  51 +++
 .../runners/core/construction/SplittableParDo.java |  23 ++
 .../construction/SplittableParDoNaiveBounded.java  | 379 +++++++++++++++++++++
 .../core/SplittableParDoViaKeyedWorkItems.java     |   6 +-
 runners/flink/build.gradle                         |  28 +-
 .../flink/FlinkBatchTransformTranslators.java      |  16 +-
 .../flink/FlinkStreamingPipelineTranslator.java    |  33 --
 .../runners/flink/FlinkTransformOverrides.java     |  34 +-
 runners/gearpump/build.gradle                      |   3 +-
 .../translators/GearpumpPipelineTranslator.java    |   9 +
 .../beam/runners/dataflow/DataflowRunner.java      |  37 +-
 runners/samza/build.gradle                         |   3 +-
 .../samza/translation/SamzaTransformOverrides.java |   9 +
 .../runtime/SamzaStoreStateInternalsTest.java      |   5 +-
 runners/spark/build.gradle                         |   2 +-
 .../org/apache/beam/runners/spark/SparkRunner.java |  18 +-
 .../runners/spark/SparkTransformOverrides.java}    |  25 +-
 .../spark/translation/MultiDoFnFunction.java       |   9 +-
 .../spark/translation/SparkProcessContext.java     |   2 -
 .../spark/translation/TransformTranslator.java     |   6 +-
 .../spark/translation/TranslationUtils.java        |   9 -
 .../streaming/StreamingTransformTranslator.java    |   7 +-
 ...eParDo.java => UsesBoundedSplittableParDo.java} |   8 +-
 ...arDo.java => UsesUnboundedSplittableParDo.java} |   8 +-
 .../java/org/apache/beam/sdk/io/FileIOTest.java    |   3 +-
 .../org/apache/beam/sdk/io/TextIOReadTest.java     |   3 +-
 .../beam/sdk/transforms/SplittableDoFnTest.java    | 320 +++++++++++++----
 .../org/apache/beam/sdk/transforms/WatchTest.java  |  17 +-
 30 files changed, 858 insertions(+), 257 deletions(-)