You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by mm...@apache.org on 2023/02/02 08:52:03 UTC

[beam] branch master updated (d7264d4ba2a -> 01aa470cf72)

This is an automated email from the ASF dual-hosted git repository.

mmack pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


    from d7264d4ba2a Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#25270)
     add 01aa470cf72 [Spark Runner] Add new experiment that provides concurrent bounded output for SDFs (resolves #23852) (#24837)

No new revisions were added by this update.

Summary of changes:
 CHANGES.md                                         |   2 +
 .../construction/SplittableParDoNaiveBounded.java  |   3 +-
 runners/spark/spark_runner.gradle                  |  41 ++-
 .../spark/translation/MultiDoFnFunction.java       |  66 ++--
 .../spark/translation/SparkInputDataProcessor.java | 374 +++++++++++++++++++++
 .../spark/translation/SparkProcessContext.java     | 149 ++------
 .../spark/translation/TransformTranslator.java     |  16 +-
 .../streaming/StreamingTransformTranslator.java    |   3 +-
 .../translation/SparkInputDataProcessorTest.java   | 278 +++++++++++++++
 9 files changed, 754 insertions(+), 178 deletions(-)
 create mode 100644 runners/spark/src/main/java/org/apache/beam/runners/spark/translation/SparkInputDataProcessor.java
 create mode 100644 runners/spark/src/test/java/org/apache/beam/runners/spark/translation/SparkInputDataProcessorTest.java