You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by mm...@apache.org on 2023/02/03 11:42:38 UTC

[beam] branch master updated (d50924d2ae9 -> 72781cb089a)

This is an automated email from the ASF dual-hosted git repository.

mmack pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


    from d50924d2ae9 [Spark Dataset runner] Fix collection encoder bug that may lead to corrupted data due to naive usage of unsafe storage (fixes #25296). (#25297)
     add 72781cb089a [Spark Dataset runner] Break linage of dataset to reduce Spark planning overhead in case of large complex query plans (relates to #24710 and #23845) (#25187)

No new revisions were added by this update.

Summary of changes:
 .../translation/EvaluationContext.java             | 10 +--
 .../translation/PipelineTranslator.java            | 97 ++++++++++++++++++----
 .../translation/TransformTranslator.java           | 13 ++-
 .../batch/CombineGloballyTranslatorBatch.java      |  4 +
 .../batch/CombineGroupedValuesTranslatorBatch.java |  4 +
 .../batch/CombinePerKeyTranslatorBatch.java        |  4 +
 .../translation/batch/FlattenTranslatorBatch.java  |  4 +
 .../batch/GroupByKeyTranslatorBatch.java           |  5 +-
 .../translation/batch/ImpulseTranslatorBatch.java  |  4 +
 .../translation/batch/ParDoTranslatorBatch.java    |  6 +-
 .../batch/ReadSourceTranslatorBatch.java           |  7 +-
 .../batch/ReshuffleTranslatorBatch.java            |  8 ++
 .../batch/WindowAssignTranslatorBatch.java         |  4 +
 13 files changed, 144 insertions(+), 26 deletions(-)