[beam] branch spark-runner_structured-streaming updated (eae506b -> 3506d6d)

[ec...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

echauchot pushed a change to branch spark-runner_structured-streaming
in repository https://gitbox.apache.org/repos/asf/beam.git.


    omit eae506b  [BEAM-8470] Fix wrong expected results in CombineTest.testBinaryCombineWithSlidingWindows
    omit ef0e870  [BEAM-8470] Fix tests: use correct SparkStructuredStreamingPipelineOptions, set testMode to true. Some renaming
    omit 6826025  [BEAM-8470] Add a test to test combine translation on binaryCombineFn with sliding windows
    omit f668e06  [temp] nexmark run in new spark runner
    omit 7ee091e  [temp] debug pardo print input and output in both spark runners and put spark in debug mode
     add c4f2575  [BEAM-8470] Add a test to test combine translation on binaryCombineFn with sliding windows
     add 2e048f9  [BEAM-8470] Fix tests: use correct SparkStructuredStreamingPipelineOptions, set testMode to true. Some renaming
     add fba2109  [BEAM-8470] Fix wrong expected results in CombineTest.testBinaryCombineWithSlidingWindows
     add a7d67b2  [BEAM-8470] Add disclaimers about this runner being experimental
     add 4a2c59d  [BEAM-8470] Fix: create an empty accumulator in combine.mergeAccumulators, because this method modifies its first input accumulator. Decrease memory usage by storing only accumulator and timestamp in the combine.merge map
     add d949942  [BEAM-8470] Apply spotless
     add 5a059d1  [BEAM-8470] Add a countPerElement test with sliding windows
     add 2e123ac  [BEAM-8470] Fix the output timestamps of combine: timestamps must be merged to a window before combining
     add 3506d6d  [BEAM-8470] set log level to info to avoid resource consumption in production mode

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (eae506b)
            \
             N -- N -- N   refs/heads/spark-runner_structured-streaming (3506d6d)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

No new revisions were added by this update.

Summary of changes:
 .../SparkStructuredStreamingRunner.java            | 19 +++--
 .../translation/batch/AggregatorCombiner.java      | 39 ++++++----
 .../translation/batch/ParDoTranslatorBatch.java    | 43 +----------
 .../spark/translation/TransformTranslator.java     | 21 +-----
 runners/spark/src/main/resources/log4j.properties  |  4 +-
 .../metrics/sink/SparkMetricsSinkTest.java         |  4 +-
 .../metrics/MetricsPusherTest.java                 |  6 +-
 .../translation/batch/CombineTest.java             | 84 +++++++++++++++-------
 .../translation/batch/ComplexSourceTest.java       |  4 +-
 .../translation/batch/FlattenTest.java             |  4 +-
 .../translation/batch/GroupByKeyTest.java          |  7 +-
 .../translation/batch/ParDoTest.java               | 19 +++--
 .../translation/batch/SimpleSourceTest.java        |  4 +-
 .../translation/batch/WindowAssignTest.java        |  7 +-
 .../translation/streaming/SimpleSourceTest.java    |  4 +-
 .../apache/beam/sdk/nexmark/NexmarkLauncher.java   |  2 -
 .../apache/beam/sdk/nexmark/queries/Query5.java    |  2 +-
 .../nexmark/src/main/resources/log4j.properties    |  2 +-
 18 files changed, 138 insertions(+), 137 deletions(-)