[beam] branch spark-runner_structured-streaming updated (8a3f75c -> 7df039b)

[ec...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

echauchot pushed a change to branch spark-runner_structured-streaming
in repository https://gitbox.apache.org/repos/asf/beam.git.


    from 8a3f75c  [BEAM-8470] input windows can arrive exploded (for sliding windows). As a result an input has multiple windows. So we need to consider that the accumulator can have multiple windows
     add 9e4235d  [BEAM-8470] Add a combine test with sliding windows
     add 7ee091e  [temp] debug pardo print input and output in both spark runners and put spark in debug mode
     add f668e06  [temp] nexmark run in new spark runner
     add 1768142  Revert "[temp] debug pardo print input and output in both spark runners and put spark in debug mode"
     add 9c41f65  [BEAM-8470] Add a test to test combine translation on binaryCombineFn with sliding windows
     add 7df039b  [BEAM-8470] Fix tests: use correct parkStructuredStreamingPipelineOptions, set testMode to true. Some renaming

No new revisions were added by this update.

Summary of changes:
 .../StructuredStreamingPipelineStateTest.java      |  7 ++-
 .../aggregators/metrics/sink/InMemoryMetrics.java  |  3 ++
 .../metrics/sink/SparkMetricsSinkTest.java         | 20 ++++++-
 ...parkBeamMetricTest.java => BeamMetricTest.java} |  7 ++-
 ...tricsPusherTest.java => MetricsPusherTest.java} | 33 ++++++++----
 .../translation/batch/CombineTest.java             | 61 ++++++++++++++++++----
 .../translation/batch/ComplexSourceTest.java       | 14 ++---
 .../translation/batch/FlattenTest.java             | 16 +++---
 .../translation/batch/GroupByKeyTest.java          | 18 ++++---
 .../translation/batch/ParDoTest.java               | 36 +++++++------
 .../translation/batch/SimpleSourceTest.java        | 15 +++---
 .../translation/batch/WindowAssignTest.java        | 14 ++---
 .../translation/streaming/SimpleSourceTest.java    | 14 ++---
 .../apache/beam/sdk/nexmark/NexmarkLauncher.java   |  2 +
 .../apache/beam/sdk/nexmark/queries/Query5.java    |  2 +-
 15 files changed, 178 insertions(+), 84 deletions(-)
 rename runners/spark/src/test/java/org/apache/beam/runners/spark/structuredstreaming/metrics/{SparkBeamMetricTest.java => BeamMetricTest.java} (91%)
 rename runners/spark/src/test/java/org/apache/beam/runners/spark/structuredstreaming/metrics/{SparkMetricsPusherTest.java => MetricsPusherTest.java} (68%)