You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Anna Qin (Jira)" <ji...@apache.org> on 2020/08/21 02:24:00 UTC
[jira] [Updated] (BEAM-10784) Spark portable streaming: failing
window-related tests
[ https://issues.apache.org/jira/browse/BEAM-10784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anna Qin updated BEAM-10784:
----------------------------
Description:
When running validatesPortableRunnerStreaming on Spark, the following tests fail. They all seem to be related to windowing. The underlying issue may have to do with incorrect handling of watermarks (they may not be getting held back properly).
Fails from timeout:
PAssertTest. testWindowedContainsInAnyOrder
PAssertTest. testWindowedSerializablePredicate
WindowTest. testNoWindowFnDoesNotReassignWindows
Fails from incorrect assertion:
CombineTest$BasicTests. testHotKeyCombining
Fails from empty assertion ("Expected: iterable with items [*] in any order
but: no item matches: * in []"):
CombineTest$WindowingTests. testFixedWindowsCombine
CombineTest$WindowingTests. testSessionsCombine
ReshuffleTest. testReshuffleAfterFixedWindowsAndGroupByKey
ReshuffleTest. testReshuffleAfterSessionsAndGroupByKey
ReshuffleTest. testReshuffleAfterSlidingWindowsAndGroupByKey
CoGroupByKeyTest. testCoGroupByKeyWithWindowing
WindowingTest. testMergingWindowing
WindowingTest. testNonPartitioningWindowing
WindowingTest. testPartitioningWindowing
WindowingTest. testWindowPreservation
was:
When running validatesPortableRunnerStreaming on Spark, the following tests fail. They all seem to be related to windowing. The underlying issue may have to do with incorrect handling of watermarks (they may not be getting held back properly?).
Fails from timeout:
PAssertTest. testWindowedContainsInAnyOrder
PAssertTest. testWindowedSerializablePredicate
WindowTest. testNoWindowFnDoesNotReassignWindows
Fails from incorrect assertion:
CombineTest$BasicTests. testHotKeyCombining
Fails from empty assertion ("Expected: iterable with items [*] in any order
but: no item matches: * in []"):
CombineTest$WindowingTests. testFixedWindowsCombine
CombineTest$WindowingTests. testSessionsCombine
ReshuffleTest. testReshuffleAfterFixedWindowsAndGroupByKey
ReshuffleTest. testReshuffleAfterSessionsAndGroupByKey
ReshuffleTest. testReshuffleAfterSlidingWindowsAndGroupByKey
CoGroupByKeyTest. testCoGroupByKeyWithWindowing
WindowingTest. testMergingWindowing
WindowingTest. testNonPartitioningWindowing
WindowingTest. testPartitioningWindowing
WindowingTest. testWindowPreservation
> Spark portable streaming: failing window-related tests
> -------------------------------------------------------
>
> Key: BEAM-10784
> URL: https://issues.apache.org/jira/browse/BEAM-10784
> Project: Beam
> Issue Type: Bug
> Components: runner-spark
> Reporter: Anna Qin
> Priority: P2
> Labels: portability-spark
>
> When running validatesPortableRunnerStreaming on Spark, the following tests fail. They all seem to be related to windowing. The underlying issue may have to do with incorrect handling of watermarks (they may not be getting held back properly).
> Fails from timeout:
> PAssertTest. testWindowedContainsInAnyOrder
> PAssertTest. testWindowedSerializablePredicate
> WindowTest. testNoWindowFnDoesNotReassignWindows
> Fails from incorrect assertion:
> CombineTest$BasicTests. testHotKeyCombining
> Fails from empty assertion ("Expected: iterable with items [*] in any order
> but: no item matches: * in []"):
> CombineTest$WindowingTests. testFixedWindowsCombine
> CombineTest$WindowingTests. testSessionsCombine
> ReshuffleTest. testReshuffleAfterFixedWindowsAndGroupByKey
> ReshuffleTest. testReshuffleAfterSessionsAndGroupByKey
> ReshuffleTest. testReshuffleAfterSlidingWindowsAndGroupByKey
> CoGroupByKeyTest. testCoGroupByKeyWithWindowing
> WindowingTest. testMergingWindowing
> WindowingTest. testNonPartitioningWindowing
> WindowingTest. testPartitioningWindowing
> WindowingTest. testWindowPreservation
--
This message was sent by Atlassian Jira
(v8.3.4#803005)