[jira] [Commented] (BEAM-8977) apache_beam.runners.interactive.display.pcoll_visualization_test.PCollectionVisualizationTest.test_dynamic_plotting_update_same_display is flaky

["Ning Kang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16998474#comment-16998474 ] 

Ning Kang commented on BEAM-8977:
---------------------------------

The flakiness is caused by indeterministic execution of asynchronous tasks to be tested.

Affected by the performance of the system executing the tests, it's possible that during the 1 second sleep, the asynchronous task is not executed at all.

However, adding a list length check, increasing the sleeping time or setting an await with timeout will not resolve the test flakiness issue.

If the asynchronous scheduled task is not executed for at least 2 iterations, there is nothing to be tested and a failure/success will not help with the tests.

I'll remove the flaky unit test since it's flaky in its nature. Mocking such asynchronous mechanism doesn't give us anything, so we will not test the asynchronous mechanism.

I'll add a different test for the underlying asynchronous task and test the logic synchronously.

 

 

 

> apache_beam.runners.interactive.display.pcoll_visualization_test.PCollectionVisualizationTest.test_dynamic_plotting_update_same_display is flaky
> ------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: BEAM-8977
>                 URL: https://issues.apache.org/jira/browse/BEAM-8977
>             Project: Beam
>          Issue Type: Bug
>          Components: test-failures
>            Reporter: Valentyn Tymofieiev
>            Assignee: Ning Kang
>            Priority: Major
>
> Sample failure: 
>  [https://builds.apache.org/job/beam_PreCommit_Python_Phrase/1273/testReport/apache_beam.runners.interactive.display.pcoll_visualization_test/PCollectionVisualizationTest/test_dynamic_plotting_update_same_display/]
> Error Message
>  IndexError: list index out of range
> Stacktrace
>  self = <apache_beam.runners.interactive.display.pcoll_visualization_test.PCollectionVisualizationTest testMethod=test_dynamic_plotting_update_same_display>
>  mocked_display_facets = <MagicMock name='display_facets' id='139889868386376'>
> @patch('apache_beam.runners.interactive.display.pcoll_visualization'
>  '.PCollectionVisualization.display_facets')
>  def test_dynamic_plotting_update_same_display(self,
>  mocked_display_facets):
>  fake_pipeline_result = runner.PipelineResult(runner.PipelineState.RUNNING)
>  ie.current_env().set_pipeline_result(self._p, fake_pipeline_result)
>  # Starts async dynamic plotting that never ends in this test.
>  h = pv.visualize(self._pcoll, dynamic_plotting_interval=0.001)
>  # Blocking so the above async task can execute some iterations.
>  time.sleep(1)
>  # The first iteration doesn't provide updating_pv to display_facets.
>  _, first_kwargs = mocked_display_facets.call_args_list[0]
>  self.assertEqual(first_kwargs, {})
>  # The following iterations use the same updating_pv to display_facets and so
>  # on.
>  > _, second_kwargs = mocked_display_facets.call_args_list[1]
>  E IndexError: list index out of range
> apache_beam/runners/interactive/display/pcoll_visualization_test.py:105: IndexError
> Standard Output
> <IPython.core.display.HTML object>
>  Standard Error
>  WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)